# Open challenges in LLM research

DevFeed: [Open challenges in LLM research](<https://devfeed.tech/articles/open-challenges-in-llm-research-31794.md>)

Original publisher: [Read original article](<https://huyenchip.com//2023/08/16/llm-research-open-challenges.html>)

Author: Chip Huyen

Published: 2023-08-16T00:00:00Z

Content type: opinion

Language: en

Sources: [Chip Huyen](<https://devfeed.tech/sources/chip-huyen.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Hallucination detection](<https://devfeed.tech/topics/hallucination-detection.md>), [context](<https://devfeed.tech/topics/context.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [context](<https://devfeed.tech/tags/context.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hallucination-detection](<https://devfeed.tech/tags/hallucination-detection.md>), [llm](<https://devfeed.tech/tags/llm.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [research](<https://devfeed.tech/tags/research.md>)

## AI overview

This article surveys ten research directions aimed at improving large language models. The supplied excerpt focuses on reducing and measuring hallucinations, optimizing context length and construction, and briefly identifies multimodality, new architectures, and GPU alternatives as areas of interest.

## Source excerpt

[LinkedIn discussion, Twitter thread] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I'm the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I'll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I've seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity (Bang et al., 2023) Contrastive Learning Reduces Hallucination in Conversations (Sun et al., 2022) Self-Consistency Improves Chain of Thought Reasoning in Language Models (Wang et al., 2022) SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (Manakul et al., 2023) A simple example of fact-checking and hallucination by NVIDIA's NeMo-Guardrails 2. Optimize context length and context construction A vast ma