AI Demystified: Your Simple Guide to LLMs, Hallucinations & Core Concepts

The realm of artificial intelligence is intricate and multifaceted, often characterized by specialized terminology employed by its researchers. Consequently, our reporting on the AI industry frequently necessitates the use of these technical terms. To enhance clarity and understanding for our audience, we have compiled a glossary defining some of the most crucial words and phrases encountered in our articles.

This glossary will be regularly updated with new entries as researchers persistently develop innovative methods to advance the frontiers of artificial intelligence, concurrently identifying emerging safety considerations.

Artificial General Intelligence, or AGI, remains a somewhat ambiguous concept. However, it generally refers to AI systems that surpass the capabilities of an average human across numerous, if not most, tasks. OpenAI CEO Sam Altman recently characterized AGI as the “equivalent of a median human that you could hire as a co-worker.” In contrast, OpenAI’s charter defines AGI as “highly autonomous systems that outperform humans at most economically valuable work.” Google DeepMind offers a slightly different perspective, viewing AGI as “AI that’s at least as capable as humans at most cognitive tasks.” If these distinctions seem confusing, rest assured that even leading AI experts share this sentiment.

An AI agent denotes a sophisticated tool that leverages AI technologies to autonomously perform a sequence of tasks on your behalf—extending beyond the basic functions of a chatbot. Examples include managing expenses, booking travel or restaurant reservations, or even writing and maintaining code. As previously elaborated, this nascent field is dynamic, meaning the interpretation of “AI agent” can vary. Furthermore, the necessary infrastructure to realize its full potential is still under development. Nevertheless, the fundamental principle involves an autonomous system capable of utilizing multiple AI components to execute multi-step operations.

A human brain can effortlessly answer a simple query like “which animal is taller, a giraffe or a cat?” without much deliberation. Yet, for more complex problems, intermediary steps often require external aids like pen and paper. For instance, determining the number of chickens and cows a farmer has, given a total of 40 heads and 120 legs, typically involves setting up a simple equation to arrive at the solution (20 chickens and 20 cows).

In the context of artificial intelligence, chain-of-thought reasoning for large language models involves dissecting a problem into smaller, sequential steps to enhance the quality of the final outcome. While this process generally requires more time to produce an answer, it significantly increases the likelihood of accuracy, particularly in logical or coding contexts. Reasoning models are derived from conventional large language models and are optimized for chain-of-thought processing through reinforcement learning.

(See: Large language model)

Though a somewhat broad term, "compute" predominantly refers to the critical computational power that enables AI models to function. This processing capability is the lifeblood of the AI industry, facilitating the training and deployment of its advanced models. The term frequently serves as a shorthand for the hardware components that provide this power, such as GPUs, CPUs, TPUs, and other foundational infrastructure elements of the modern AI landscape.

Deep learning is a specialized branch of self-improving machine learning where AI algorithms are structured with a multi-layered artificial neural network (ANN). This architecture empowers them to identify more intricate correlations compared to simpler machine learning systems, such as linear models or decision trees. The design of deep learning algorithms draws inspiration from the complex, interconnected pathways of neurons within the human brain.

Deep learning AI models possess the inherent ability to identify salient features within data, eliminating the need for human engineers to explicitly define these characteristics. This structure also supports algorithms that can learn from errors, iteratively refining their outputs through repetition and adjustment. However, deep learning systems demand extensive datasets—millions or more data points—to achieve optimal results. They also typically require longer training periods compared to simpler machine learning algorithms, leading to higher development costs.

Diffusion technology is central to many AI models capable of generating art, music, and text. Drawing inspiration from physics, diffusion systems systematically "destroy" the structure of data—such as images or audio—by progressively introducing noise until the original form is lost. While diffusion in physics is spontaneous and irreversible (e.g., dissolved sugar cannot be reformed into a cube), AI diffusion systems are designed to learn a "reverse diffusion" process. This allows them to reconstruct the original data, effectively recovering it from noise.

Distillation is a technique employed to transfer knowledge from a larger, more complex AI model (the ‘teacher’) to a smaller, more efficient one (the ‘student’). Developers send queries to the teacher model and record its outputs. These responses are sometimes validated against a dataset for accuracy. Subsequently, these outputs are used to train the student model to emulate the teacher's behavior.

This method enables the creation of a more compact and efficient model based on a larger one, incurring minimal "distillation loss." This process is widely believed to be how OpenAI developed GPT-4 Turbo, a faster iteration of GPT-4.

While all AI companies utilize distillation internally, some may have also employed it to rapidly catch up with frontier models developed by competitors. However, distilling knowledge from a rival's model typically constitutes a violation of the terms of service for AI APIs and chat assistants.

Fine-tuning refers to the subsequent training of an AI model to optimize its performance for a highly specific task or domain, beyond its initial broad training. This typically involves feeding the model new, specialized, or task-oriented data.

Many AI startups are adopting large language models as a foundational element for commercial products. They then strive to enhance utility for a particular sector or task by augmenting the initial training with fine-tuning based on their unique domain-specific knowledge and expertise.

(See: Large language model [LLM])

A Generative Adversarial Network, or GAN, is a machine learning framework that underpins significant advancements in generative AI, particularly in producing highly realistic data, including but not limited to deepfake technologies. GANs operate with a pair of neural networks: one network generates an output from its training data, which is then passed to the second network for evaluation. This second model, known as the discriminator, acts as a classifier, assessing the generator's output and thereby enabling the generator to improve over time.

The GAN architecture is structured as a competition, hence "adversarial," with both models programmed to continuously challenge each other. The generator endeavors to produce outputs that can deceive the discriminator, while the discriminator strives to accurately identify artificially generated data. This structured contest autonomously optimizes AI outputs for greater realism, reducing the need for additional human intervention. However, GANs are most effective for narrower applications, such as generating realistic photos or videos, rather than for general-purpose AI.

Hallucination is the AI industry's term for instances where AI models fabricate information, literally generating incorrect data. This phenomenon represents a significant challenge to AI quality and reliability.

Hallucinations result in generative AI outputs that can be misleading and potentially lead to real-world hazards, with dangerous consequences (consider a health query returning harmful medical advice). Consequently, the fine print of most GenAI tools now advises users to verify AI-generated answers, even though these disclaimers are often far less prominent than the readily available information the tools provide.

The problem of AI fabricating information is largely attributed to gaps in training data. For general-purpose generative AI, also known as foundation models, this issue proves particularly difficult to resolve. There simply isn't enough existing data to comprehensively train AI models to answer every conceivable question. In essence, we have not yet created an omniscient intelligence.

The prevalence of hallucinations is driving a strategic shift towards increasingly specialized and/or vertical AI models—domain-specific AIs requiring narrower expertise—as a means to mitigate knowledge gaps and reduce the risks of disinformation.

Inference refers to the process of operating an AI model. It involves deploying a trained model to make predictions or derive conclusions from previously unseen data. Crucially, inference cannot occur without prior training; a model must first learn patterns within a dataset before it can effectively extrapolate from that training data.

Various hardware types can perform inference, ranging from smartphone processors to powerful GPUs and custom-designed AI accelerators. However, their performance varies significantly. Very large models, for instance, would take an inordinate amount of time to make predictions on a standard laptop compared to a cloud server equipped with high-end AI chips.

Large language models, or LLMs, are the foundational AI models powering popular AI assistants such as ChatGPT, Claude, Google’s Gemini, Meta’s AI Llama, Microsoft Copilot, and Mistral’s Le Chat. When engaging with an AI assistant, you are interacting with an LLM that processes your request, either directly or by leveraging various integrated tools like web browsing or code interpreters.

It is important to note that AI assistants and the underlying LLMs may have distinct names. For example, GPT designates OpenAI’s large language model, while ChatGPT refers to the specific AI assistant product.

LLMs are sophisticated deep neural networks comprising billions of numerical parameters (or weights). These networks learn the intricate relationships between words and phrases, constructing a rich, multidimensional representation of language.

These models are developed by encoding patterns discovered across billions of books, articles, and transcripts. When a user provides a prompt to an LLM, the model generates the most probable pattern that aligns with the input. It then evaluates and predicts the most likely subsequent word based on the preceding context, repeating this iterative process to form coherent responses.

Memory cache refers to a vital process that significantly enhances inference, which is the mechanism by which AI generates responses to user queries. Essentially, caching is an optimization technique designed to improve inference efficiency. AI operations are inherently driven by intensive mathematical calculations, and each calculation consumes computational power. Caching aims to reduce the number of calculations a model must perform by storing specific calculations for future user queries and operations. Among various types of memory caching, KV (key-value) caching is particularly well-known. KV caching operates within transformer-based models, boosting efficiency and accelerating results by minimizing the time and algorithmic effort required to produce answers to user questions.

A neural network describes the multi-layered algorithmic structure that forms the bedrock of deep learning and, more broadly, the recent surge in generative AI tools following the advent of large language models.

Although the concept of emulating the densely interconnected pathways of the human brain as a design principle for data processing algorithms dates back to the 1940s, it was the more recent proliferation of graphical processing hardware (GPUs)—primarily driven by the video game industry—that truly unleashed this theory's potential. These chips proved exceptionally adept at training algorithms with far more layers than previously feasible, enabling neural network-based AI systems to achieve significantly superior performance across diverse domains, including voice recognition, autonomous navigation, and drug discovery.

(See: Large language model [LLM])

“RAMageddon” is a playful yet apt term for a concerning trend sweeping the tech industry: a persistent and escalating shortage of random access memory (RAM) chips, which are fundamental to nearly all modern technology products. As the AI industry flourishes, major tech companies and AI laboratories, all vying for the most powerful and efficient AI, are acquiring vast quantities of RAM to power their data centers. This intense demand has left insufficient supply for other sectors, and the resulting bottleneck has driven up the cost of available RAM significantly.

This scarcity impacts diverse industries, including gaming (where leading companies have had to increase console prices due to difficulties in sourcing memory chips), consumer electronics (where the memory shortage could lead to the largest decline in smartphone shipments in over a decade), and general enterprise computing (as businesses struggle to secure adequate RAM for their own data centers). The surge in prices is only expected to subside once the dreaded shortage ends; however, there is unfortunately little indication that this will happen anytime soon.

The development of machine learning AIs involves a crucial process known as training. In simple terms, this entails feeding data into a model to enable it to learn patterns and subsequently generate useful outputs.

At this stage of the AI development cycle, the process can take on a somewhat philosophical dimension. Prior to training, the mathematical structure that serves as the foundation for a learning system is merely a collection of layers and random numbers. It is exclusively through training that the AI model truly begins to take shape. Essentially, training is the process by which the system responds to characteristics within the data, allowing it to adapt its outputs toward a desired objective—whether that involves identifying images of cats or composing a haiku on demand.

It is important to note that not all AI systems necessitate training. Rules-based AIs, which are programmed to follow manually predefined instructions—such as linear chatbots—do not undergo this process. However, such AI systems are typically more constrained in their capabilities compared to well-trained, self-learning systems.

Nonetheless, training can be an expensive endeavor due to the substantial volume of inputs required, a quantity that has generally been increasing. Hybrid approaches can sometimes offer shortcuts in model development and assist in managing these costs. An example is data-driven fine-tuning of a rules-based AI, which reduces the need for extensive data, computational power, energy, and algorithmic complexity compared to building a model entirely from scratch.

In the realm of human-machine communication, inherent challenges exist. Humans communicate through natural language, while AI programs execute tasks and respond to queries via complex algorithmic processes informed by data. In their most fundamental definition, tokens represent the basic building blocks of human-AI communication, functioning as discrete segments of data that have either been processed by or produced by a large language model (LLM).

Tokens are generated through a process called “tokenization,” which breaks down raw data and refines it into distinct units that are digestible for an LLM.