AI Concepts
AI terms, actually explained.
Plain-language definitions for engineering leaders and executives building with AI. Every term defined the way a practitioner would explain it to a colleague who needs to understand, not be impressed.
A
Agent (AI)
An AI system that perceives inputs, decides on actions, and executes those actions autonomously to accomplish a goal - rather than simply returning a single response to a prompt. Agents can call tools, browse the web, write code, and loop through multiple steps before finishing.
Agentic Loop
The repeating cycle an AI agent runs through: observe, think, act, observe again. The loop continues until the task is complete or a stopping condition is met, allowing the agent to handle multi-step tasks that a single model call cannot.
AI Gateway
A middleware layer that sits between your application and one or more LLM providers, handling routing, rate limiting, caching, logging, and cost management in one place. Teams use gateways to abstract provider details and apply consistent policy across all model calls.
AI Governance
The policies, processes, and controls an organization uses to manage risk, ensure compliance, and maintain accountability around AI systems. Governance covers model selection, data handling, output review, audit trails, and escalation paths.
AI Readiness
A measure of how prepared an organization is to adopt and operate AI systems, covering data quality, infrastructure, team skills, and governance maturity. A readiness assessment identifies the gaps that must be closed before AI investments will pay off.
API
Application Programming Interface - a defined contract that lets two software systems communicate. In AI contexts, the LLM API is the endpoint your code calls to send a prompt and receive a model response, typically over HTTP with a JSON payload.
Autonomous Agent
An agent that executes tasks with minimal human oversight, making decisions and taking actions end-to-end without requiring approval at each step. The level of autonomy is a design choice balanced against safety and error-recovery needs.
B
Batch Inference
Running model predictions on a large dataset all at once, rather than one request at a time. Batch inference is significantly cheaper and is the right pattern when low latency is not required - for example, nightly processing of documents or weekly scoring of customer records.
Benchmark
A standardized test suite used to compare AI model performance on a defined task - such as coding, reasoning, or question answering. Benchmarks let teams make model selection decisions on objective data rather than vendor marketing.
BM25
A classical keyword-based ranking algorithm that scores documents by term frequency and document length, used widely in search engines before neural methods became dominant. In hybrid search systems, BM25 handles exact keyword matches while a vector index handles semantic similarity.
C
Chain-of-Thought
A prompting strategy that instructs a model to show its reasoning steps before giving a final answer, substantially improving accuracy on math, logic, and multi-step problems. You trigger it by adding phrases like "think step by step" or by including worked examples in the prompt.
Chunking
The process of splitting a large document into smaller segments before embedding and indexing it in a vector database. Chunk size and overlap are key tuning parameters - too large and retrieval precision suffers; too small and context gets fragmented.
Classification
An AI task where a model assigns input to one of a fixed set of categories - for example, labeling a support ticket as billing, technical, or general. LLMs can perform zero-shot classification without training data, though fine-tuned models are typically more reliable for production use.
Context Window
The maximum amount of text a model can process in a single call, measured in tokens. Everything the model "knows" during a response - the system prompt, conversation history, retrieved documents, and tool outputs - must fit within this window.
Cosine Similarity
A mathematical measure of how similar two vectors are, computed from the angle between them (ignoring magnitude). Vector databases use cosine similarity or dot product to find the stored embeddings closest to a query embedding during retrieval.
D
Data Pipeline
An automated sequence of steps that moves, transforms, and loads data from source systems into the format needed by an AI model or downstream application. For RAG systems, this typically means ingesting documents, chunking, embedding, and upserting into a vector store.
E
Embeddings
Numerical representations of text (or images, audio, etc.) where similar content maps to nearby points in high-dimensional space. Embeddings are the foundation of semantic search and RAG systems because they let you find conceptually related content even without exact keyword matches.
Embedding Model
A specialized model that converts text into fixed-size embedding vectors, separate from and typically much smaller than a generative LLM. Common options include OpenAI's text-embedding-3 series and open-source models like nomic-embed-text.
Evaluation Framework
A system for measuring whether an AI application is performing correctly, including test datasets, scoring rubrics, and automated judges. Without structured evals, teams discover failures in production rather than during development.
Evals
Short for evaluations - the tests you run against an AI system to measure accuracy, safety, and reliability before and after changes. Good evals are the difference between confident deployments and hoping nothing breaks.
F
Few-Shot Learning
Providing a handful of input-output examples directly in the prompt to guide the model's behavior, without any gradient updates or fine-tuning. It is the fastest way to improve output format and style when you have limited labeled data.
Fine-Tuning
Additional training on a pre-trained foundation model using your own labeled examples, adjusting the model's weights to specialize its behavior for a specific domain or task. Fine-tuning is typically the right tool when prompt engineering alone cannot reach the required accuracy.
Foundation Model
A large model trained on massive datasets that can be adapted to a wide range of downstream tasks - either through prompting, fine-tuning, or integration with tools. GPT-4, Claude, and Gemini are all foundation models serving as the base for most production AI applications today.
Function Calling
A model capability that outputs a structured function name and arguments - rather than free text - so your application can execute a specific action and feed the result back to the model. It is the mechanism that turns a chat model into an agent that can interact with APIs and databases.
G
Generative AI
AI systems that produce new content - text, images, code, audio - rather than simply classifying or ranking existing content. Generative AI is distinguished from earlier "predictive" AI by its ability to create novel outputs that were not present in training data.
Graph RAG
An extension of standard RAG that uses a knowledge graph to represent relationships between entities, enabling retrieval that follows semantic connections rather than just finding similar text chunks. Graph RAG improves answers to questions that require reasoning across multiple related documents.
Grounding
Connecting a model's responses to verifiable, real-world information - such as retrieved documents, database records, or live web results - to reduce hallucination and increase factual accuracy. Grounding is a core function of RAG architectures.
Guardrails
Input and output validation layers that intercept model calls to enforce policy - blocking harmful inputs, sanitizing outputs, and detecting when a model is going off-task. Guardrails are a production engineering requirement, not an optional safety feature.
H
Hallucination
When an AI model generates confident-sounding content that is factually incorrect, fabricated, or unsupported by its context. Hallucination is an inherent property of probabilistic language models, not a bug to be patched - which is why grounding, evals, and output validation are essential.
Human-in-the-Loop
A system design where a human reviews or approves AI outputs at defined checkpoints before actions are executed. Human-in-the-loop is the standard pattern for high-stakes or irreversible decisions, with the goal of gradually reducing the review surface as confidence in the system grows.
Hybrid Search
A retrieval approach that combines keyword-based search (BM25) with semantic vector search, then merges the ranked results using a technique like Reciprocal Rank Fusion. Hybrid search consistently outperforms either method alone and is the recommended baseline for production RAG systems.
I
Inference
The process of running a trained model to generate a prediction or response - what happens every time your application sends a prompt and receives a reply. Inference is distinct from training, which is the (much more expensive) process of building the model's weights in the first place.
Inference Cost
The compute expense of running a model, typically billed per input and output token by cloud providers. Inference cost is a major architectural driver - it determines whether you use a large frontier model, a smaller specialized model, or a cached response, and at scale it becomes a significant line item.
In-Context Learning
The ability of a model to adapt its behavior based on examples or instructions provided in the prompt, without any weight updates. In-context learning is the core mechanism behind few-shot prompting and makes LLMs unusually flexible compared to traditional ML models.
Instruction Tuning
A fine-tuning approach that trains a model on pairs of instructions and desired responses, teaching it to follow directions reliably. Most production-ready chat models (including GPT-4 and Claude) are instruction-tuned versions of larger base models.
J
JSON Mode
A model setting that constrains output to valid JSON, making it easier to parse and integrate into downstream systems. Combining JSON mode with structured output schemas is the standard pattern for extracting typed data from unstructured text.
Judge Model
An LLM used to evaluate the outputs of another LLM, scoring them for quality, correctness, or adherence to criteria. Judge models are central to LLM-as-a-judge eval pipelines because they scale evaluation far beyond what human review can cover.
K
Knowledge Graph
A structured representation of entities and the relationships between them, stored as nodes and edges. Knowledge graphs let AI systems reason about connections that plain text retrieval misses - such as "what products are affected by this supplier's issues?"
KV Cache
A performance optimization in transformer models that stores intermediate attention computations (keys and values) from the prompt, so they do not need to be recomputed on every output token. For applications with long, repeated system prompts, KV caching can cut inference cost by 50–90%.
L
Latency
The time between sending a request to an AI model and receiving the complete response. In user-facing products, latency is a first-class engineering concern - streaming responses, smaller models for time-sensitive tasks, and caching are the main tools for managing it.
LLM (Large Language Model)
A deep learning model trained on massive text corpora to predict and generate language. LLMs underpin most of the AI capabilities enterprises are integrating today - from chatbots and code assistants to document analysis and agent systems.
LLM Ops
The operational practice of deploying, monitoring, and continuously improving LLM-based applications in production - covering prompt versioning, cost tracking, output quality monitoring, and incident response. LLM Ops is the AI-specific extension of DevOps principles to the new failure modes that models introduce.
Long-Context Model
An LLM with a context window of 100,000 tokens or more, enabling it to process entire codebases, lengthy legal documents, or hours of transcripts in a single call. Long-context models trade off some speed and cost against the ability to avoid chunking and retrieval for large documents.
M
MCP (Model Context Protocol)
An open protocol that standardizes how AI models connect to external tools, data sources, and services - similar to USB-C for AI integrations. MCP lets you build one tool server and connect it to any compliant model, eliminating per-integration boilerplate.
MLOps
The practice of applying DevOps principles to machine learning - covering model versioning, automated retraining pipelines, deployment, monitoring, and rollback. MLOps is the discipline that bridges data science and production engineering.
Model
In the AI context, a trained artifact - a set of numerical weights - that maps inputs to outputs according to patterns learned during training. "The model" is what you call via API; everything else in your stack is application logic built around it.
Model Card
A document published alongside an AI model that describes its intended use cases, training data, performance benchmarks, and known limitations or risks. Model cards are a governance artifact used for responsible AI deployment.
Multi-Agent System
An architecture where multiple AI agents collaborate to complete a task, each handling a specialized role. Multi-agent systems are used for tasks too complex or large for a single agent call - for example, a research agent that spawns sub-agents to search, read, and synthesize in parallel.
O
Observability (AI)
The practice of instrumenting AI systems so you can see what inputs went in, what the model returned, how long it took, and what it cost - for every call, in production. Without observability, debugging AI failures and measuring quality improvements is guesswork.
Orchestration
The logic that coordinates multiple model calls, tool invocations, and data retrievals into a coherent workflow. An orchestration layer decides what to call next, passes context between steps, and handles errors and retries - the control plane of a multi-step AI system.
Output Validation
Programmatic checks applied to model outputs before they reach users or downstream systems - verifying structure, content policy compliance, factual groundedness, and task completion. Output validation is how you catch hallucinations and policy violations at scale.
P
Prompt Engineering
The practice of designing and iterating on the instructions and context you send to an LLM to reliably get the outputs you want. Prompt engineering is a core skill for AI application development - the difference between a model that works and one that almost works.
Q
Quantization
A technique that reduces the precision of a model's numerical weights (for example, from 16-bit to 4-bit), shrinking its memory footprint and speeding up inference. Quantized models are the standard approach for running large models on-device or in cost-constrained environments.
R
RAG (Retrieval-Augmented Generation)
An architecture that combines a retrieval system with a generative model - first fetching relevant documents from a knowledge base, then passing them to the LLM as context for its response. RAG is the standard approach for grounding LLMs in your organization's proprietary data.
Red Teaming
Structured adversarial testing where a team attempts to make an AI system produce harmful, unsafe, or policy-violating outputs. Red teaming is how you find the failure modes that standard evals miss before they surface in production.
Reranking
A second-pass relevance scoring step applied to an initial set of retrieved documents, using a more computationally expensive model to re-order results before passing them to the LLM. Reranking substantially improves RAG quality at the cost of added latency.
RLHF (Reinforcement Learning from Human Feedback)
A training technique where human raters evaluate model outputs, and those ratings are used to train a reward model that guides further fine-tuning. RLHF is the primary technique used to align large language models with human preferences and safety requirements.
S
Semantic Search
Search that finds results based on meaning rather than exact keyword matches, using embedding vectors to identify conceptually related content. Semantic search is more robust than keyword search for natural-language queries and is the retrieval backbone of most RAG systems.
Structured Output
Model output constrained to a defined schema - such as a specific JSON structure - rather than free-form text. Structured output makes AI integration into downstream systems deterministic and eliminates the need for fragile parsing logic.
System Prompt
Instructions sent to a model at the start of every conversation to define its persona, constraints, and behavior. The system prompt is the primary mechanism for controlling how a model responds across all user turns in an application.
T
Tokenization
The process of splitting text into tokens - the units a model processes and generates. Most English words are one or two tokens; rare words, numbers, and non-English text often tokenize into more. Token counts determine context window usage and API pricing.
Tool Use
The ability of an AI model to call external functions, APIs, or services and incorporate the results into its response. Tool use is what distinguishes an agent that can act in the world from a chat model that can only generate text.
Trace
A complete record of all inputs, outputs, tool calls, and timing for a single AI workflow execution - the AI equivalent of a distributed system trace. Traces are the primary debugging artifact for multi-step agent systems and the foundation of LLM Ops observability.
V
Vector Database
A database optimized for storing and querying high-dimensional embedding vectors, enabling fast approximate nearest-neighbor search across millions of documents. Vector databases are the storage layer for RAG systems.
Z
Zero-Shot Learning
Prompting a model to perform a task it has no explicit examples for, relying entirely on its pre-trained knowledge and the task description in the prompt. Zero-shot is the starting point for most LLM integrations - if it works, you ship it; if not, you add few-shot examples or fine-tune.
Ready to put these concepts into practice?
VSP helps engineering teams and executives move from understanding AI concepts to deploying AI systems that work in production.