"Is this an agent or just an LLM?" is the most useful question a buyer can ask in 2026, and the one most vendors avoid answering directly. The distinction is not pedantic. An LLM is a component; an agent is a system built around it. The buying decision, the operational expectations, and the reliability characteristics differ enough that conflating the two leads to disappointing outcomes on both sides.

The framing is straightforward once stated clearly: the LLM is the engine, the agent is the car. You can buy engines; you can buy cars; both are valid purchases. But you have to know which you are buying. This piece walks through what an LLM actually does, what an agent adds, and where the confusion shows up in real buying conversations.

Component vs system: the framing

An LLM (large language model) is a function: text in, text out. Given a prompt, it returns a continuation. The continuation can be a chat response, a structured output, a code snippet, or a function-call request. The LLM has no state between calls, no ability to take actions in the world, no memory of past interactions unless you pass them back in.

An AI agent is a system that wraps an LLM with the components needed to do work autonomously. The agent has a goal, a tool layer (APIs the agent can call), a control loop (decide what to do next, do it, observe, repeat), a memory layer (state across iterations), and safety guardrails (refusal policy, blast-radius controls, audit logs). The LLM is one component inside the agent; the rest of the system is what makes the agent useful.

Agent system contains an LLM as one component Agent (the system) + Goal + Tool layer (APIs the agent can call) + Control loop (think, act, observe, repeat) + Memory layer (state across iterations) + Safety guardrails (refusal, blast-radius, audit) LLM (the component) Text in, text out, per iteration Source: Gravity architecture model, May 2026.
The LLM does the reasoning step within the loop. Everything else is the agent system.

What an LLM actually does

An LLM does one thing very well: given text, produce more text that follows. The "more text" can be a chat response, a structured output (JSON, code), or a request to call a function. With function-calling support, the model can produce a call request like "call send_email with these arguments", but the model itself does not call anything; it produces the request, and a wrapping system has to actually execute the call.

The LLM has no native concept of tools, only of text patterns that look like tool calls. It has no native concept of state across calls, only of what is in the current context window. It has no native concept of safety beyond the refusal training built into the model weights. All of these properties have to be added by the agent system.

OpenAI and Anthropic both provide LLMs as APIs and document the function-calling and tool-use patterns explicitly (OpenAI function calling guide, Anthropic on building effective agents). Reading those docs makes the component-vs-system distinction obvious; the documentation describes the LLM API and points out that an agent loop is something you build around it.

What an agent adds on top

Five things, each of which has its own implementation complexity.

The goal

The agent has a defined goal that holds across iterations. The goal is encoded in the system prompt covered in AI agent prompt vs traditional prompt but the goal exists at the system level, not just at the prompt level. Different agent runs with different goals are different agents from the user's perspective.

The tool layer

The set of APIs the agent can call. Each tool has a description, a schema, and an implementation. The tool implementations are the bridge between the model's text output and the world; the agent system runs them.

The control loop

The "decide what to do next, do it, observe results, repeat" loop. Without the loop, the LLM produces a tool-call request once and stops. With the loop, the LLM is asked again after each tool call, with the result fed back in, until the goal is met or a stopping condition triggers.

The memory layer

State the agent maintains across iterations and sometimes across runs. Short-term memory (the working context for one run) is usually managed inside the loop; long-term memory (state that persists across runs) typically uses a vector store or key-value store. Both are infrastructure the LLM does not provide.

The safety guardrails

The four-layer architecture covered in AI agent safety and guardrails: prompt-level refusal policy, tool-level permission scopes, network-level allow-listing, audit-level circuit breakers. The model's refusal training is part of layer one; the rest is system-level work.

Why the buying decision differs

The buying decision for an LLM API and an agent product are different in three ways.

Capability vs system. An LLM API gives you a capability you wrap into whatever agent loop you build. An agent product gives you a working system end-to-end. The first requires engineering investment to make useful; the second hides the engineering and presents the result.

Pricing model. LLM APIs are nearly always per-token. Agent products span the cost-model spectrum covered in AI agent cost models explained: per-task, capability-based, or flat. The buyer pays for the system, not just the model time.

Reliability expectations. LLM APIs are evaluated on model quality benchmarks. Agent products are evaluated on the reliability methodology covered in how we test AI agents, including the eight failure-mode categories. Buyers who treat agent products like LLM APIs miss the reliability questions that matter most.

Where the confusion shows up

Three common confusions in buying conversations.

"Why is your agent so much more expensive than the underlying LLM?" Because the agent is the system, not the model. The price covers the loop, the tool integrations, the safety layer, the reliability methodology, and the support burden of running someone's autonomous workload. The model API is a fraction of the system's full cost.

"Can the agent just do X like the LLM does?" The agent's capabilities are not a strict superset of the LLM's. An agent can be more constrained than the LLM behind it because the safety layer or tool layer limits what the agent will do. An agent that refuses to summarise a document is doing what the system is configured to do, not what the LLM alone would do.

"We tried agents; they don't work as well as the chat assistant." Two products solving different jobs. The chat assistant is interactive; the agent is autonomous. The reliability bar is higher for the agent because there is no human in the loop catching mistakes. An agent that hits 95% on the eight failure-mode categories is shippable; a chat assistant at 95% feels worse because the user catches every miss.

The framework I learned across three startups applies again: the abstraction the buyer is buying has to be precise enough that they know what they are getting. "AI" is too vague; "AI agent" is more specific; "an autonomous AI agent that owns task X end-to-end with these safety guarantees" is precise enough to evaluate. The LLM-vs-agent distinction is a precondition for that precision.

Frequently asked questions

What is the difference between an AI agent and an LLM?

An LLM (large language model) is a component that takes text input and returns text output. An AI agent is a system built around an LLM that adds a goal, a tool list, a memory layer, and a control loop. The LLM does the reasoning; the agent does the work. You buy an LLM API; you buy an agent product.

Can an LLM act on its own without being an agent?

Not really. An LLM by itself produces text. Without a tool layer to call APIs and a control loop to decide when to call them, the model cannot send emails, query databases, or take any action with side effects. The agent system is what turns text generation into autonomous action.

Why does the agent vs LLM distinction matter for buyers?

Because the buying decision is different. Buying LLM API access means buying capability that you wrap into your own agent. Buying an agent product means buying the agent loop, the tool integrations, the safety layer, and the reliability methodology. Confusing the two leads to underpaying for an LLM API and expecting agent features, or overpaying for an agent and complaining about LLM characteristics.

Is GPT or Claude an agent or an LLM?

Both. The underlying model (GPT-4, Claude Sonnet) is an LLM. The chat product wrapped around it can run an agent loop with tools, in which case the product is an agent built on the LLM. The terminology is loose in marketing copy; the technical distinction is between the model component and the agent system that uses it.

What does an AI agent add on top of an LLM?

Five things: a goal (what success looks like), a tool layer (what the agent can call), a control loop (deciding what to do next), a memory layer (state across calls), and safety guardrails (refusal policy, blast-radius control, audit). The LLM provides the reasoning step within each iteration of the loop; the agent infrastructure provides everything else.

Three takeaways before you close this tab

Sources