From Agent Builders to Agent Runtimes: The Software Stack Taking Shape in 2026
OpenAI's latest agent-tool changes reveal a broader shift toward code-first runtimes, deliberate retrieval, tracing and repeatable tests.
Recent moves from OpenAI, Google, Anthropic and Mistral point to a practical shift: the model matters, but dependable agents are built in the layers around it.
Two small notices from OpenAI tell a larger story about where AI agents are heading.
On August 24, 2026, OpenAI deprecated the codex mcp-server command and directed developers toward the Codex app server. Less than three months earlier, it had updated its AgentKit announcement to say that the visual Agent Builder and its Evals product would disappear from the platform after November 30, recommending the code-based Agents SDK for workflows that need to continue.
Those decisions do not mean visual agent builders are finished. They do reveal something more useful: the centre of gravity is moving away from the canvas and toward the runtime. The durable parts of an agent system are increasingly the pieces that can be versioned, tested, traced, swapped and operated like software.
That shift is arriving just as frontier models are getting better at coding, tool use and long-running tasks. Google is marketing Gemini 3.7 Flash specifically for coding and agents. Anthropic describes Claude Opus 5 as more deliberate about verifying and iterating on its work. Mistral has turned search itself into a multi-step agent loop. The model layer is improving quickly, but production value is accumulating in the surrounding stack.
For developers and technology leaders, the practical question is no longer simply, “Which model should we use?” It is, “What system will keep working when the model, tools, data and requirements change?”
A product sunset can be an architecture signal
AgentKit’s original 2025 launch presented a broad platform: a visual canvas for multi-agent workflows, a connector registry, embeddable chat interfaces and evaluation tooling. Its June 2026 update draws a sharper boundary. Natural-language workflows can live as workspace agents; workflows that must persist as code should move to the Agents SDK.
OpenAI has not publicly given a detailed post-mortem for the decision, so it would be careless to invent one. But the architectural implication is clear. A visual graph can be an excellent place to prototype and communicate a flow. A code-first runtime is a stronger place to own the contracts that matter in production: which tool can run, what data it receives, when control transfers, what happens after a failure and how the result is tested before release.
The current Agents SDK documentation defines an agent as a model combined with instructions, tools and optional runtime behaviour such as handoffs, guardrails and structured outputs. Its orchestration guidance distinguishes two patterns: a manager that invokes specialist agents as tools, and handoffs that transfer the conversation to another agent. That is less glamorous than drawing boxes on a canvas, but it describes the decisions that determine whether an agent remains understandable once it handles real users and real data.
The August deprecation of codex mcp-server points in the same direction. OpenAI is steering Codex integrations toward an app-server interface and a plugin for Claude Code. The detail may affect only a subset of developers today. The broader lesson is that agent interfaces are still moving, so teams should avoid welding business logic to one temporary command or vendor-specific surface.
The five-layer agent stack
The emerging stack can be understood as five layers. This is an analytical framework, not a standard announced by any one vendor.
| Layer | What it does | The production question |
|---|---|---|
| Model | Reasons, writes, interprets and chooses actions | Is it capable enough at an acceptable cost and latency? |
| Tools and interfaces | Expose APIs, files, browsers, code execution and other actions | Are permissions narrow, inputs validated and outputs structured? |
| Retrieval and context | Find the evidence and state needed for the current step | Can the system locate, inspect and cite the right source rather than guess? |
| Runtime and orchestration | Manage turns, handoffs, budgets, retries and human approval | Who controls the loop, and where does it stop? |
| Observability and evaluation | Record behaviour and test repeatability | Can a team explain failures and detect regressions? |
The stack is becoming more explicit because agents are no longer single model calls wrapped in a chat box. They are processes. A research agent may search, open a source, discover that the source is incomplete, revise the query, compare dates, write a synthesis and ask for approval. A coding agent may inspect a repository, change several files, run tests and revise the patch. Every added step creates another place where the system can drift, repeat itself, use the wrong tool or confidently stop too early.
Model improvements make those workflows possible. They do not remove the need to engineer them.
Better models increase the value of the surrounding layers
Google’s August 13 release of Gemini 3.7 Flash is a useful example. Google positions it as a high-throughput model for coding and agents and reports gains over Gemini 3.6 Flash across software engineering, document comprehension and workflow benchmarks. The published figures include 43.6% versus 34.4% on FrontierCode 1.1 and 30.4% versus 17.0% on Zapier’s AutomationBench. These are vendor-reported results, and benchmark scores never guarantee performance in a particular organisation. Still, they show where model development is being aimed: not only at answering questions, but at completing sequences of work.
Anthropic makes a similar case for Claude Opus 5. Its release announcement emphasises software engineering, computer use and end-to-end business tasks. Anthropic says the model is more likely to verify its work and iterate until it succeeds, and it describes examples in which the model created its own test harness or found a deeper root cause rather than treating a symptom. Those examples are selected by the vendor and should be read as demonstrations, not universal behaviour. What matters architecturally is the behaviour they reward: agents that inspect their own output need access to tests, tools and feedback channels that make inspection meaningful.
A more capable model therefore raises the return on a good harness. Give it a precise tool contract, a realistic test and a trace of what happened, and it has a chance to correct itself. Give it unrestricted tools and a vague definition of “done,” and capability can simply produce a more elaborate failure.
Retrieval is becoming an action loop, not a lookup
Mistral’s Agentic Search, released August 20, makes the retrieval layer unusually visible. Traditional retrieval-augmented generation usually fetches a fixed set of text chunks before the model writes an answer. Mistral’s system instead lets a model repeatedly search, open, navigate, read and grep through existing indexes.
That difference matters when evidence is buried in a footnote, spread across several documents or dependent on a reference discovered halfway through the task. In such cases, retrieval is not a preprocessing step. It is part of the agent’s reasoning loop.
Mistral reports large gains on FinanceBench and OfficeQA Pro, along with lower token use and latency. Those numbers are company measurements and need independent reproduction before being treated as general results. Even without accepting every performance claim, the design direction is important: retrieval systems are being exposed as tools the agent can use deliberately, with a visible sequence that can be inspected afterward.
This also changes what teams should measure. “Did the final answer sound right?” is too weak. A useful evaluation asks whether the agent searched the correct corpus, opened the authoritative document, followed the relevant reference, represented uncertainty accurately and attached evidence to the conclusion. The path matters because a plausible answer reached through the wrong evidence will fail when the question changes.
Reliability is a different problem from capability
The strongest reason for a runtime-and-observability layer comes from independent evaluation. METR’s task-completion time-horizon work estimates the duration of software, machine-learning and cybersecurity tasks at which an agent reaches a given probability of success. The organisation stresses that a 50% time horizon is not the length of work that can safely be delegated. Its tasks are comparatively clean and well specified, while real work often depends on tacit context, collaboration and difficult-to-score outcomes.
METR’s separate note on the limits of the metric is even more direct: reliability-critical tasks may need success rates above 98%, and fewer human interventions do not necessarily mean less human labour if failures become harder to diagnose.
A March 2026 preprint, “Beyond pass@1”, approaches the issue from another angle. Across 23,392 episodes and 396 tasks, its authors report that capability rankings and reliability rankings can diverge as tasks get longer. As a preprint, the paper should be treated as emerging evidence rather than settled consensus. Its core distinction is nevertheless valuable: a model succeeding once shows possibility; a system succeeding repeatedly shows operability.
This is why tracing and deterministic tests are becoming first-class features. OpenAI’s Agents SDK tracing records model generations, tool calls, handoffs, guardrails and custom events across a run. Its testing utilities can exercise agent workflows in memory without making live model or sandbox requests. That lets a team test the structure it owns—tool selection, transitions, failure handling and output contracts—without paying for a model call or introducing model variance into every unit test.
Tracing is not proof that an answer is correct, and tests can only enforce the behaviours their authors specify. Together, however, they turn a mysterious conversation into a debuggable program.
What builders should do now
The stack is still changing, but the following design choices are durable across vendors.
1. Keep the workflow’s source of truth in code
Use a visual canvas for exploration if it helps. Before a workflow becomes important, represent its tool contracts, state transitions, stop conditions and approvals in version-controlled code or a portable declarative format. A migration should be inconvenient, not existential.
2. Make tools small and explicit
An agent does not need “database access”; it needs a narrowly defined action such as “retrieve an order by ID” or “propose a refund below a limit.” Validate inputs, return structured outputs and separate read actions from write actions. Smaller tools reduce both accidental damage and the number of behaviours that must be evaluated.
3. Treat retrieval as evidence collection
Record which documents were searched and opened. Preserve dates, source identities and citations. Let the agent say that evidence is missing. For high-value tasks, score the research path as well as the final prose.
4. Trace every meaningful production run
A trace should reveal the model version, prompt and policy version, tool calls, handoffs, latency, token use, errors and final disposition. Sensitive data requires redaction and careful retention controls; observability must not become a second data leak. The goal is a record that lets an operator answer, “Why did the system do that?”
5. Evaluate reliability by task class
Do not compress an agent into one average score. Track success across short and long tasks, common and rare cases, clean and messy inputs, and repeated runs. Include failures caused by the harness, not only failures attributed to the model. A workflow with a 95% completion rate may sound impressive until the remaining 5% contains irreversible actions.
6. Design an escape hatch
Set budgets for steps, time and spend. Define conditions for retry, fallback, human review and termination. Long-running autonomy should be earned by evidence. The safest default is bounded delegation: give the agent enough room to be useful, while keeping costly or irreversible actions behind explicit gates.
A more global and more portable developer market
This architecture shift matters beyond the largest US labs. The underlying pattern is visible across American and European vendors, and the developer opportunity is global. A small team in Accra, Nairobi, Lagos, Bengaluru or São Paulo does not need to train a frontier model to build a valuable agent. It can combine a suitable model with local data, carefully designed tools, language and domain expertise, and an operating layer built for local realities.
Portability is especially important for teams working with uneven connectivity, strict data-location requirements or volatile API economics. If the model is one replaceable layer rather than the entire application, a builder can route simple tasks to a smaller model, reserve a frontier model for difficult cases, or run an open model where control matters more than peak benchmark performance. The business advantage sits in the workflow and its feedback data, not merely in access to the newest endpoint.
What to watch next
Three developments will show whether this stack is truly consolidating.
First, watch for common trace formats and portable tool contracts. Model Context Protocol helped normalise one way of connecting models to tools, but the surrounding runtime interfaces remain fluid. Second, watch evaluation move closer to software testing: reproducible fixtures, failure injection, regression suites and task-specific reliability curves. Third, watch vendors separate agent authoring from agent operation. The interface used to sketch a workflow may not be the system that schedules, secures and observes it in production.
The visual builder will survive because it makes complex flows easier to understand. It is simply becoming the front door rather than the foundation.
The real product is the system around the model
Frontier models will keep trading places on leaderboards. Their prices, context windows and tool-use abilities will keep changing. A durable agent product cannot depend on any one of those positions remaining stable.
The more lasting advantage is a system that knows what the agent may do, finds the evidence it needs, records the path it took, tests expected behaviour and stops safely when confidence runs out. OpenAI’s product changes make that direction unusually visible, while recent releases from Google, Anthropic and Mistral show why the surrounding infrastructure is becoming more valuable.
The next generation of agents will not be defined by how impressive their demos look. It will be defined by whether operators can understand them on a bad day.
Disclosure: This article was researched and drafted with AI assistance. Material factual claims were checked against the primary and research sources linked throughout. The featured image is original AI-generated editorial artwork created for Eazzy Tech News.
