Think about two startups constructing related merchandise. Each select Claude Sonnet 5 and ask it to automate the processing of incoming buyer requests. One month later, the primary agent classifies inquiries, retrieves the proper information, and completes duties persistently. The second hallucinates lacking particulars, loses monitor of the unique request, and falls right into a loop after two steps. The mannequin is similar. The outcomes are usually not.
This hole helps clarify why AI tasks fail even when groups begin with a succesful LLM. Gartner predicts that greater than 40% of agentic AI tasks will probably be canceled by the tip of 2027 due to escalating prices, unclear enterprise worth, or insufficient threat controls. For corporations exploring agentic AI for enterprise, mannequin choice is just a part of the choice.
So what creates the distinction? It’s not merely the intelligence of the mannequin, however the system by which it operates, the harness. At SCAND, we’ve designed and shipped agentic methods throughout industries, and we see this sample persistently: the LLM isn’t the issue. The design, structure round it virtually at all times is.
Why AI Brokers That Work within the Demo Break in Manufacturing
A demo exhibits that an AI agent can full a predefined activity underneath managed circumstances. The system normally receives a transparent request, works with a restricted set of information, and makes use of instruments whose conduct is already identified. A number of profitable eventualities can create the impression that the agent is prepared for a full-scale launch, however real-world operation shortly exposes its weaknesses.
In manufacturing, consumer requests could also be incomplete, ambiguous, or contradictory. Information turns into outdated, context home windows are overflown, APIs return sudden responses, entry permissions change, and exterior providers change into briefly unavailable. The extra steps an agent should full, the better the prospect that an error at one stage will have an effect on each motion that follows. Resulting from Context Rot or Context Fatigue.
The issue is amplified by the truth that AI brokers don’t function like deterministic packages. With the identical enter, conventional code normally follows a predefined sequence of actions. An LLM selects the following step probabilistically, which implies that the identical mannequin with the identical immediate could use totally different instruments, interpret retrieved information in a different way, and produce totally different outcomes.
In apply, three kinds of failure happen significantly usually. Dumb RAG gives the agent with incomplete, irrelevant, or outdated context. Brittle Connectors disrupt execution when APIs, parameter schemas, permissions, or response codecs change. Compounding Error causes errors to build up: an incorrect resolution enters the context, influences the following step, and progressively strikes the agent away from the unique activity.
These points could stay hidden in a demo. In manufacturing, they recur throughout a a lot wider vary of eventualities and switch remoted deviations into systemic failures. Because of this LLM agent reliability relies upon not solely on the capabilities of the LLM but additionally on how context, instruments, validation, and constraints are designed round it.
What Really Determines How Your AI Agent Performs
An AI agent’s efficiency relies on greater than the LLM it makes use of. Even a extremely succesful mannequin won’t deal with multi-step duties reliably if it receives irrelevant context, selects the unsuitable instruments, loses vital info between actions, or fails to acknowledge when the method ought to cease.
These features are managed by harness design — the engineering layer across the LLM. It determines what information, context the mannequin receives at every stage, how the agent interacts with APIs and exterior methods, the place it shops the duty state, the way it handles errors, and what constraints it should comply with. In apply, the harness turns a standalone mannequin right into a manageable agentic system that may function inside actual enterprise processes.
Harness design is broader than immediate engineering. A immediate defines the agent’s function, purpose, and behavioral guidelines, nevertheless it doesn’t management all the execution movement. It doesn’t resolve which information must be retrieved, whether or not a device response will be trusted, what number of occasions a failed motion must be retried, if movement is safe, when compaction begins, or when human approval is required. All of this belongs to agentic system design.
That is particularly vital in multi-step eventualities. For instance, the ReAct agent structure for LLM methods follows a cycle of reasoning, motion, and commentary. Nevertheless, the sample alone doesn’t assure reliability. With out management over context, reminiscence, device calls, and stopping circumstances, the agent could select the unsuitable motion, enter a loop, or progressively drift away from the unique purpose. The standard of the system subsequently relies upon not on a single immediate, however on how fastidiously the mannequin’s work is managed at each stage.
The Elements of Agentic System Design That Really Matter
The reliability of an AI agent doesn’t rely on a single expertise. It relies on how effectively all of the elements surrounding the mannequin work collectively. The weather under type efficient AI agent design patterns and distinguish a secure manufacturing system from a powerful however fragile prototype.
Reasoning Structure
That is the route an agent follows to succeed in a end result. ReAct alternates between reasoning, motion, and commentary, whereas PlanReAct introduces an preliminary starting stage for extra complicated duties. With out this construction, an agent could instantly take the primary accessible motion, lose sight of the unique goal, and start making disconnected choices.
Reminiscence Layer
Reminiscence serves because the agent’s working pocket book. It shops the present activity state, vital outcomes from earlier steps, and data from earlier periods. RAG, information, databases, and devoted state shops could all assist this layer. With out managed reminiscence, each new request feels virtually like the primary one, and important context progressively disappears.
Abilities administration
Abilities administration defines which capabilities an agent can use and the way they need to be utilized. Every talent could embody directions, instruments, and validation guidelines for a particular activity. With out clear talent administration, the agent could choose unsuitable instruments, repeat actions, or full the identical activity inconsistently.
Subagent Administration and Handoffs
In multi-agent system design, specialised brokers resemble departments throughout the identical firm, with each answerable for a specific space of labor. AI agent orchestration determines which agent receives a activity, when it ought to cross the end result to a different agent, and the way separate outputs are mixed. With out clear coordination, brokers duplicate work, use conflicting variations of information, or ignore each other’s findings.
Loop and Execution Price range Limits
These limits act as an emergency cease that stops the system from repeating an unsuccessful motion indefinitely. They might cowl the variety of steps, retries, device calls, execution time, and price. With out them, an agent can change into trapped in a loop involving a paid API or proceed working after it’s not making progress.
Guardrails and Entry Permissions
Guardrails and permissions operate as security insurance policies and entry controls. They outline which information the agent could view, which instruments it might use, and which actions it’s approved to carry out. With out these restrictions, a mannequin error may end in a deleted file, an electronic mail despatched to the unsuitable recipient, or an operation the agent ought to by no means have been allowed to execute.
Approval Administration
Approval administration introduces checkpoints earlier than important actions. An agent could acquire info and put together a choice independently, nevertheless it ought to request human affirmation earlier than making a fee, deleting information, or sending an official doc. With no human within the loop, the system receives an excessive amount of autonomy in conditions the place the price of an error is particularly excessive.
Context and Software Schema Administration
The context window is just like a workspace: when it turns into overcrowded, vital info is more durable to determine. Context compaction removes repetition and retains essentially the most related particulars, whereas clear device schemas clarify how every device must be used. With out them, the agent turns into confused by gathered context and generates incorrect API calls.
Logging and Tracing
Logging information the agent’s whole execution path, together with the context it acquired, the actions it chosen, device responses, and the rationale the method ended. With out this historical past, the workforce sees solely the wrong final result and can’t determine the place the failure started. Diagnosing the agentic system then turns into guesswork, whereas recurring issues stay unresolved.
What Good Agentic System Design Seems Like in Follow
In apply, a strong agentic AI implementation doesn’t start with connecting an LLM. It begins with defining how the agent will obtain context, retain activity state, choose instruments, and reply to sudden conditions.
This may be illustrated by an AI agent constructed by SCAND for an actual property platform. As an AI agent improvement firm, SCAND needed to handle not solely the mannequin layer but additionally reminiscence (context layer), device orchestration, entry controls, and monitoring.
One of many principal challenges in such a system is reminiscence. Passing all the dialog historical past to the mannequin with each request shortly fills the context window and will increase processing prices. Storing too little info creates the alternative drawback: the agent forgets beforehand outlined circumstances and begins requesting the identical information once more. The AI agent reminiscence structure subsequently must protect activity state, compress lengthy histories, and supply the mannequin with solely the knowledge related to the present step.
Software administration is equally vital. When an agent has entry to dozens of features, it should decide which device to make use of, in what order to carry out actions, and easy methods to interpret the response. Orchestration and expertise administration buildings this course of and helps forestall the agent from choosing an unsuitable device, repeating a accomplished step, or counting on conflicting information.
AI agent guardrails on this state of affairs transcend filtering outputs. They limit entry to the system, outline permitted actions, and make the agent’s work extra clear. Customers and directors ought to have the ability to perceive which instruments are being referred to as and the place extra oversight is required.
A separate monitoring layer information the agent’s actions, device responses, and modifications in context. This permits the workforce to research not solely the ultimate reply but additionally all the path that led to it. That’s particularly vital in lengthy workflows, the place an error could happen a number of steps earlier than the wrong end result turns into seen. Monitoring permits alternatives for self-evolving characteristic assist.
This instance exhibits that agent reliability comes from the mixture of reminiscence, orchestration, constraints, and observability. The LLM stays a central element, however system stability relies on how persistently its work is managed across the mannequin.
Why Harness Design Is a Enterprise Resolution, Not Simply an Engineering One
The structure round an AI agent determines not solely the standard of its outputs but additionally the extent of operational threat. If the system doesn’t limit the mannequin’s actions, a single mistake could end in an electronic mail being despatched to the unsuitable recipient, deleted information, repeated calls to a paid API, or a choice based mostly on unverified info. Guardrails, entry permissions, and necessary approval for important operations should subsequently mirror the enterprise course of and the potential price of failure.
For many corporations, predictability issues greater than most execution pace. An agent can’t change into a part of an everyday workflow if, underneath the identical circumstances, it often selects a special device, skips a validation step, or fails to finish the duty. Harness design doesn’t make an LLM totally deterministic, nevertheless it narrows the vary of acceptable conduct by defining execution sequences, limits, validation standards, escalation guidelines, and stopping circumstances.
The return on funding additionally relies on structure. Enterprise worth doesn’t come merely from utilizing a extra succesful mannequin, however from making certain that the agent interacts reliably with information, individuals, and enterprise methods. Trade analysis identifies orchestration, centralized governance, role-based entry, tracing, and human-in-the-loop controls as important for scaling AI brokers. In a single enterprise use case, agentic workflows diminished improvement cycles by as much as 60% and reduce manufacturing errors by half.
Harness design ought to subsequently not be handled as a one-time configuration earlier than launch. It’s an ongoing engineering self-discipline, just like DevOps and safety — constructed into the system from the beginning and repeatedly revised as instruments, information, dangers, and enterprise necessities evolve.
The place to Begin Earlier than Constructing an AI Agent
Earlier than improvement begins, step one is to decide on a particular enterprise course of. The workforce ought to outline the duty the agent will carry out, the anticipated final result, and the conditions by which the agent shouldn’t be used. A broad purpose akin to “automate buyer assist” just isn’t sufficient; the preliminary scope have to be clear and measurable.
The subsequent step is to evaluate the price of failure. This determines the place the agent could act independently and the place its output have to be reviewed, authorised, or escalated to a human. The upper the potential influence of an error, the narrower the agent’s autonomy must be.
High quality standards must also be established earlier than implementation. These could embody response accuracy, execution time, price per activity, acceptable error charges, and circumstances underneath which the method should cease. With out predefined metrics, it’s troublesome to differentiate a profitable agent from a convincing demo.
Lastly, the system must be launched by a restricted pilot based mostly on actual eventualities. Testing ought to embody incomplete information, ambiguous requests, unavailable APIs, and sudden device responses. Solely after the agent behaves persistently underneath these circumstances ought to its tasks and entry be expanded.
The Actual Variable in AI Agent Efficiency
Selecting the best LLM issues, nevertheless it doesn’t assure that an AI agent will work reliably in manufacturing. Variations between GPT-5, Claude, and different fashions can have an effect on reasoning high quality, pace, and price, but the ultimate end result relies upon largely on the system constructed across the mannequin.
Reliability is formed by the standard of context, reminiscence structure, tool-use guidelines, intermediate validation, autonomy limits, and stopping circumstances. A highly effective LLM can’t compensate for a poorly structured course of, whereas considerate system design makes agent conduct extra constant, predictable, and controllable.
When constructing an agentic system, corporations ought to consider not solely the mannequin’s capabilities but additionally the way it will function inside actual enterprise processes. SCAND gives AI agent improvement providers for corporations that have to design, combine, and launch dependable agentic methods.
Incessantly Requested Questions (FAQs)
What Is Harness Design in AI Brokers?
Harness design is the engineering layer round an LLM that manages context, reminiscence, instruments, entry permissions, error dealing with, and stopping circumstances. It determines how the mannequin interacts with exterior methods and the way predictably the agent operates in real-world processes.
Why Do AI Brokers Fail in Manufacturing If They Work within the Demo?
A demo normally covers a restricted set of ready eventualities. In manufacturing, an agent encounters incomplete requests, outdated information, API failures, sudden device responses, and lengthy execution chains. These circumstances expose weaknesses in reminiscence, orchestration, validation, and system constraints.
What Is the Distinction Between ReAct and PlanReAct Structure?
ReAct alternates between reasoning, motion, and commentary at every stage. PlanReAct provides an preliminary planning step earlier than execution begins. ReAct is appropriate for comparatively easy sequential processes, whereas PlanReAct works higher for multi-step eventualities that contain uncertainty and require the plan to be adjusted.
How Can I Make My AI Agent Extra Dependable?
Outline clear autonomy limits, construction reminiscence and context, management device utilization, and validate intermediate outcomes. Reliability additionally relies on execution and price limits, human approval for important actions, and logging and tracing for each run.
Do I Want a Totally different LLM to Get Higher Outcomes From My AI Agent?
Not at all times. A extra succesful mannequin could enhance reasoning high quality, nevertheless it won’t repair issues with context, reminiscence, instruments, or stopping circumstances. The whole system across the LLM must be assessed first. In lots of circumstances, enhancing the structure has a better impact than switching fashions.
