Most agent tooling is Python-first. LangChain, AutoGen, CrewAI, and LangGraph all goal Python. Provided that Python is the second-most-popular programming language, the present ecosystem would possibly work effectively for groups already utilizing it. Nonetheless, organizations operating JVM infrastructure or Erlang/OTP techniques face the query of whether or not to maneuver brokers to Python or construct them within the runtime they already function.
As ambassadors of purposeful programming, we have been toying with the agentic techniques in our languages of selection, Elixir and Clojure. This text, which partially summarizes our earlier endeavors, compares them with Python and examines how every handles the particular necessities of manufacturing agent techniques.
Brokers? What are these?
However let’s begin with trivia for many who want it. An LLM agent combines a language mannequin with the flexibility to name capabilities. The core loop — typically known as ReAct (Reasoning and Appearing) — works like this: the LLM examines the dialog and obtainable instruments, decides whether or not to name a device or reply, and if it calls a device, the end result will get fed again into the dialog. The loop continues till the agent produces a closing reply or hits a step restrict.
Anthropic distinguishes workflows (LLMs orchestrated by way of predefined code paths) and brokers (LLMs that dynamically direct their very own processes and gear utilization). Each observe the identical fundamental loop. The distinction is in how a lot the LLM controls the sequencing.
What varies throughout languages is the way you characterize instruments, state and the loop itself.
The stub agent in three languages
We’ll use a easy analytic agent because the comparability level. It would question the database to, let’s say, return statistics on weekly customers, optionally producing charts if requested.
Python
Python offers us with prepared frameworks for spinning up brokers. We can not omit them, although we may also write Python brokers from scratch.
LangChain
```python
from langchain_openai import ChatOpenAI
from langchain.brokers import initialize_agent, Software
def run_sql(question: str):
...
llm = ChatOpenAI(mannequin="gpt-4.1-mini")
instruments = [
Tool(name="run_sql", func=run_sql,
description="Run an SQL query on the analytics db.")
]
agent = initialize_agent(
instruments=instruments, llm=llm,
agent="zero-shot-react-description", verbose=True,
)
end result = agent.run("What number of lively customers did now we have final week?")
```
The agent loop runs inside `initialize_agent`. State and hint are accessed by way of framework APIs. Instruments are `Software` class cases.
With out a framework
```python
TOOLS = {
"run_sql": {"run": run_sql},
"render_chart": {"run": render_chart},
}
def run_agent(query: str) -> dict:
state = {
"dialog": [{"role": "user", "content": question}],
"hint": [],
}
determination = call_llm(state["conversation"], TOOLS)
if determination["type"] == "tool_call":
tool_name = determination["tool"]
params = determination["params"]
end result = TOOLS[tool_name]["run"](params)
state["conversation"].append({
"function": "device", "title": tool_name,
"content material": repr({"params": params, "end result": end result}),
})
state["trace"].append({
"step": 1, "device": tool_name,
"params": params, "end result": end result
})
return state
```
Instruments are dictionaries. State is a dictionary. The management circulate is seen. This model is testable in the identical approach because the Clojure model beneath. The trade-off right here is that Python’s mutable information buildings imply {that a} device perform can modify `state` by way of a reference with out that modification exhibiting up within the hint. Some would argue that such behaviour is a language flaw; we imagine that it’s a property to handle.
Clojure
Clojure represents the agent as information transformations on immutable maps.
Software definitions
```clojure
(def run-sql-tool
{:title "run_sql"
:description "Run an SQL question on the analytics db"
:params [:map [:query string?]]
:run (fn [{:keys [query]}]
(db/run-sql question))})
(def instruments
{"run_sql" run-sql-tool
"render_chart" render-chart-tool})
```
Instruments are maps. Parameter schemas use Malli, which defines schemas as information buildings fairly than courses or decorators. It means schemas may be programmatically generated, serialized and remodeled, which is helpful when changing to the JSON format that LLM APIs anticipate.
The agent loop
```clojure
(defn run-agent-once [state config]
(let [decision (llm/call-llm-with-tools
(:model config) (:api-key config)
tools/tools (:conversation state))]
(case (:sort determination)
:message
{:state (append-message state "assistant" (:content material determination))
:finished? true}
:tool-call
(let [{:keys [tool params]} determination
tool-def (get instruments/instruments device)
params' (instruments/validate-params tool-def params)
end result ((:run tool-def) params')]
{:state (append-tool-result state device params' end result)
:finished? false}))))
(defn run-agent [user-question config]
(loop [state (initial-state user-question)
steps 0]
(let [{:keys [state done?]} (run-agent-once state config)]
(if (or finished? (>= steps (:max-steps config 8)))
state
(recur state (inc steps))))))
```
Every iteration takes a state and returns a brand new state. The previous state is unchanged. It means you possibly can diff two states to see what a particular iteration modified. You possibly can serialize the total state to EDN, reserve it and replay execution later. Throughout improvement, the REPL enables you to name `run-agent-once` with a captured state and step by way of execution manually.
Testing
```clojure (deftest agent-produces-trace (let [state (core/run-agent "How many active users?" config)] (is (= 1 (depend (:hint state)))) (is (= "run_sql" (-> state :hint first :device))))) ```
You name a perform and assert on the returned map. The stub LLM makes habits deterministic. No mocking libraries are wanted as a result of there are not any framework internals to mock.
Elixir
Elixir fashions every agent as a course of utilizing the Actor Mannequin. Processes are light-weight (kilobytes of reminiscence), talk by way of message passing, and are supervised for fault restoration.
Agent as a GenServer
```elixir
defmodule AnalyticsAgent do
use GenServer
def start_link(opts) do
GenServer.start_link(__MODULE__, opts)
finish
def init(opts) do
{:okay, %{
dialog: [],
hint: [],
instruments: %{
"run_sql" => &Instruments.run_sql/1,
"render_chart" => &Instruments.render_chart/1
}
}}
finish
def handle_call({:ask, query}, _from, state) do
state = update_in(state.dialog, &[%{role: "user", content: question} | &1])
{end result, new_state} = run_loop(state, max_steps: 8)
{:reply, end result, new_state}
finish
defp run_loop(state, opts) do
case LLM.call_with_tools(state.dialog, state.instruments) do
{:message, content material} ->
{content material, append_message(state, "assistant", content material)}
{:tool_call, device, params} ->
end result = state.instruments[tool].(params)
new_state = append_tool_result(state, device, params, end result)
run_loop(new_state, opts)
finish
finish
finish
```
The message-passing mannequin maps straight to straightforward agent workflow patterns. Immediate chaining is processes passing messages ahead. Routing is a classifier course of dispatching to specialised agent processes. An orchestrator course of spawns and manages employee processes. A number of agent processes run concurrently by default as a result of that’s the core Elixir’s supply.
Supervision
```elixir
defmodule AgentSupervisor do
use Supervisor
def init(_opts) do
youngsters = [
{AnalyticsAgent, name: :analytics},
{CodeGenAgent, name: :codegen},
{ReviewAgent, name: :review}
]
Supervisor.init(youngsters, technique: :one_for_one)
finish
finish
```
If one agent course of crashes (because of dangerous LLM output, API timeout, or malformed device end result), the supervisor restarts it. The opposite agent processes are unaffected. On this approach, Erlang/OTP has dealt with course of failures because the Nineteen Eighties; this method applies to LLM brokers with out modification.
How every runtime handles manufacturing necessities
Parallel Processing
Python makes use of `asyncio`, threading, or multiprocessing. The GIL limits CPU-bound parallelism. For I/O-bound agent work (which most LLM API calls are), `asyncio` works adequately. For CPU-bound work or massive numbers of concurrent brokers, exterior instruments like Ray or Celery are frequent.
Clojure has concurrency primitives (atoms, refs, brokers, core.async) and runs on JVM threads. Operating a number of brokers concurrently requires express use of those primitives however is well-supported.
Elixir runs light-weight processes on the BEAM VM with preemptive scheduling. A single machine can run hundreds of thousands of processes distributed throughout all CPU cores. Operating brokers concurrently requires no particular setup; you simply begin processes.
State administration
Python state is mutable by default. In framework-based brokers, state is usually inside to class cases. In plain-code brokers, the state is in dictionaries that may be mutated from anyplace with a reference. Traceability is dependent upon logging self-discipline.
Clojure state is immutable. Every agent iteration produces a brand new state map with out modifying the earlier one. States may be diffed, serialized, saved, and replayed. The REPL permits direct inspection of any intermediate state throughout improvement.
Elixir processes have an remoted state — every course of maintains its personal state that different processes can not straight entry. It prevents unintentional state corruption throughout brokers. Inspection is obtainable by way of `:sys.get_state/1` and `:observer`, however the mannequin is process-centric fairly than data-centric.
Fault tolerance
Python offers strive/besides. Retry logic and circuit breakers are applied manually or through libraries. Agent frameworks fluctuate in how they deal with failures — some have retry mechanisms, others depart it to the developer.
Clojure inherits JVM exception dealing with. Supervision patterns may be constructed utilizing libraries, however the language and runtime don’t present them natively.
Elixir has supervision bushes as a core runtime characteristic. Supervisors monitor processes and restart them in response to configurable methods. This method has been the usual in Erlang/OTP techniques for many years and applies on to agent processes.
Distribution
Python requires exterior infrastructure (Kubernetes, Celery, Ray) for distributing brokers throughout machines. Coordination protocols should be added individually.
Clojure can use JVM clustering options. The Agent-o-Rama library offers distributed agent execution on Rama. Distribution isn’t constructed into the language however is obtainable by way of the JVM ecosystem.
Elixir inherits Erlang’s clustering. Message passing between processes works the identical approach whether or not processes are on the identical machine or totally different machines. You possibly can develop on one machine and scale to a cluster with out altering the agent code.
Ecosystem and library help
Python has the most important AI ecosystem. Each main LLM supplier ships a Python SDK. Agent frameworks, embedding libraries, vector retailer integrations, and analysis instruments are all Python-first. Should you want a particular integration, it’s in all probability already obtainable in Python.
Clojure has a smaller ecosystem for AI-specific libraries. OpenAI and Anthropic API shoppers exist. The JVM offers entry to Java libraries. For a lot of integrations, you’ll write wrapper code.
Elixir has an rising AI ecosystem—Nx for numerical computing, Bumblebee for mannequin inference, Teacher for structured outputs. LLM API integrations exist however are much less complete than Python’s.
Testing
Python testing is dependent upon the method. Plain-code brokers (instruments as dictionaries, state as dictionaries) take a look at the identical approach as another Python code. Framework-based brokers typically require mocking framework internals, which {couples} checks to the framework’s implementation.
Clojure testing follows straight from the data-oriented design. Name the perform and verify the returned map. Swap in a stub LLM, run the agent, assert on the hint, and no particular take a look at infrastructure.
Elixir testing makes use of ExUnit with process-based isolation. Testing particular person brokers is easy. Testing interactions between concurrent brokers requires extra setup to deal with asynchronous message passing.
Documentation and AI Context
Brokers want structured details about the capabilities they will name and the info varieties they work with.
Elixir treats documentation as a first-class language characteristic. `@doc`, `@moduledoc`, and `@spec` annotations are a part of the usual workflow. These present sort signatures, utilization examples, and hierarchical descriptions that an AI agent can learn to know a module earlier than utilizing it. Documentation examples may be run as checks to maintain them updated.
Clojure has docstrings and specs (clojure.spec). Malli schemas serve each as validation and as documentation. Since schemas are information, brokers can examine them programmatically.
Python has docstrings and kind hints. Kind hints are optionally available and never enforced at runtime by default (instruments like mypy add static checking). The data is obtainable, however it’s much less persistently structured throughout the ecosystem.
When to make use of which
Python is sensible whenever you want particular AI library integrations, your staff already works in Python, and also you deal with concurrency and fault tolerance by way of infrastructure or exterior instruments.
Clojure is sensible whenever you’re on the JVM, you need agent state to be inspectable and replayable, and you like testing brokers as pure information transformations. It matches when you want to perceive and audit agent habits after the very fact.
Elixir is sensible when you want to run many brokers concurrently with automated fault restoration, and also you need distribution as a built-in runtime functionality. It matches techniques the place a number of brokers coordinate in actual time and the place particular person agent failures shouldn’t have an effect on the remainder of the system.
These are usually not mutually unique. A company may prototype brokers in Python for quick iteration on prompts and gear design, then implement the manufacturing orchestration layer in Elixir or Clojure, relying on whether or not the first operational concern is concurrency or traceability.
SD Instances Q&A:
How does Elixir’s GenServer sample work for LLM agent loops?
Every LLM agent is modeled as an Elixir GenServer course of with remoted state. The agent receives a query through message passing, runs a recursive tool-call loop utilizing sample matching, and returns the ultimate end result. If the method crashes because of a foul LLM response or API timeout, an OTP Supervisor mechanically restarts it with out affecting different agent processes.
What are the tradeoffs of utilizing Clojure for AI agent state administration vs. Python?
Clojure’s immutable information buildings imply every agent iteration produces a brand new state map, leaving the earlier one unchanged. This lets you diff states, serialize them to EDN, and replay execution — helpful for auditing agent habits. Python’s mutable dictionaries are easier however enable any perform holding a reference to silently modify state, which may complicate debugging and tracing.
Does Python’s GIL have an effect on LLM agent efficiency?
For many LLM agent workloads, that are I/O-bound (ready on API responses), Python’s World Interpreter Lock (GIL) has minimal affect and asyncio handles concurrency adequately. The GIL turns into a bottleneck for CPU-bound parallel work or very excessive numbers of concurrent brokers, through which case exterior instruments like Ray or Celery are usually used.
Which language is finest for operating many LLM brokers concurrently in manufacturing?
Elixir is the strongest match for high-concurrency agent techniques. Its BEAM VM runs light-weight processes (on the order of kilobytes of reminiscence every) with preemptive scheduling throughout all CPU cores, and distribution throughout machines works with the identical message-passing mannequin as native processes. Python and Clojure require further infrastructure or express concurrency primitives to realize comparable scale.
