Amazon OpenSearch Service now brings software monitoring, native Amazon Managed Service for Prometheus integration, and AI agent tracing collectively in OpenSearch UI‘s observability workspace. You’ll be able to question Prometheus metrics with PromQL alongside logs and traces saved in Amazon OpenSearch Service, hint an AI agent’s full reasoning chain all the way down to the failing software name, and drill from a service-level well being view to the precise span that precipitated a checkout failure, all with out leaving the interface.
On this submit, we stroll via two real-world eventualities utilizing the OpenTelemetry pattern app: a multi-agent journey planner dealing with gradual processing, and a checkout move quietly failing on one microservice. We chase every one to its root trigger utilizing these new capabilities.
State of affairs 1: An underperforming AI agent
Your multi-agent journey planner is dwell and customers begin reporting gradual responses. With the brand new AI agent tracing functionality in Amazon OpenSearch Service, you’ll be able to hint the agent’s full processing path to pinpoint precisely the place issues went improper.
In any observability workspace in OpenSearch UI, navigate to Utility Map within the left navigation pane.
You’ll be able to see the total topology of your system together with the journey agent and the sub-agents it calls. The journey agent node reveals elevated latency and occasional errors. Choose it, and the aspect panel confirms that latency is up however the latency chart reveals intermittent spikes fairly than constant degradation.
The appliance map tells you one thing is improper, however understanding why an AI agent is underperforming requires seeing its reasoning chain. Choose Agent Traces within the left navigation pane, then filter by service title and time vary.
Choose one of many traces to see the hint tree. Not like a conventional span waterfall, this view organizes across the agent’s reasoning chain: the foundation agent span, the LLM calls it made, the instruments it invoked, and the way they nested every step color-coded by kind. The hint map supplies a visible directed graph of the identical execution. You’ll be able to see which mannequin was known as, what number of enter and output tokens have been consumed, and the precise messages despatched to and obtained from the mannequin.
A software name contained in the climate agent errored out. The agent then spent further time reasoning in regards to the failure earlier than returning a partial response explaining the intermittent latency spikes and occasional faults.
Why this issues for AI brokers
Brokers make autonomous selections primarily based on LLM responses, software outcomes, and chained reasoning. Not like conventional microservices with deterministic code paths, agent habits varies throughout executions. With out semantic tracing that captures these AI-specific alerts, root-cause evaluation is guesswork. The hint tree surfaced the mannequin title, token counts, and failing software name as a result of the journey planner was instrumented with OpenTelemetry’s generative AI semantic conventions. The following part describes how.
Instrumenting AI brokers
OpenTelemetry auto-instrumentation enriches spans with well-known attributes for HTTP, database, and gRPC calls. AI brokers want a distinct set of attributes akin to which LLM was known as, what tokens have been consumed, which instruments have been invoked, that commonplace instrumentation doesn’t cowl.
The OpenTelemetry gen_ai semantic conventions outline commonplace attributes for these alerts, together with gen_ai.operation.title, gen_ai.utilization.input_tokens, gen_ai.request.mannequin, and gen_ai.software.title. When Amazon OpenSearch Service receives spans with these attributes, it categorizes them by operation kind (agent, LLM, software, embeddings, retrieval) and renders the agent hint tree and hint map views.
The Python SDK supplies one technique to generate these spans. To ship traces to Amazon OpenSearch Ingestion, configure the SDK with AWS Signature Model 4 (SigV4) authentication. The AWSSigV4OTLPExporter cryptographically indicators every HTTP request to assist forestall unauthorized knowledge ingestion. The calling identification wants an IAM coverage that grants osis:Ingest in your pipeline’s ARN. Credentials are resolved via the usual AWS credential supplier chain.
from opensearch_genai_observability_sdk_py import register, AWSSigV4OTLPExporter
exporter = AWSSigV4OTLPExporter(
endpoint="https://pipeline.us-east-1.osis.amazonaws.com/v1/traces",
service="osis",
area="us-east-1",
)
register(service_name="my-agent", exporter=exporter)
Use the @observe decorator to hint agent features and enrich() so as to add mannequin metadata:
@observe(op=Op.EXECUTE_TOOL)
def get_weather(metropolis: str) -> dict:
return {"metropolis": metropolis, "temp": 22, "situation": "sunny"}
@observe(op=Op.INVOKE_AGENT)
def assistant(question: str) -> str:
enrich(mannequin="gpt-4o", supplier="openai")
knowledge = get_weather("Paris")
return f"{knowledge['condition']}, {knowledge['temp']}C"
end result = assistant("What is the climate?")
The SDK additionally helps auto-instrumentation for OpenAI, Anthropic, Amazon Bedrock, LangChain, LlamaIndex, and others. As a result of the instrumentation is constructed on OpenTelemetry requirements, any agent framework that emits spans with gen_ai.* attributes is suitable with OpenSearch UI.
State of affairs 2: Investigating a microservice problem
AI brokers are just one a part of most manufacturing environments. The identical interface surfaces telemetry from typical microservices, the place the troubleshooting workflow follows a extra acquainted path.
Your ecommerce checkout begins paging throughout a busy site visitors window. From OpenSearch UI, navigate to APM Providers within the left navigation pane. Each instrumented service is listed alongside its well being indicators. The checkout service reveals an elevated error charge.
Choose the affected service. The element view reveals Request, Error, and Period (RED) metrics: request charge is climbing, fault charge has spiked within the final quarter-hour, and p99 period has doubled. You’ll be able to see precisely when the degradation began.
Drill into the correlated spans for the affected time window. The span checklist reveals a number of failed requests, all hitting the identical endpoint. Choose one to see the total hint waterfall. The checkout service known as prepareOrder, which failed making an attempt to retrieve a product from the catalog. The error message within the span particulars tells you precisely what went improper, that’s your root trigger.
Checking the infrastructure with PromQL
In each eventualities, the pure subsequent query is whether or not the issue originates within the software or within the infrastructure beneath it. With the brand new Amazon Managed Service for Prometheus integration, you’ll be able to reply that query with out leaving OpenSearch UI.
Prometheus metrics at the moment are queryable straight from the identical workspace utilizing native PromQL syntax, alongside the logs and traces you’ve already been navigating.
For the database timeout in State of affairs 2, run a PromQL question to verify the database occasion’s learn/write throughput for a similar time window. For the agent latency problem in State of affairs 1, verify the LLM endpoint’s response time metrics to see if the slowness originates from the mannequin supplier.
It is a key architectural resolution: metrics proceed to dwell in Amazon Managed Service for Prometheus, logs and traces proceed to dwell in Amazon OpenSearch Service, and neither sign is copied or warehoused right into a second retailer. Every backend stays the only retailer for the info kind it’s purpose-built to deal with, whereas OpenSearch UI federates queries throughout each at runtime. The price, retention, and operational mannequin of every retailer keep intact whereas the troubleshooting workflow collapses right into a single interface.
To configure the OpenTelemetry Collector and OpenSearch Ingestion pipelines that route metrics into Amazon Managed Service for Prometheus, see Ingesting software telemetry.
The way it’s wired collectively
The next diagram reveals the end-to-end structure. Functions instrumented with OpenTelemetry ship traces, logs, and metrics over OTLP to Amazon OpenSearch Ingestion. OpenSearch Ingestion routes every sign to the suitable retailer: traces and logs land in Amazon OpenSearch Service, whereas metrics move into Amazon Managed Service for Prometheus. OpenSearch UI then queries each shops to render the Utility Map, Providers catalog, Agent Traces, and Metrics views.
Your complete expertise rests on open-source foundations, Prometheus for metrics, OpenSearch for logs and traces, and OpenTelemetry for instrumentation, so groups already working an OpenTelemetry collector can undertake it by updating the collector’s export configuration to level at Amazon OpenSearch Ingestion, with no proprietary brokers or rewritten instrumentation required.
Getting began
To allow these capabilities, log in to OpenSearch UI’s observability workspace, choose the Gear icon within the backside left nook to open Settings and setup, and confirm that the Observability:apmEnabled toggle is on below the Observability part. OpenSearch UI is offered at no further cost for Amazon OpenSearch Service prospects.
Discover domestically first. The OpenSearch Observability Stack offers you a completely configured setting together with software monitoring, agent tracing, and Prometheus integration, working in your machine with a single set up command. It ships with pattern instrumented providers, together with a multi-agent journey planner, so you’ll be able to discover the total workflow with actual telemetry knowledge out of the field.
For AI agent improvement. Agent Well being is an open-source, evaluation-driven observability software designed for native improvement. It offers you execution move graphs, token monitoring, and power invocation visibility proper in your improvement loop, earlier than you push to manufacturing.
For manufacturing. The Python SDK supplies one-line setup and decorator-based tracing with gen_ai semantic conventions, with auto-instrumentation assist for OpenAI, Anthropic, Amazon Bedrock, LangChain, LlamaIndex, and others. See the Amazon OpenSearch Service documentation and the Amazon Managed Service for Prometheus integration information for the total managed expertise.
Concerning the authors
