Getting Began with Langfuse [2026 Guide]

The creation and deployment of purposes that make the most of Massive Language Fashions (LLMs) comes with their very own set of issues. LLMs have non-deterministic nature, can generate believable however false data and tracing their actions in convoluted sequences might be very troublesome. On this information, we’ll see how Langfuse comes up as an important instrument for fixing these issues, by providing a powerful basis for complete observability, evaluation, and immediate dealing with of LLM purposes.

What’s Langfuse?

Langfuse is a groundbreaking observability and evaluation platform that’s open supply and particularly created for LLM purposes. It’s the basis for tracing, viewing, and debugging all of the phases of an LLM interplay, ranging from the preliminary immediate and ending with the ultimate response, whether or not it’s a easy name or a sophisticated multi-turn dialog between brokers.

Langfuse will not be solely a logging software but additionally a method of systematically evaluating LLM efficiency, A/B testing of prompts, and gathering person suggestions which in flip helps to shut the suggestions loop important for iterative enchancment. The primary level of its worth is the transparency that it brings to the LLMs world, thus letting the builders to: 

• Perceive LLM behaviour: Discover out the precise prompts that have been despatched, the responses that have been acquired, and the intermediate steps in a multi-stage software. 
• Discover points: Find the supply of errors, low efficiency, or surprising outputs quickly. 
• High quality analysis: Effectiveness of LLM responses might be measured in opposition to the pre-defined metrics with each handbook and automatic measures. 
• Refine and enhance: Knowledge-driven insights can be utilized to good prompts, fashions, and software logic.
• Deal with prompts: management the model of prompts and take a look at them to get the perfect LLM.

Key Options and Ideas

There are numerous key options that Langfuse affords like: 

1. Tracing and Monitoring 

Langfuse helps us capturing the detailed traces of each interplay that LLM has. The ‘hint’ is mainly the illustration of an end-to-end person request or software move. Inside a hint, logical items of labor is denoted by “spans” and calls to an LLM refers to “generations”.

2. Analysis 

Langfuse permits analysis each manually and programmatically as nicely. Customized metrics might be outlined by the builders which might then be used to run evaluations for various datasets after which be built-in as LLM-based evaluators.

3. Immediate Administration 

Langfuse gives direct management over immediate administration together with storage and versioning capabilities. It’s attainable to check varied prompts via A/B testing and on the similar time preserve accuracy throughout numerous locations, which paves the best way for data-driven immediate optimization as nicely.  

4. Suggestions Assortment 

Langfuse absorbs the person solutions and incorporates them proper into your traces. It is possible for you to to hyperlink explicit remarks or person scores to the exact LLM interplay that resulted in an output, thus giving us the real-time suggestions for troubleshooting and enhancing.  

Why Langfuse? The Drawback It Solves

Conventional software program observability instruments have very completely different traits and don’t fulfill the LLM-powered purposes standards within the following features: 

• Non-determinism: LLMs is not going to all the time produce the identical consequence even for an equivalent enter which makes debugging fairly difficult. Langfuse, in flip, data every interplay’s enter and output giving a transparent image of the operation at that second. 
• Immediate Sensitivity: Any minor change in a immediate may alter LLM’s reply fully. Langfuse is there to assist holding observe of immediate variations together with their efficiency metrics. 
• Advanced Chains: The vast majority of LLM purposes are characterised by a mixture of a number of LLM calls, completely different instruments, and retrieving information (e.g., RAG architectures). The one method to know the move and to pinpoint the place the place the bottleneck or the error is the tracing. Langfuse presents a visible timeline for these interactions. 
• Subjective High quality: The time period “goodness” for an LLM’s reply is commonly synonymous with private opinion. Langfuse permits each goal (e.g., latency, token rely) and subjective (human suggestions, LLM-based analysis) high quality assessments. 
• Price Administration: Calling LLM APIs comes with a worth. Understanding and optimizing your prices can be simpler you probably have Langfuse monitoring your token utilization and name quantity. 
• Lack of Visibility: The developer will not be in a position to see how their LLM purposes are performing in the marketplace and due to this fact it’s onerous for them to make these purposes steadily higher due to the dearth of observability. 

Langfuse doesn’t solely supply a scientific methodology for LLM interplay, nevertheless it additionally transforms the event course of right into a data-driven, iterative, engineering self-discipline as a substitute of trial and error. 

Getting Began with Langfuse

Earlier than you can begin utilizing Langfuse, you should first set up the shopper library and set it as much as transmit information to a Langfuse occasion, which might both be a cloud-hosted or a self-hosted one. 

Set up

Langfuse has shopper libraries obtainable for each Python and JavaScript/TypeScript. 

Python Shopper 

pip set up langfuse 

JavaScript/TypeScript Shopper 

npm set up langfuse 

Or 

yarn add langfuse 

Configuration 

After set up, keep in mind to arrange the shopper together with your undertaking keys and host. Yow will discover these in your Langfuse undertaking settings.   

• public_key: That is for the frontend purposes or for circumstances the place solely restricted and non-sensitive information are getting despatched.
• secret_key: That is for backend purposes and eventualities the place the total observability, together with delicate inputs/outputs, is a requirement.   
• host: This refers back to the URL of your Langfuse occasion (e.g., https://cloud.langfuse.com).   
• setting: That is an non-compulsory string that can be utilized to differentiate between completely different environments (e.g., manufacturing, staging, growth).   

For safety and suppleness causes, it’s thought-about good observe to outline these as setting variables.

export LANGFUSE_PUBLIC_KEY="pk-lf-..." 
export LANGFUSE_SECRET_KEY="sk-lf-..." 
export LANGFUSE_HOST="https://cloud.langfuse.com" 
export LANGFUSE_ENVIRONMENT="growth"

Then, initialize the Langfuse shopper in your software: 

Python Instance 

from langfuse import Langfuse
import os

langfuse = Langfuse(public_key=os.environ.get("LANGFUSE_PUBLIC_KEY"),    secret_key=os.environ.get("LANGFUSE_SECRET_KEY"),    host=os.environ.get("LANGFUSE_HOST"))

JavaScript/TypeScript Instance 

import { Langfuse } from "langfuse";

const langfuse = new Langfuse({  publicKey: course of.env.LANGFUSE_PUBLIC_KEY,  secretKey: course of.env.LANGFUSE_SECRET_KEY,  host: course of.env.LANGFUSE_HOST});

Establishing Your First Hint

The basic unit of observability in Langfuse is the hint. A hint usually represents a single person interplay or an entire request lifecycle. Inside a hint, you log particular person LLM calls (era) and arbitrary computational steps (span). 

Let’s illustrate with a easy LLM name utilizing OpenAI’s API. 

Python Instance 

import os
from openai import OpenAI
from langfuse import Langfuse
from langfuse.mannequin import InitialGeneration

# Initialize Langfuse
langfuse = Langfuse(
    public_key=os.environ.get("LANGFUSE_PUBLIC_KEY"),
    secret_key=os.environ.get("LANGFUSE_SECRET_KEY"),
    host=os.environ.get("LANGFUSE_HOST"),
)

# Initialize OpenAI shopper
shopper = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def simple_llm_call_with_trace(user_input: str):
    # Begin a brand new hint
    hint = langfuse.hint(
        title="simple-query",
        enter=user_input,
        metadata={"user_id": "user-123", "session_id": "sess-abc"},
    )

    strive:
        # Create a era inside the hint
        era = hint.era(
            title="openai-generation",
            enter=user_input,
            mannequin="gpt-4o-mini",
            model_parameters={"temperature": 0.7, "max_tokens": 100},
            metadata={"prompt_type": "commonplace"},
        )

        # Make the precise LLM name
        chat_completion = shopper.chat.completions.create(
            mannequin="gpt-4o-mini",
            messages=[{"role": "user", "content": user_input}],
            temperature=0.7,
            max_tokens=100,
        )

        response_content = chat_completion.selections[0].message.content material

        # Replace era with the output and utilization
        era.replace(
            output=response_content,
            completion_start_time=chat_completion.created,
            utilization={
                "prompt_tokens": chat_completion.utilization.prompt_tokens,
                "completion_tokens": chat_completion.utilization.completion_tokens,
                "total_tokens": chat_completion.utilization.total_tokens,
            },
        )

        print(f"LLM Response: {response_content}")
        return response_content

    besides Exception as e:
        # Report errors within the hint
        hint.replace(
            stage="ERROR",
            status_message=str(e)
        )
        print(f"An error occurred: {e}")
        elevate

    lastly:
        # Guarantee all information is distributed to Langfuse earlier than exit
        langfuse.flush()


# Instance name
simple_llm_call_with_trace("What's the capital of France?")

Finally, the next step after executing this code can be to go to the Langfuse interface. There can be a brand new hint “simple-query” that consists of 1 era “openai-generation”. It’s attainable so that you can click on it in an effort to view the enter, output, mannequin used, and different metadata. 

Core Performance in Element

Studying to work with hint, span, and era objects is the principle requirement to make the most of Langfuse. 

Tracing LLM Calls

• langfuse.hint(): This command begins a brand new hint. The highest-level container for a complete operation. 
  • title: The hint’s very descriptive title.  
  • enter: The primary enter of the entire process.  
  • metadata: A dictionary of any key-value pairs for filtering and evaluation (e.g., user_id, session_id, AB_test_variant).  
  • session_id: (Optionally available) An identifier shared by all traces that come from the identical person session.  
  • user_id: (Optionally available) An identifier shared by all interactions of a specific person.  

• hint.span(): This can be a logical step or minor operation inside a hint that isn’t a direct input-output interplay with the LLM. Instrument calls, database lookups, or complicated calculations might be traced on this approach. 
  • title: Title of the span (e.g. “retrieve-docs”, “parse-json”).  
  • enter: The enter related to this span.  
  • output: The output created by this span.  
  • metadata: The span metadata is formatted as further.  
  • stage: The severity stage (INFO, WARNING, ERROR, DEBUG).  
  • status_message: A message that’s linked to the standing (e.g. error particulars).  
  • parent_observation_id: Connects this span to a mother or father span or hint for nested buildings. 

• hint.era(): Signifies a specific LLM invocation. 
  • title: The title of the era (as an example, “initial-response”, “refinement-step”).  
  • enter: The immediate or messages that have been communicated to the LLM.  
  • output: The reply acquired from the LLM.  
  • mannequin: The exact LLM mannequin that was employed (for instance, “gpt-4o-mini“, “claude-3-opus“).  
  • model_parameters: A dictionary of explicit mannequin parameters (like temperature, max_tokens, top_p).  
  • utilization: A dictionary displaying the variety of tokens utilized (prompt_tokens, completion_tokens, total_tokens).  
  • metadata: Further metadata for the LLM invocation.  
  • parent_observation_id: Hyperlinks this era to a mother or father span or hint.  
  • immediate: (Optionally available) Can determine a specific immediate template that’s underneath administration in Langfuse. 

Conclusion

Langfuse makes the event and maintenance of LLM-powered purposes a much less strenuous endeavor by turning it right into a structured and data-driven course of. It does this by giving builders entry to the interactions with the LLM like by no means earlier than via intensive tracing, systematic analysis, and highly effective immediate administration.  

Furthermore, it encourages the builders to debug their work with certainty, velocity up the iteration course of, and carry on enhancing their AI merchandise by way of high quality and efficiency. Therefore, Langfuse gives the mandatory devices to make it possible for LLM purposes are reliable, cost-effective, and actually highly effective, regardless of in case you are growing a fundamental chatbot or a complicated autonomous agent. 

Steadily Requested Questions

Knowledge Science Trainee at Analytics Vidhya
I’m at present working as a Knowledge Science Trainee at Analytics Vidhya, the place I give attention to constructing data-driven options and making use of AI/ML methods to resolve real-world enterprise issues. My work permits me to discover superior analytics, machine studying, and AI purposes that empower organizations to make smarter, evidence-based choices.
With a powerful basis in pc science, software program growth, and information analytics, I’m obsessed with leveraging AI to create impactful, scalable options that bridge the hole between know-how and enterprise.
📩 It’s also possible to attain out to me at [email protected]