Skip to content

Langfuse: observability for LLM applications

MLOps / LLMOps · Observability
Not studied

Added 2026-10-29.

Problem

The model's answer is bad, and nobody knows why: which prompt was sent, how much the request cost, at which step of the chain things broke. Ordinary logs don't show this.

Solution

Langfuse is "an open source LLM engineering platform. It helps teams collaboratively develop, monitor, evaluate, and debug AI applications" and "can be self-hosted in minutes". Self-hosted Langfuse gives request traces (including via OpenTelemetry), tokens and cost, datasets and quality evaluations. It connects to LiteLLM with one line.

Example

Start it with docker compose up from the repository. Before that:

  • replace every value marked # CHANGEME in docker-compose.yml: database passwords, SALT, ENCRYPTION_KEY (openssl rand -hex 32);
  • change the web UI port mapping 3000:3000 to 127.0.0.1:3000:3000 — otherwise on a VPS the port opens to the outside, bypassing the firewall; external access only through Nginx with TLS.

Sending traces from LiteLLM:

litellm_settings:
  success_callback: ["langfuse"]

For applications with OpenTelemetry — the /api/public/otel/v1/traces endpoint.

When to use

When an LLM application is in production and bad answers and costs need investigating.

Limitations

It needs Postgres, ClickHouse, Redis and S3-compatible storage — heavy for a small VPS. Traces contain prompts and responses: configure retention and access. The ee/ directories are under a separate license.

Source

How to apply it yourself

  1. Run it on a test machine with replaced secrets and the port bound to 127.0.0.1.
  2. Send traces from LiteLLM.
  3. Configure retention and check that old traces get deleted.
  4. Mark it "Studied" once a real bad answer can be investigated from its trace.