Add Langfuse traces

[hanna-paasivirta]

Short Description

Adds Langfuse observability to all three chat services (global_chat, job_chat, workflow_chat) with per-user opt-in control.

Fixes #410

Requested Lightning changes: OpenFn/lightning#4595
We need to figure out self-hosting: #439

Implementation Details

Tracing

Every chat request now produces a structured trace in Langfuse — the full call tree from the top-level main() through routing, planning, subagent calls, RAG retrieval, and Anthropic API calls. Multi-turn conversations are grouped into Langfuse sessions via session_id. Anthropic SDK calls are auto-instrumented via OpenTelemetry.

Tracking controls

• Tracing is opt-in per request via a metrics_opt_in boolean in the payload. When absent or false, a should_export_span filter drops all spans before they leave the process — zero data sent to Langfuse.
• Exception: global_chat is temporarily force-tracked regardless of metrics_opt_in (one-line removal when we're ready to rely on the flag).
• When job_chat/workflow_chat are called as subagents from global_chat, they inherit the parent's tracing context automatically — no need to pass metrics_opt_in through subagent payloads.

User identity

A new user object in the payload carries id (mapped to Langfuse user_id) and employee (added as a Langfuse tag). This lets us filter traces by user and segment employee vs external usage.

API key safety

The @observe decorator on main() was capturing the entire input dict — including api_key — as the trace input. Fixed by adding capture_input=False and explicitly setting only the user's message as the span input.

Payload changes (frontend action needed)

Three new optional fields on all services:

{
  "user": { "id": "user-abc", "employee": true },
  "metrics_opt_in": true,
  "meta": { "session_id": "..." }
}

meta.session_id is new and consistent across all three services (global_chat previously used metadata — renamed to meta for consistency).

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

• Code generation (copilot but not intellisense)
• Learning or fact checking
• Strategy / design
• Optimisation / refactoring
• Translation / spellchecking / doc gen
• Other
• I have not used AI

You can read more details in our Responsible AI Policy

[josephjclark]

work out how to remove API keys from what we send to langfuse. Any other sensitive stuff?

[josephjclark]

next steps here:

• fix the api key thing
• Use an env var to drive which server to talk to
• Only send to langfuse if there's some kind of metrics_opt_in
• (but maybe hard-code the global chat to use langfuse for now?)

[josephjclark]

We're likely to be waiting a while for self-hosting - so let's set up an integration with langfuse cloud which only sends openfn staff chats. That lets us merge and experiment with platform before committing to a self-hosted instance

[hanna-paasivirta]

I will add filtering for employee emails

[hanna-paasivirta]

@josephjclark I have added the changes but didn't test it properly yet. So please wait until Monday for this.

[hanna-paasivirta]

I have been testing this and I think it should work ok. Once the front-end adds the requested changes, we can start tracking on Langfuse.

[josephjclark]

So we can safely release this without waiting for Lighting right @hanna-paasivirta ? We just won't get the metrics until lightning sends us the right data

EDIT: Oh actually we get it for global assistant anyways

[hanna-paasivirta]

So we can safely release this without waiting for Lighting right @hanna-paasivirta ? We just won't get the metrics until lightning sends us the right data

EDIT: Oh actually we get it for global assistant anyways

Yes all new keys are optional, so we can push this without waiting for Lightning to add changes.