Observability for AI Features.

Your existing APM tells you the route was slow. It cannot tell you the model returned a length-truncated answer, the tool call looped four times, or the cache hit rate dropped after a prompt edit. Those are LLM-specific failure modes, and your generic tracer is blind to them.

The good news: in 2026 you do not need a special vendor to see them. OpenTelemetry has a stable GenAI semantic convention that every major observability platform now consumes — you instrument once and pick the dashboard later.

The shift in what you measure:

• Tokens, not just bytes. Tokens are the unit of cost and the unit of behavior. Track both directions.
• Time-to-first-token, not just total. TTFT is what the user feels on a streaming endpoint; total time is what your bill reads.
• Finish reason, not just status. A 200 OK with finish_reason: length is a user-facing bug your APM thinks went fine.
• Tool-call depth. An agent that loops list/get/list/get five times is broken; HTTP-level metrics will never tell you.

The spec lives under gen_ai.*. Span names are operation-shaped (chat, execute_tool, invoke_agent), attributes carry the metadata you actually want to query.

If you're on an instrumentation pinned to GenAI v1.36 or earlier, the spec is explicit: don't change the default version. Opt in to newer conventions with an env var:

export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

import { trace, SpanStatusCode } from '@opentelemetry/api';
import { streamText } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';

const tracer = trace.getTracer('chat-route');

export async function POST(req: Request) {
  const { messages } = await req.json();

  return tracer.startActiveSpan('chat', async (span) => {
    span.setAttributes({
      'gen_ai.operation.name': 'chat',
      'gen_ai.system': 'anthropic',
      'gen_ai.request.model': 'claude-sonnet-4-5',
    });

    const t0 = performance.now();
    try {
      const result = await streamText({
        model: anthropic('claude-sonnet-4-5'),
        messages,
        abortSignal: req.signal,
        onFinish: ({ usage, finishReason, response }) => {
          span.setAttributes({
            'gen_ai.response.model': response.modelId,
            'gen_ai.usage.input_tokens': usage.promptTokens,
            'gen_ai.usage.output_tokens': usage.completionTokens,
            'gen_ai.response.finish_reasons': [finishReason],
          });
        },
      });

      return result.toUIMessageStreamResponse();
    } catch (e: any) {
      span.recordException(e);
      span.setStatus({ code: SpanStatusCode.ERROR, message: e.message });
      throw e;
    } finally {
      span.setAttribute('duration_ms', performance.now() - t0);
      span.end();
    }
  });
}

Most provider SDKs ship auto-instrumentation packages (@traceloop/instrumentation-anthropic, OpenAI's own OTel exporter, Vercel's experimental telemetry option) — use them when they exist, fall back to manual when they don't. The attribute names are the same either way; the dashboard speaks the same language.

Add these to a custom attribute alongside the standard ones (the conventions explicitly allow it — you can migrate to a standardized version later if the spec catches up):

• gen_ai.cost.total_usd — computed at log time from token counts and current prices. Saves you from joining against a price table at query time.
• gen_ai.ttft_ms — time-to-first-token, the user-facing latency metric. Measure between request received and first stream chunk emitted.
• gen_ai.user.tenant_id — the tenant or workspace the call is for. Lets you spot a noisy tenant or charge them back accurately.

The mistake everyone makes the first time: SLO total response time. It varies legitimately with prompt complexity, gets worse the day someone enables long-context, and tells you nothing actionable. Use two SLIs instead.

That gives you a real error budget. A 99.5% SLO over 28 days is ~3.6 hours of permitted bad time per month. Burn through more than half of it in the first week and you have an alert; burn it all and you stop shipping risky changes until the window resets. Standard SRE math, applied to LLM features.