AI
Why Traditional Telemetry Misses GenAI Failures
Standard distributed tracing tracks duration, error rate, throughput, and token usage. For deterministic systems, that coverage is enough. GenAI systems accept arbitrary human input, which means the same trace structure can show a clean, error-free run while the model has just leaked sensitive data. Molkova demonstrated this with a live demo: a user logged in as Alice Johnson claimed to be CEO Alex Morgan and asked for the CEO’s salary. The system complied. No errors fired. No alerts triggered.
Prompt Injection in a Multi-Agent HR Assistant
The demo application, called “Hus HR,” routes questions through multiple agents. Molkova asked for a salary under a false identity. The response arrived fast, the telemetry showed success, and the correct salary figure came back for the wrong person. The failure mode is non-determinism: a developer who tested the application once and saw correct output has no guarantee the same prompt produces the same behavior after a configuration change or model update. Classic unit tests catch regressions; they do not catch inputs no one anticipated.
Capturing Content, Not Just Metadata
OpenTelemetry GenAI semantic conventions record full conversation history on spans, including tool call arguments and results. This makes one trace show the complete sequence of events, timing, and errors in one place. The cost is real: backends index everything, and chat history is large. More importantly, the content is sensitive. Health queries, legal questions, and personal disclosures should not be accessible to every SRE on the team. Two audiences exist with different access needs, and a single telemetry store serves neither well.
Routing Sensitive Data to Separate Storage
Molkova’s answer is to store large content blobs in blob storage and write only a reference URL on the span. Blob storage is cheap, and it carries an independent access policy. Operations staff see performance signals without touching conversation content. Investigators or audit teams get access to the actual messages through a separate path. The OpenTelemetry instrumentation libraries already support configuring this external storage. Molkova showed Claude reading traces from a local S3-compatible store and summarizing what happened, including flagging that Alice queried data belonging to Alex.
LM-as-a-Judge Evaluation Results as Telemetry
Offline evaluations run before a release catch known problems but miss production inputs. Continuous LM-as-a-judge evaluation runs asynchronously against live traffic and records results as OpenTelemetry log events named gen_ai.evaluation.result. Each event carries the evaluator name, a score, a trace ID, and a span ID pointing to the operation it assessed. Molkova’s dashboard showed a review decision correctness score of 0.4. A top-10 worst operations list surfaced every flagged case, all of them either disclosing data to the wrong identity or returning irrelevant results.
Guard Rails and the OpenTelemetry GenAI SIG
Evaluations find problems after the fact. Guard rails stop them synchronously. For the salary leak, the correct guard rail is a deterministic check: verify the queried identity matches the logged-in identity before the database call, no AI required. The OpenTelemetry Generative AI semantic conventions SIG defines how to record all of this. Molkova noted current gaps: skills, guard rails, and memory operations have no conventions yet. GCP and Microsoft actively contribute; OpenAI implemented tracing in their SDK without following the standard. The SIG meets regularly and welcomes instrumentation library authors.
Q&A
Does capturing all inputs and outputs help when the LLM infers the wrong context from ambiguous data? Capturing everything that enters and exits the model is the prerequisite, but correct application logic, not just telemetry, is the real fix. ▶ 24:58
Are AI vendors participating in the GenAI SIG? GCP and Microsoft actively collaborate; OpenAI implemented their own tracing without using OpenTelemetry conventions, and Molkova said she would like to fix that. ▶ 25:56
Can users manipulate LM-as-a-judge metrics through adversarial inputs? Yes, Molkova confirmed this is possible and said she discovered it while preparing the talk. ▶ 28:55
Are agent trajectory conventions planned? The SIG is aware of the gap; guard rails, memory operations, and trajectories are all missing and the group is actively looking to cover them. ▶ 30:07
Notable Quotes
there are no errors alerts your telemetry is perfect right Liudmila Molkova · ▶ 05:51
you can cheat the LM as a judge, right? Absolutely. Yes. This is all playable. Liudmila Molkova · ▶ 17:40
the classic telemetry that we have just the fact that something has happened is not really enough for AI we need to capture the content of that that actually happened. Liudmila Molkova · ▶ 21:07
there is a new ai library every hour Liudmila Molkova · ▶ 22:31
Key Takeaways
- A prompt injection attack can leak sensitive data while telemetry shows zero errors.
- Full conversation content belongs on spans, but sensitive blobs belong in separate storage with an independent access policy.
- LM-as-a-judge scores recorded as correlated OpenTelemetry log events make evaluation quality queryable alongside performance metrics.
About the Speaker(s)
Liudmila Molkova is a Staff Developer Advocate at Grafana Labs. She serves on the OpenTelemetry Technical Committee, maintains the Semantic Conventions, and contributes to the Generative AI SIG. Previously she led observability efforts on Azure client libraries.