cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

What is AI and LLM Observability?

flo_lettner
Dynatrace Participant
Dynatrace Participant

AI observability is a modern, complete approach to understanding how your AI applications behave, how data flows, and how performance changes over time.

At Dynatrace, we provide advanced model observability with predictive capabilities for efficient cost management, end-to-end traceability in RAG pipelines and agentic frameworks, compliance and governance to adhere to laws like the EU AI Act, and LLM safeguards such as PII leakage prevention, language toxicity assessment, and hallucination detection. This ensures service health and performance, service quality, compliance, and governance.

Service Health & Performance

amazon-bedrock-3024-18972e2d48

Monitor standard metrics like response times, error rates, and cost signals, and chart them based on different AI models. Leverage the predictive capabilities of Dynatrace Davis AI® to detect changes in usage behavior and predict cost changes to help team members understand model performance and optimization opportunities.

Key Features:

  • Cost Prediction
  • Token Consumption
  • Request Duration
  • Service Health
  • Open Problems

Service Quality & Guardrails

amazon-bedrock-guardrails-2058-7075d2f063

Monitor safety, privacy, and truthfulness safeguards for your generative AI applications with Dynatrace. Detect toxic language, track Personally Identifiable Information (PII) leakage, identify attempts at LLM misuse such as malicious prompt injection, and monitor model hallucinations.

Key Features

  • Toxicity
  • PII Leakage
  • Denied Topics
  • Hallucinations

End-to-End Tracing & Debugging

trace-agentic-pipeline-1708-3a40424e8a

Map dependencies between multiple large language models that work in concert in your RAG pipelines or agentic frameworks to provide end-to-end observability of the entire system. Track every step - from request to response - and gain full visibility into your pipeline's performance.

Key Features

  • RAG Pipeline Tracing
  • Agentic Framework Tracing
  • Prompt Data Lineage

Data Compliance & Governance

data-compliance-preview-4536-62bb6d4834

Dynatrace automatically tracks every input and output with no sampling to provide an audit trail of accurate monitoring and observability for your GenAI invocations. This helps ensure that applications comply with applicable laws such as the EU AI act and the US Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.

Key Features

  • Comprehensive Audit Trailing
  • Real-time Data Access
  • Long-term Data Retention
0 REPLIES 0