# \#llm-as-judge

**URL:** https://community.dynatrace.com/tag/llm-as-judge/716.md

[Latest](https://community.dynatrace.com/latest.md) · [Categories](https://community.dynatrace.com/categories.md) · [Tags](https://community.dynatrace.com/tags.md)

---

## [Introducing dt-evals: an open source continuous evaluation tool for LLM apps in Dynatrace AI Observability](https://community.dynatrace.com/t/introducing-dt-evals-an-open-source-continuous-evaluation-tool-for-llm-apps-in-dynatrace-ai-observability/56035)

<div class="topic-metadata">

**Author:** [@krismuhi](https://community.dynatrace.com/u/krismuhi)\
**Replies:** 0\
**Last updated:** [June 12, 2026, 7:46am UTC](https://community.dynatrace.com/t/introducing-dt-evals-an-open-source-continuous-evaluation-tool-for-llm-apps-in-dynatrace-ai-observability/56035 "2026-06-12T07:46:07Z")

</div>

TL;DR: We open-sourced dt-evals, a CLI toolkit for evaluating LLM and agent quality using real GenAI traces in Dynatrace AI Observability. Run evaluations on live or recent gen\_ai.\* spans, score responses with an LLM jud…