Community Voices
Explore thought pieces, announcements and product news
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
sebastian_kruk
Dynatrace Champion
Dynatrace Champion
You have all your microservices instrumented and Dynatrace tells you the moment a payment API slows down. But with Snowflake as part of your data platform, dynamic tables falling behind or warehouses burning through credits may never surface in your monitoring — because your application layer and your data layer are still observed in two different worlds.
 

TL;DR: We released Dynatrace Snowflake Observability Agent (DSOA) as open source — a tool we originally built for our internal data platform teams to bring Snowflake telemetry into Dynatrace and provide observability across their data infrastructure stack.

What Is DSOA?

DSOA is a set of views, procedures, and tasks that run inside your Snowflake account to deliver telemetry into your Dynatrace tenant. No external infrastructure, no sidecar processes.

It collects the observability data that's already available — in Snowflake's ACCOUNT_USAGE views, INFORMATION_SCHEMA functions, and SHOW commands — and ships it to Dynatrace as logs and traces (via OpenTelemetry), as well as metrics and events (via Dynatrace APIs directly). Think of it as OneAgent for Snowflake internals: something that runs close to your data and sends structured telemetry out to where you can act on it.

DSOA is plugin-based — you pick what to monitor and how. There are already 16 plugins covering everything from query history to security trust scores, and more are on the way.

Data Platform Observability

When we talk about observing a data platform, there are five things that actually matter to the teams running it:

  • Security — Who is accessing what? Are there anomalous login patterns? Are sensitive tables properly governed?
  • Operations — Are my pipelines running? Did that task graph finish? Is anything stuck?
  • Costs — Where are my credits going? Will this resource monitor run out before the quota resets?
  • Performance — Which queries are getting slower? Which tables are causing partition scans? What's blocking my warehouses?
  • Quality — Is my data fresh? Did a schema change break a downstream consumer? Is my data volume drifting?

These aren't new problems — every data team deals with them. But historically, they've been solved in isolation: a custom Snowflake dashboard here, a manual credit report there, a DBA who checks query history when someone complains.

DSOA brings visibility across all five themes into Dynatrace, where you can leverage dashboards, workflows, Davis AI, and anomaly detection already monitoring the rest of your stack.

Example Use Cases

There are many use cases DSOA can already address. Here are two worth highlighting.

Keep Snowflake costs under control

Snowflake provides ways to help you keep tabs on credits spent: you can set up Budgets to monitor costs over the whole month, or set up Resource Monitors to notify you when crossing a certain percentage threshold and stop monitored warehouses if you exceed your quota.

But it's hard to tell if today's usage is unusually high — and whether it will leave you without credits to run important tasks later that day. You also can't easily tell if your daily usage is slowly creeping toward the 100% cutoff a few days from now.

With DSOA delivering this telemetry to Dynatrace, you can leverage built-in forecasting to react proactively — before your tasks start failing due to credit exhaustion.

Catch slow queries in the act — and understand them after the fact

When it comes to slow queries, it's a crime scene investigation. DSOA continuously sends telemetry — bytes scanned, spill volumes, partition scan ratios — on all queries, but it enters "CSI mode" each time a long-running query is detected. This extra forensic detail, such as the query profile or acceleration estimates, helps you understand why a query is slow and how to improve its performance.

But DSOA doesn't only leave you with post-mortem analysis. It also monitors currently running queries to detect those running unusually long. This telemetry enables Dynatrace to act — alerting an on-call engineer or using a Dynatrace Workflow to intervene.

How It Works

DSOA deploys all its objects — views, procedures, and tasks — into a dedicated database (DTAGENT_DB). Tasks run the DTAGENT() Snowpark procedure on a schedule using the DTAGENT_VIEWER role, which taps into the existing telemetry in Snowflake. Once new telemetry is gathered and packaged into standardized views, DSOA transfers it as logs, traces, metrics, and events to the Dynatrace Ingest APIs.

DSOA delivering telemetry with the query_history pluginDSOA delivering telemetry with the query_history plugin

Once in Grail, you can consume it via DQL in dashboards or notebooks, or make it actionable with workflows, anomaly detection, and forecasting.

// DQL example: queries running more than 60 min
fetch logs
| filter dsoa.run.plugin == "active_queries"
| filter isNotNull(snowflake.query.id)
| filter isNotNull(snowflake.time.running)
| fieldsAdd snowflake.time.running.in_min = toLong(snowflake.time.running)/1000/60
| filter toLong(snowflake.time.running.in_min) > 60

Getting Started

DSOA is available on GitHub. Once you unpack the installation package, you just need an API Token for your Dynatrace tenant and a simple configuration profile:

core:
	dynatrace_tenant_address: YOUR_TENANT.live.dynatrace.com
	# This identifies your instance in telemetry
	deployment_environment: PRODUCTION
	snowflake:
		# Use organization-account format (recommended)
		account_name: "myorg-myaccount"
# ... other settings

plus a user with the ACCOUNTADMIN role (only needed for the initial setup), and you can simply run:

./deploy.sh production

Hint: You can also ask your Snowflake administrator to set up the basic objects for you and deploy the agent itself without any admin rights.

Depending on your needs, you can fine-tune your deployment — choosing which plugins to deploy, when to run them, or adjusting the scope and frequency of your monitoring.

We are landing the v0.9.4 release soon — adding Snowpipes monitoring along with more ready-to-use dashboards and further improvements.

If you're running Snowflake alongside Dynatrace, give it a try. Star the repo if it's useful. And drop a comment below — we'd love to hear what use cases matter most to your team.