Incremental ingest of custom sql extension dimensions/metrics?

moilejter — Thu, 10 Sep 2026 02:07:10 GMT

Hi!

We've just started looking into custom sql extensions, to run database queries and make the results available within dynatrace. The simple case seems straightforward enough: run a database query, and the complete result set becomes available within dynatrace. But then, when the full query is rerun some interval later, there will be a lot of overlap between the two. Does the ingest process map the "same" rows so that they only appear once? Or, if the write the query so that it's incremental (get the rows in the last minute, run every minute) - does the ingest process automatically record all the results across all invocations?

The docs I found don't seem to go into this kind of detail...

Thank you!

Re: Incremental ingest of custom sql extension dimensions/metrics?

sujit_k_singh — Thu, 10 Sep 2026 03:52:36 GMT

Hi @moilejter

It depends on Output type
The SQL extension does not deduplicate rows between executions. There is no "same row" detection. What happens to repeated data depends entirely on which output type you're using:

1. Output Type: Metrics
Each row maps to a metric data point with a timestamp of when the query ran. The metric store is a time series it doesn't care about row identity. Every execution writes new data points at the new timestamp.

Implication for your overlap question:

If you run a full query every minute and the same rows come back, you get new metric data points each minute not duplicates in the traditional sense, because each has a different timestamp.

2. Output Type: Logs or Business Events
This is where overlap becomes a real problem. Each row becomes a discrete record stored in Grail with a timestamp. There is no deduplication.

Full query every minute → every row ingested as a new log/bizEvent record every minute → massive duplication
A row that existed 5 minutes ago and still exists now will appear 5 times in Grail

Thanks,

Sujit

topic Incremental ingest of custom sql extension dimensions/metrics? in Extensions

Incremental ingest of custom sql extension dimensions/metrics?

Re: Incremental ingest of custom sql extension dimensions/metrics?