Workflow for creating custom metric from DQL Query Result

[36Krazyfists]

Before I go down the rabbit hole of trying to do this from scratch, I'm hoping someone out there has already done this (Dynatrace really needs a sharing site where users can share their custom creations...).

What I need is relatively simple.  I have a DQL Query that spits out some timeseries metrics.  I'd like to take those and create a new custom metric in Dynatrace using the metrics ingest API.

I'm really hoping someone has done this already and can share their code with us?

(I also wish Dynatrace would create an integration for that, just like they have integrations for creating/editing ownership, SRG, and Davis Anomaly Detectors)

[haris]

Hi 36Krazyfists,

can you pass more details for your use case? What data does the DQL query fetch, and can you share an example query? 

[36Krazyfists]

I mean, it can be anything.  The what isn't that important.  I'm just looking for something similar to calculated service metrics, for example.

I do have a specific use case where I want to be able to query the first time a log entry that has a certain id value in one of its columns occurs and then the last time an entry with that same id value occurs, then get the difference as a duration (effectively getting the total time taken for each user).  Then average up the result of all of those calculations into a single value that we can then use to create a metric.

This isn't possible in OpenPipeline because OP can't look at other log records since it is processing logs as they flow in.  The only way to create this metric is with a DQL query and then creating a metric out of that query.

But that's just one example.  I want to be able to do that with the result of any DQL query, no matter what it is.

[Peter_Youssef]

Hi @36Krazyfists 

We would appreciate providing sufficient details to enable community team give the required support.

Regards,

Peter

[christian_kreuz · Accepted solution]

Hi,

assuming you have a DQL Task like this one, called "get_data"

timeseries p95 = percentile(dt.service.request.response_time, 95)
| fields response_time_95 = (arrayAvg(p95))

You can ingest a metric using the docs provided here: https://developer.dynatrace.com/develop/data/ingest-data/#metrics

Example, using a "Run JavaScript" task:

// optional import of sdk modules
import { execution } from '@dynatrace-sdk/automation-utils';
import { metricsClient } from '@dynatrace-sdk/client-classic-environment-v2';

export default async function ({ executionId }) {
  // your code goes here
  // e.g. get the current execution
  const ex = await execution(executionId);

  const dqlResult = await ex.result('get_data');
  // dqlResult = { records: [...], metaData: ..., types: ... }
  // get response_time_95 from records of dqlREsult
  const response_time_95 = dqlResult.records[0].response_time_95;

  console.log(`Going to ingest ${response_time_95}`);

  // ingest metric called my.response_time_95 using metricsClient
  return await metricsClient
    .ingest({ body: `my.response_time_95 ${response_time_95}` })
    .then((response) => {
      console.log(response);
     })
    .catch((e) => {
      console.error(e);
    });
}

You can run this Workflow based on a schedule. I've attached a Workflow Template as an example (zipped, as .yaml is not allowed).

You can observe the result using a Notebook or a Dashboard, accessing the metric you ingested:

Note: Please ensure that your User and the Workflow Actor have appropriate permissions to read and write data from/to Grail, specifically metrics.

Hope this helps!

[36Krazyfists]

Really great stuff @christian_kreuz !  Exactly what I was looking for!  I still contend that Dynatrace should provide an integration (like Owners and Anomaly Detection) for creating custom metrics (and even integrations for setting maintenance periods on entities and other common stuff that people will use Workflows for within Dynatrace.  Or, at least setup a sharing site where users can post and share creations they've made with each other).

Thanks so much!

[christian_kreuz]

I agree, built-in actions that make certain-uses cases easier, essentially a no-code solution for the end-user, would be a great addition. I will forward this feedback internally!