topic Confusion around the "Number of calls to other services" metric in Data Explorer in Dashboarding

Confusion around the "Number of calls to other services" metric in Data Explorer

mdouds — Mon, 21 Feb 2022 21:00:36 GMT

Hello,

I'm trying to get a list of services that have a high number of calls to child-services. The data explorer seemed like the best place to do this, but I'm getting confusing results.

As a test to make sure we're using the right metrics....

Using a multi-dimensional view for a specific service, I'm seeing the Number of calls to other services metric report the following:

avg: 1.8

90th percentile: 2

max: 76

Good so far, we know there are some higher outliers...

When I open a data explorer and filter to the same service, same time period:
avg: 162k

95th: 162k

max: 162k

This is using the builtin:service.nonDbChildCallCount metric, which is what we get when searching for "Number of calls to other services."

What's going on here?

Also, if we create a calculated service metric using "Number of calls to other services", no splittings or dimensions, and add that to the Data Explorer, it shows the correct numbers we see in the MDA.

Is there a bug with the "Number of calls to other services" built-in metric in the data explorer?

Re: Confusion around the "Number of calls to other services" metric in Data Explorer

dannemca — Mon, 21 Feb 2022 21:13:40 GMT

That's because the data source are different for both.

Unlike multidimensional analysis, the Data explorer uses metrics data, not session storage, so values on custom charts might differ from values in a multidimensional analysis.

https://www.dynatrace.com/support/help/shortlink/multidimensional-analysis#data-source

Re: Confusion around the "Number of calls to other services" metric in Data Explorer

mdouds — Mon, 21 Feb 2022 21:20:18 GMT

Meaning...the metrics data is not calculating the average number of calls per request?

Re: Confusion around the "Number of calls to other services" metric in Data Explorer

Julius_Loman — Mon, 21 Feb 2022 21:33:09 GMT

@mdouds in the MDA - it's per request. The metric is a SUM of the child calls per chart resolution (lowest being one minute).
So you are comparing number of child calls per request vs. number of child calls per minute that's why you have such odd numbers. Just use the code level tab in data explorer and divide the metric builtin:service.nonDbChildCallCount with number builtin:service.requestCount.total to get number of calls per request.

Re: Confusion around the "Number of calls to other services" metric in Data Explorer

mdouds — Mon, 21 Feb 2022 22:09:11 GMT

Thanks, that does help unless we want to find the services with higher percentile or max counts. I assume in those cases we could try the calculated service metric route?

Re: Confusion around the "Number of calls to other services" metric in Data Explorer

Julius_Loman — Tue, 22 Feb 2022 07:23:09 GMT

@mdouds no this is not possible. It will still be a metric and you can't use aggregations there.

However, you can use the global MDA view to find such services, then focus on them and find the requests with excessive calls. You don't have to go to the MDA for each individual service.