Difficulty with sum aggregation with varying duration

[dynadynat]

We have two different types of metrics and we want to showcase "Graph" using data from them.

Objective: Count the number of jobs in a particular job status and show it's showing a historical time chart

Metric1: JobsInPlanCount

application_wa_JobsInPlanCount_job{jobstatus="ERROR"} 16
application_wa_JobsInPlanCount_job{jobstatus="SUCCESSFUL"} 15
application_wa_JobsInPlanCount_job{jobstatus="RUNNING"} 0
application_wa_JobsInPlanCount_job{jobstatus="SUPPRESS"} 0
application_wa_JobsInPlanCount_job{jobstatus="READY"} 4
application_wa_JobsInPlanCount_job{jobstatus="WAITING"} 10
application_wa_JobsInPlanCount_job{jobstatus="UNDECIDED"} 0
application_wa_JobsInPlanCount_job{jobstatus="HELD"} 0
application_wa_JobsInPlanCount_job{jobstatus="CANCELED"} 0
application_wa_JobsInPlanCount_job{jobstatus="BLOCKED"} 0

Dimension key:  "jobstatus" (Only one)

Aggregation: Auto(avg) and Graph.

 

Result: Correct result was observed even though time duration was varied (2 hrs , 6hrs, 24hrs , 72 hrs , 7 days  etc)

Code used is : wa.metric.application_wa_JobsInPlanCount_job:splitBy(jobstatus):sort(value(auto,descending))

 Metric2 : JobsByWorkstation

application_wa_JobsByWorkstation_jobs{jobstatus="UNDECIDED",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="HELD",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="HELD",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUPPRESS",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUCCESSFUL",workstation="/HWA-GKE-SERVE_XA"} 7
application_wa_JobsByWorkstation_jobs{jobstatus="CANCELED",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="CANCELED",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="BLOCKED",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="ERROR",workstation="/HWA-GKE-SERVE_XA"} 1
application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/HWA-GKE-SERVE_XA"} 4
application_wa_JobsByWorkstation_jobs{jobstatus="READY",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="READY",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUPPRESS",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="UNDECIDED",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="UNDECIDED",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="READY",workstation="/MASTERAGENTS"} 4
application_wa_JobsByWorkstation_jobs{jobstatus="RUNNING",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUCCESSFUL",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="BLOCKED",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="BLOCKED",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="BLOCKED",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="ERROR",workstation="/WA_SAGT_0"} 6
application_wa_JobsByWorkstation_jobs{jobstatus="CANCELED",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="ERROR",workstation="/WA_SAGT_1"} 9
application_wa_JobsByWorkstation_jobs{jobstatus="SUCCESSFUL",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="RUNNING",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="HELD",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUCCESSFUL",workstation="/WA_SAGT_0"} 8
application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/WA_SAGT_1"} 2
application_wa_JobsByWorkstation_jobs{jobstatus="UNDECIDED",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/WA_SAGT_0"} 4
application_wa_JobsByWorkstation_jobs{jobstatus="RUNNING",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="READY",workstation="/HWA-GKE-SERVE_XA"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="RUNNING",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="HELD",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="CANCELED",workstation="/MASTERAGENTS"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUPPRESS",workstation="/WA_SAGT_0"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="SUPPRESS",workstation="/WA_SAGT_1"} 0
application_wa_JobsByWorkstation_jobs{jobstatus="ERROR",workstation="/MASTERAGENTS"} 0

It has two dimenshion keys : "jobstatus" and "workstatus"

Dimension key:  "jobstatus" and "workstation (Only two)

Aggregation:

(a) Auto(avg) not working because giving a decimal value for it (which is incorrect).

(b) Sum is giving extrapolated value

• For short duration - giving twice of the actual count or sometimes other value.
• For long duration- With the increase in time duration, its value is linearly increasing.

[ have a look to the attached screenshots ]

 

Kindly, tell me how to avoid these erroneous results. 

Thank You

[uros_djukic1]

Hello,

At the first glance,  all depends on your fold transformation selection.
According your selection, the aggregation Auto(avg) and an Auto fold transformation on the result, the single average values are summed up (same as described here).
You can do the following things:

• calculate the average values of the average metric: configure aggregation Auto and Fold transformation Average
• calculate the average of the average values: configure Average as the aggregation and Average as the Fold transformation
• calculate the average of the maxima: choose aggregation Maximum as the aggregation and Average as the fold transformation
  useful link :
  https://www.dynatrace.com/support/help/dynatrace-api/environment-api/metric-v2/metric-faq#value-exce...
  
  I hope it helps,
  Thanks
  
  
  

[zietho]

@fcourbon fyi

[fcourbon · Accepted solution]

It is difficult to tell without an idea of the underlying data. It looks like the data is constant over time e.g. this line below: 

application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/HWA-GKE-SERVE_XA"} 4

 is ingested multiple times with a constant value of 4. In which case the average would be 4 as expected, but the sum with this line

application_wa_JobsByWorkstation_jobs{jobstatus="WAITING",workstation="/WA_SAGT_1"} 2

would lead to 6.

If aggregated over time, then it is expected to increase with coarser resolutions.

What would be the desired output when using the 2nd metric and aggregating over Workstation?