cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

seeking information on how performance metrics are calculated

shafiz_hatta
Guide

i would like to know how the performance metrics are calculated.When run a 2 diffrent versions of same test i see some metrics generated like:

Volatility,Improved runs,Expected maximum (for purepath),degraded and improved runs etc...

Please can you help me on this.

3 REPLIES 3

andreas_grabner
Dynatrace Guru
Dynatrace Guru

Hi.

I suggest you check out the documentation on Continuous Delivery and Test Automation. There are several doc pages that explain that these metrics come from the PurePaths and are associated with your tests. We also have a baselining of each Metric for each Test which allows us to calculate a "Performance Corridor". The first corridor will show up after we have at least 5 test results. After that we look whether the measures of the current tests falls within that corridor or outside. if outside it means that you have a violation. If you have 2 consecutive violations we consider the test as degraded. If a measure constantly produces different results we mark it volatile. But - check out the doc for more information on this

I also have a youtube tutorial where I discuss this feature: https://www.youtube.com/watch?v=Pla4dWeqJgI&list=PLqt2rd0eew1bmDn54E2_M2uvbhm_WxY_6&index=25

Andi

shafiz_hatta
Guide

Thanks Andi for the response.

In my case running some Units for Different build's for single run.even i could see them in Test results dashboard.Form pure path comparison i could see that variation in response time for some Unit test in the builds's when compared.

But in Test results dashboard i don't see any information showing that test is Improved or Degraded.

And even i see Expected Minimum and Expected Maximum time for selected test run in Pure paths.

To show the Test as Improved or Degraded do i need to change any settings in Tests automation under system profile.

Even please let me know how the Expected Minimum and Expected Maximum time for each Unit test pure path for example:

I have a test in

1st run :Pure path value is 57042.97 MS settings.pngcorridor.png

2nd run:Pure path value is 57354.31 MS

from the above i can see that 2nd Run is taking more time to execute.

Expected Minimum:50193.37

Expected Maximum:64203.91

But on what basis the Expected Minimum and Expected Maximum time are calculated.

Please let me know how to view the degraded and Improved runs inTest results dashlet that are executed only once for different build.

PFA the screenshots.

Sorry for the late response on this. Dynatrace "learns" what the values should be by looking at the measures for every single test that you execute for every build. The default "learning phase" is 5 runs which is configurable through the Settings Dialog. Dyntrace also automatically updates this "performance funnel" or "expected value range" with every new test you execute on a new build.

Key is that you execute the same test on different builds. Make sure these tests are correctly marked with the right build number. Then you should end up seeing each measure and how it evolves over time. Please check out the doc pages that explain pretty well how this is supposed to work:

It would also be interesting to see a screenshot of your corridor chart. not just the details dialog for a single test run - but for the whole timerange

Andi