Showing results for 
Show  only  | Search instead for 
Did you mean: 

Seeking information on Metrics in Tests Results like:Degraderuns,Improved runs,Expected Maximum Time)



I have Test Result which shows the Performance test that ran multiple time.

Defiantly from the Pure path i can see that second run is more performance improved from 1st Run.

But in the Test results dashlet it not showing any information in below columns like:

Improved Runs as '0' even there is an improvement same for Degraded runs.

I would like to know on what basis the metrics are calculated.

Please find the attached screen show of my Test results.



Hello Shafiz,

After you execute your tests a few times we calculate some statistical values for each measure (we like to call the "corridor") which represent the normal, expected value range. When the measure value goes above the expected value range, we say that the measure was degraded, when it goes below, we say it's improved.

In your case, it looks like the "Pure Path Duration" measure value decreased -- which is a good thing, but it didn't cross the corridor boundary, so we still treat it as the expected state. Had the change been more dramatic, we'd mark that as an improvement.

Hope that answers your question. If you're interested in more details, please see (part describing the corridor).

Best regards,

Wiktor Bachnik


Thanks .

Can you give an example how and on what basis you calculate the corridor value(which represent the normal, expected value range).

As i have only 2 runs for same tests i want to see the degraded and Improved runs.In pure path duration for some test i could see that Pure path duration is more than the expected time.but i don't see any degraded or improved runs.

And even i would like to know is there any way to show degraded and improved runs based on actual value rather than normal, expected value range.

for this do i need to do

For do i need to change any settings in System profile under Test automation like:

Corridor Settings :Setting Calibration Runs to 1.

And even i see in system profile settings under Testautomation cofiguration settings like: