Information:

Environment

  • Backbone
Symptoms

My tests ran without any errors or failures, but I see gaps in the related charts. Why do I see these gaps?

Solution

There are several reasons you may see gaps in charted data:

  • Test runs on different nodes may not start at the same time.
  • The charting interval is shorter than the test run interval.
  • Test runs are suspended during a maintenance window.
  • Data was automatically reclassified.

Test runs on different nodes may not start at the same time

When a test is initially activated, the test immediately runs from all of the nodes assigned to the test. Over time, variations in node response times lead to differences in test scheduling so that the tests no longer start at the same time across all nodes. 

The root cause is that after an individual node completes a test, the node places a time stamp on the collected data. At this point, the node checks a test frequency setting and schedules the next test transaction based on the recorded time stamp.

For example, when Node A completes a test in 5 seconds, but Node B completes a test in 7 seconds; Node B starts its next test 2 seconds later than Node A does.

Over time, test scheduling is spaced out based on node response time and the test frequency setting.

For this reason, you occasionally see a gap when charting a test when the time range is equal to the test frequency.

For example, a chart may not always display results for each 15-minute increment for a test that takes 15 minutes with a 15-minute scheduling frequency.

This would occur on a node where a test was run at 1:14 PM then again at 1:31 PM - no data would exist for the 15 minute span between 1:15 PM and 1:30 PM. The 2-minute variation results from node response times in collecting data, writing the related time stamp, and waiting 15 minutes before conducting the next test.

The charting interval is shorter than the test run interval

If the charting interval is shorter than the test run interval, there only be data for the charting intervals that coincide with the test runs. For example, if the chart interval is every 5 minutes but the test runs every 15 minutes, there will only be a data point every 15 minutes, and the 5-minute and 10-minute interval after a test run will have no data. Because of these gaps, the data points are not connected in the chart.

Test runs are suspended during a maintenance window

The time period for a maintenance window will be included in the chart although there is no data collected during that time. Maintenance windows can be defined globally — applied to all new tests by default — or for the individual test. You can review the maintenance windows for a test in the Advanced Settings of the test configuration. For more information, see the help page Editing Tests.

Data was automatically reclassified

When automatic analysis of the test runs identifies data that was impaired by factors such as provider issues or hardware problems and therefore does not reflect "real world" behavior, the data is reclassified and removed so it does not affect the validity of the overall results. The removal of reclassified data may cause gaps in the chart.

 

 

 

 

  • No labels