CI Performance Testing Automated Analysis

Platform Dynatrace Managed
1

I am working with Dynatrace Managed and want to include performance test analysis/validation as part of the build pipeline to pass or fail a test.

I have watched the online perf clinic from October 2017 which highlights the integration possible through Neoload. This demonstrated that you can tag requests as in AppMon and register tests as events in Dynatrace using Neoload but I am using Jmeter and LoadRunner. I’ve found this page around registering third party events but the documentation for the API is very limited: https://www.dynatrace.com/support/help/dynatrace-api/events/how-do-i-push-events-from-3rd-party-systems/.

Is this the correct way to start/stop tests? If so, can the docs be extended with more explanation around API request/response parameters and further examples on how to use this for test events?

Following on from this, it does not seem that it is possible to use Dynatrace to compare tests in a pipeline scenario so registering tests is of little value in this scenario.

Registering a maintenance window like was mentioned in the perf clinic for helping the AI component would not work for this CI/CD case as tests are only run when builds are triggered which is only 1-2 times a day max, so a maintenance window would almost always be set.

This post also talks about approaches for performance engineering but does not mention that it is possible to do any kind of automated analysis. https://www.dynatrace.com/news/blog/load-testing-r…

2

Hi Andrew

Let me provide you with some more details

#1: Pushing “Custom Events” to Dynatrace

The Events API allows you to push custom events to Dynatrace so that these events, e.g: Deployment, Configuration Change, Start Test, Stop Test show up as so called “Natural Events” in the Dynatrace UI when you open the overview screen of your monitored entites, e.g: Service, Host, ..

In the webinar with Neoload they were sending these events to the service entity (easyTravel Frontend) that they were testing against. You can achieve the same with other load testing tools by simply calling the REST API as explained in the documentation. I also have additional resources you can check out

* Dynatrace CLI: this is a python based CLI that provides an easy way to push events to a Dynatrace Entity:

* Blog about Custom Events: https://www.dynatrace.com/news/blog/diy-events-api…

#2: Register Test Run

In AppMon we had a baked in concept of a Test Run which you could register and you could then get results for all metrics that were captured for that run. Something like this doesnt exist in Dynatrace.

HOWEVER - with Dynatrace - and the APIs we have - we can elevate our Continuos Build Validation. Please have a look into my work around the Unbreakable Pipeline. Here I am introducing a concept of monitoring as code which can act as automated quality gates in your CI/CD after executing your load tests:

* Blog: https://www.dynatrace.com/news/blog/unbreakable-de…

* YouTube: https://www.youtube.com/watch?v=r-0xunTqwQ0&index=…

* GitHup Repo: GitHub - Dynatrace/AWSDevOpsTutorial: Unbreakable DevOps Pipeline Tutorial with AWS CodeDeploy, AWS CodePipeline, AWS Lambda, EC2 and Dynatrace · GitHub

#3: Load Test Analysis

If you want to learn more about Load Test Analysis with Dynatrace please have a look at the following resources

* Blog: https://www.dynatrace.com/news/blog/load-testing-r…

* YouTube with Neoload

* YouTube on Basic Diagnostics

#4: Integration into Jenkins

This wednesday at 10AM US East I will record another Performance Clinic where I am going to show how to integrate Dynatrace into Jenkins. My goal is to also execute a short load test. I try to get JMeter into the mix. In any case - I am going to show several use cases of leveraging the Dynatrace API to integrate into your CI/CD DevOps Tool Chain.

I hope this helps

Andi

3

We have now officially release LoadRunner request tagging for Dynatrace - check it out: https://www.dynatrace.com/news/blog/loadrunner-req…