Integrating Dynatrace in unit and integration tests to use the Test Automation features consists of two steps:
- You have to run your tests with the Dynatrace Agent injected, so you get end-to-end PurePaths starting in your tests and going all the way through your backend
- For the Test Automation dashlet, Dynatrace allows you to pass in some additional meta information like the build version. This information is passed via a REST call to the Dynatrace Server, and you can either use the Automation Library (Ant, Maven) for Dynatrace for popular frameworks like Ant, Maven or MSBuild, or execute the required call directly.
The full process looks as described in this diagram:
- A build is triggered, for example by a developer checking in code into the repository, which in turn triggers a new build on the CI server.
- The CI server executes the build based on a Ant/Maven/... script, and then runs the included tests with a Dynatrace agent injected. It also sets test metadata which can include information like the version, build ID, subversion committers,...
- The Dynatrace agent collects PurePath data, and at the end of the build, the CI server can pull the compiled performance metrics from the server and display them either on a dashboard (ie, using the Test Automation Plugin for Jenkins), or by storing the HTML or PDF report provided.
- It is also possible to export a session file for a specific test run, which allows you to store the exact transactions executed by a test suite and have it available for later comparison and analysis.
This page focuses on steps 4-7 of the diagram above: We will walk through adapting the build script used in a way to support the integration with Dynatrace.
Build Automation Tools
Currently we provide the Automation Library (Ant, Maven) for Dynatrace for the following build tools:
The automation library provides an easier way to use the relevant REST interfaces on the Dynatrace Server. It does not perform any injection of agents!
- Extract Automation Library Package to your local file system
dtTaskDefs.xml(defines all Ant Task) and
com.dynatrace.diagnostics.automation.jar(the actual automation library)
As a prerequisite to use the Dynatrace tasks in your Ant build, import the Dynatrace task definitions in your build.xml. The dtTaskDefs.xml file assumes that the com.dynatrace.diagnostics.automation.jar is located in the same directory
Before you run your tests (JUnit, TestNG, NUnit), use the DtStartTest Ant task to register a new test run on the Dynatrace Server and pass additional metadata like the major/minor version. This way, the Dynatrace Server is able to identify PurePaths that originate in the same test run and allows you to perform parallel test runs.
As a best practice, we recommend to define the individual bits of metadata as separate properties so they can be easily changed afterwards. For example, we can rely on the fact that properties in Ant are immutable (they cannot be changed after a value has been assigned), so we can define default values in the script and overwrite them when the build is run through a CI server.
After setting these properties, you can call DtStartTest before you actually run the tests. In this example, a target named "test" is called that depends on compiling code and tests first, and then executes the JUnit tests. At the beginning of this target, you call DtStartTest and pass in the various parameter defined before:
What happens here is that DtStartTest uses the Dynatrace Server REST interface to register a test run on the server, and sets a property called dtTestrunID that contains an ID used to by the agent to tag all PurePaths belonging to a test run. This ID has to be passed to the dynaTrace agent instrumenting the JVM running the tests as a parameter called optionTestRunIdJava (see example below).
Now that you have told Dynatrace about the tests to come, you can move forward with actually executing the tests. To make sure our integration works, you have to make sure that the JVM running the tests has
a) a Java agent injected and
b) gets the test run ID set in the optionTestRunIdJava parameter.
Therefore, it is important to use fork="on" on the JUnit task as otherwise the tests would execute in the JVM running Ant, and you would not get an agent injected. The other configuration of the junit task looks just like it would without dynaTrace, we define a classpath and a fileset:
And that's it! You can continue with your build script as you usually would, for example with building an HTML report of JUnit results and bundling up artifacts. The full test target looks as follows:
- Download the Maven plugin from the Automation Library (Ant, Maven) for Dynatrace download page
Import the Maven plugin into your local repository using the following command:
In the properties section of the pom.xml, you can define Dynatrace-related properties that are used by the Dynatrace plugin. You can use this to define default properties that are then overwritten in the actual plugin configuration (for example, if you use multiple profiles for unit and performance tests):
As a prerequisite to use the Dynatrace plugin in your Maven build, add a dependency to your pom.xml:
Before you run your tests (JUnit, TestNG, NUnit), use the startTest goal of the Dynatrace plugin to register a new test run on the Dynatrace Server and pass additional metadata like the major/minor version. This way, the Dynatrace Server is able to identify PurePaths that originate in the same test run and allows you to perform parallel test runs. In Maven, you have two ways to achieve this: You can attach the plugin execution to a lifecycle phase that's happening before the tests are executed (recommended), or you can manually invoke to appropriate plugin goal (only for Dynatrace versions prior to 6.1).
Attaching plugin execution to lifecycle phase: This is the recommended of usage way as it guarantees that the startTest goal is always invoked when tests are run. In order to identify the correct phase to attach to you have to know which tests you are executing as Maven distinguishes between unit tests (executed by the Surefire plugin), and integration tests (executed by the Failsafe plugin), and of course it's also possible to use Dynatrace for both phases.
What is the difference between these plugins? The Surefire plugin will fail the build immediately when a test fails, while the Failsafe plugin will execute all tests regardless of any failure, and then look at the build results and fail the build if necessary builds (also see the Maven documentation). This means that with the Surefire plugin, a failing test will cause the build to stop in the test phase, while with Failsafe it is guaranteed to pass through the post-integration-test phase and stop in the verify phase if necessary.
If you want to use Dynatrace to monitor your unit tests executed by Surefire, take a look at the following configuration snippet:
Monitoring tests run by Failsafe works as follows:
Manually invoking Dynatrace goals (for Dynatrace versions prior to 6.1): You can also manually execute the Dynatrace plugin goals, for example as follows:
And that's it! You can continue with your build script as you usually would, for example with building an HTML report of JUnit results and bundling up artifacts.
For instructions on calling Dynatrace automation tasks from MSBuild, see MSBuild Tasks Library.
For instructions on calling Dynatrace automation tasks from NAnt, see NANT Task Library.