Agile Development Practices have widely been adopted in R&D Organizations. A core component is Continuous Integration where code changes are continuously integrated and tested to achieve the goal of having "potential shippable code" at the end of every Sprint/Iteration.

In order to verify code changes agile team members write Unit or Functional Tests that get executed against every build and every milestone. The results of these tests tell the team whether the functionality of all features is still given and that the recent code changes have not introduced a regression.

Verify for Performance and Scalability

Now we have all these JUnit, NUnit, TestNG, Selenium, WebDriver, Silk or QTP tests that verify the functionality of the code. By adding dynaTrace to the Continuous Integration Process these existing tests automatically verify performance and scalability expectations. Besides knowing that the Product Search feature returns the correct result we want to know:

  • How much CPU and Network bandwidth it takes to execute the search query?
  • How many database statements are executed to retrieve the search result?
  • Whether product images will be cached on the browser?
  • How JavaScript impacts the Page Load Time in the browser?
  • Whether the last code change affected any of Performance or Scalability metrics?

dynaTrace analyzes all Unit and Browser Tests and validates execution characteristics such as number of database statements, transferred bytes, cache settings ... against previous builds and test runs. In case there is a change (regression) the developers are notified about what has actually changed.

dynaTrace automatically detects abnormal behavior on all subscribed measures, e.g.: execution time, number of database statements, number of JavaScript files, ...


Compare the difference between the problematic and the Last Known Good run

Besides being confident about functionality these additional checks ensure that the current code base performs and scales.

Step-by-Step Guide to enable dynaTrace in CI

In order to integrate dynaTrace we need to modify the CI Process. The following is a High-Level Step-By-Step Guide that explains all steps in a typical CI environment. When a new build is triggered the Build Server executes an Ant, NAnt, Maven (or any other type of automation script) script that will execute the following tasks:

  1. Check-out current Code Base
  2. Generate new Buildnumber and then Compile Code
  3. (dynaTrace) Start Session Recording (through REST)
  4. (dynaTrace) Set Test Meta Data Information (through REST)
  5. Execute JUnit, Selenium, ... Tests - (dynaTrace Agent gets loaded into Test Execution Process)
  6. (dynaTrace) Stop Session Recording (through REST)
  7. Generate Test Reports including dynaTrace Results (through REST)

dynaTrace provides an Automation Library that provides both a Java and .NET Implementation to call the dynaTrace Server REST Services. It also includes Ant, NAnt and Maven Tasks that make it easy to add the necessary calls to dynaTrace. The Demo Application includes a fully configured sample including Ant, JUnit and Selenium.

Once these steps are done dynaTrace will automatically

  • identify Unit and Browser Tests
  • learn the expected behavior of each individual test
  • raise an incident when tests start behaving unexpected

Captured results of tests are stored in individual dynaTrace Sessions which makes it easy to compare and share.
More details on Test Automation in the Online Documentation.

Conclusion

Take your Continuous Integration Process to the next level by adding Performance and Scalability Validations without needing to write any additional tests. This allows you to find more problems earlier in the development lifecycle which will reduce the time spent in load testing and minimizes the risk of production problems.

 

Selected Topics

Not Logged In? Customers and AJAX Edition Users Login with your Community Account