Jenkins Pipeline Analytics using OpenTelemetry and Dynatrace

anton_konikov · ‎27 Sep 2024

Prerequisites

Jenkins as CI/CD tool
OpenTelemetry plugin for Jenkins - https://plugins.jenkins.io/opentelemetry/
Generate API token in scopes: openTelemetryTrace.ingest, metrics.ingest, logs.ingest.

To illustrate, we’ll employ Podman as our container runtime and set up Jenkins within it. For that, you could use the following command:

podman run --name jenkins -d -e OTEL_LOGS_EXPORTER="otlp" -p 8080:8080 jenkins/jenkins:lts

Once the container is up and running in your favorite browser, let's navigate to http://localhost:8080/ and, follow the shown instruction and login into Jenkins.

As the second step, from the main Jenkins view Dashboard, click on Manage Jenkins and then on Plugins:

On this step, let's install OpenTelemetry plugin. Select Available pugins and type OpenTelemetry in the search bar. Select the plugin and depending on the Jenkins version, press Install button or Install without restart.

After the installation is sucesfully finished, scroll down and select Restart Jenkins when installation is complete and no jobs are running:

Configuration

Now let's go to Dynatrace UI and generate an API token

From Manage Jenkins view, navigate to System configuration:

Scroll to the OpenTelemetry section and in the OTLP endpoint sectrion specify your SaaS OTLP API endpoint, e.g. https://{your-environment-id}.live.dynatrace.com/api/v2/otlp, in the Authentication section, select Header Authetication and type in the filed Header Name: Authorization:

For the Header Value, create a new secret tocken by seceting Add > Jenkins. You’ll need to create a secret text with the following fields:

For Kind, choose Secret text.
The Secret field should be in the follwoing format: Authorization: Api-Token dt.....
Optional field ID can be left blank.
Optionally add a description such as Dynatrace API ingest token.

Then scroll down to Advanced configuration and expand it. In the Configuration properties filed paste: otel.exporter.otlp.protocol=http/protobuf, as gRPC is not yet supported:

Additionally you can configure visualisation backend to show the link between build and trace in Dynatrace:

Receiving Jenkins data in Dynatrace

Please make sure that OTLP Endpoint set up earlier is accesible from Jenkins location. Let's create the first test pipeline and verify the monitoring data can be observed.

Click on New Item and create the Pipeline job:

Then scroll down to the Pipeline configuration and select the Github+Maven sample or paste the bellow snippet:

pipeline {
    agent any
    tools {
        // Install the Maven version configured as "M3" and add it to the path.
        maven "M3"
    }
    stages {
        stage('Build') {
            steps {
                // Get some code from a GitHub repository
                git 'https://github.com/jglick/simple-maven-project-with-tests.git'

                // Run Maven on a Unix agent.
                sh "mvn -Dmaven.test.failure.ignore=true clean package"

                // To run Maven on a Windows agent, use
                // bat "mvn -Dmaven.test.failure.ignore=true clean package"
            }
            post {
                // If Maven was able to run the tests, even if some of the test
                // failed, record the test results and archive the jar file.
                success {
                    junit '**/target/surefire-reports/TEST-*.xml'
                    archiveArtifacts 'target/*.jar'
                }
            }
        }
    }
}

Note: the prerequisite is to install Maven, which requires defining Maven instance in the Mange Jenkins > Tools:

After finishing the configuration of the pipeline job you can click on the Build Now button which then trigger execution of the pipeline:

As it can be seen in the screenshot above, under OpenTelemetry appeared a link navigating to the particular trace in Dynatrace UI. In the trace view you could see all logs from the Jenkins build by clicking on View logs for this trace or click on the particular trace sub-node/span and open Logs tab: