Manual Testing of OTEL Tracing

gopher — Wed, 15 Jan 2025 02:49:43 GMT

Not really a question but more a "how to" on testing an OTEL trace based on some fun I've had getting this going.

A couple of assumptions here.
1. you are on Linux
2. you are using otlphttp not otlp (grpc)

3. you've already got a functional and configured OTEL collector (GitHub - Dynatrace/dynatrace-otel-collector: Dynatrace distribution of the OpenTelemetry Collector ) that's listening for 'otlphttp' and connected to dynatrace otlp endpoints.

4. if you are running Istio, you have configured the required Service Entry and Net Pol to allow traffic to the otel collector service.

Anyway, next step is to

1. shell into a curl pod or target application pod.

2. switch to the /tmp directory

3. run the below script.

Things you might like to change
** the curl command is pointing at the default telemetry service, if you've customised your service address, you'll need to modify the endpoint "http://service.namespace:4318/v1/traces"

(e.g http://otel-collector-opentelemetry-collector.dynatrace:4318/v1/traces)

-- service name that is displayed in dynatrace.
"key": "service.name",
"value": {
"stringValue": "my.service_name"

-- span attributes that can be searched in dynatrace.

"key": "my.scope.attribute",
"value": {
"stringValue": "some scope attribute"

"key": "my.span.attr",
"value": {
"stringValue": "some value"

When it comes in, it will look like this.

Script is below, it will put in the recent timestamp and generate unique trace and span id's.

#!/bin/sh # Generate a unique traceId and current timestamp in nanoseconds TRACE_ID=$(cat /proc/sys/kernel/random/uuid | tr -d '-') SPAN_ID=$(cat /proc/sys/kernel/random/uuid | tr -d '-' | cut -c1-16) START_TIME=$(date -u +"%s%N") # Ensure the start time is 19 digits long with trailing zeros START_TIME=$(printf "%s%09d" ${START_TIME:0:10} ${START_TIME:10}) # Calculate the end time by adding 1 second (1000000000 nanoseconds) to the start time END_TIME=$((${START_TIME} + 1000000000)) # Ensure the end time is 19 digits long END_TIME=$(printf "%019d" $END_TIME) # Create a unique filename for the span data FILENAME="span_$(date -u +"%Y%m%d%H%M%S").json" # Create the span data with the unique traceId and current timestamp cat <<EOF > $FILENAME { "resourceSpans": [ { "resource": { "attributes": [ { "key": "service.name", "value": { "stringValue": "my.service_name" } } ] }, "scopeSpans": [ { "scope": { "name": "my.library", "version": "1.0.0", "attributes": [ { "key": "my.scope.attribute", "value": { "stringValue": "some scope attribute" } } ] }, "spans": [ { "traceId": "$TRACE_ID", "spanId": "$SPAN_ID", "parentSpanId": "EEE19B7EC3C1B173", "name": "I'm a server span", "startTimeUnixNano": "$START_TIME", "endTimeUnixNano": "$END_TIME", "kind": 2, "attributes": [ { "key": "my.span.attr", "value": { "stringValue": "some value" } } ], "droppedAttributesCount": 0, "events": [], "droppedEventsCount": 0, "status": { "code": 1 } } ] } ] } ] } EOF # Send the span data using curl curl -vvv -i http://otel-collector-opentelemetry-collector:4318/v1/traces -X POST -H "Content-Type: application/json" -d @$FILENAME # Print the timestamps and traceId echo "Trace ID: $TRACE_ID" echo "Start Time (nanoseconds): $START_TIME" echo "End Time (nanoseconds): $END_TIME" echo "Span data saved to: $FILENAME"

Hope it helps someone.

topic Manual Testing of OTEL Tracing in Dynatrace tips

Manual Testing of OTEL Tracing