cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Dynatrace release monitoring

kapilkodwani
Visitor

Hi, I have been trying to configure release monitoring for services running in kubernetes.
I can see that there is a new release when i have supplied to appropriate env and labels as shown in this dynatrace document.


I can see it being released in release inventory see picture
Screenshot 2022-11-10 at 8.41.29 PM.png

but there nothing happens  when I go check my service from services page.

Screenshot 2022-11-10 at 8.44.53 PM.png

6 REPLIES 6

rgarzon1
Pro

hi  kapikodwani

 

did you check the process page ? 

 

what CI/CD are you using ? or you are only setting the env and labels for the nodes ?

 

i know that maybe this its not what you cant bu if nothing works you can try

config in your CI/CD the post of the event to the service that you want to ? like this 

events ingestion

 

hope it helps

 

fuelled by coffee and curiosity.

 Thanks for the reply,
> what CI/CD are you using ? or you are only setting the env and labels for the nodes ?
we are using argo-cd to deploy helm package into our kubernetes cluster. I have attached a demo deployment file (I hade to hid some sensitive data). It sets the env variable to the pod.

I have checked processes, process groups, process group instances as well as service (that is created as a result of deploying this spring application), I don't see any deployment event.
The only thing i see is a release being created and nothing else.

> i know that maybe this its not what you cant bu if nothing works you can try

config in your CI/CD the post of the event to the service that you want to ?

I tried doing this and it works, I see the deployment event when i use events API. Is that the only way to do it? I would really ideally like it to work with kubernetes labels.

 

This is ringing a bell with my attempts at doing this.

 

I think as a user of DT we assume Release Management Inventory "updates" would explicitly be tied to service "events" and if I recall correctly I don't think it is, even if it reads as if it should. (or I completely misconfigured it!) 

 

I ended up abandoning the release page inventory for exactly this reason, But if it's changed or you can figure out an approach that doesn't require both setting up the below, let me know.

  • env_vars/k8s-tags - for tracking the release inventory 
  • event api - for listing the deployment event for a service/host/pgi

Last I checked the v2 event api also didn't support filling in all the data on the event itself.

 

I'd be keen to revisit this and get it working on my side. I'm not using K8s but I don't think the problem is around that.

 

I'll run some tests today and come back with my findings

 

 

 

1.Ran a docker container locally passing in the DT env Vars.
2.Redeployed same container incrementing the number in the vars 1.0.0 -> 1.0.1

no "event" is generated in the release inventory or the event stream for the host/pgi/service.

benjamin_mullan_0-1668723943598.png

benjamin_mullan_1-1668724077234.png

 

 

3. Hit the v2 event api to make an event record. Targeting all the relevant services within the container by name and instance (I didn't scope the release tag set by the ENV Vars' so both 1.0.0 and 1.0.1 were tagged. Easy enough fix)

benjamin_mullan_2-1668724154887.png

 

API Payload, you could add another tag here to scope it to the particular release by adding something like
tag([Environment]DT_RELEASE_VERSION:1.0.1) . 

 

 

 

{
    "eventType": "CUSTOM_DEPLOYMENT",
    "title": "Testing Events",
    "entitySelector": "type(PROCESS_GROUP_INSTANCE),tag([CONTEXTLESS]Instance:B01),tag([CONTEXTLESS]Container:web-api-v2)",
    "properties": {
        "title": "Testing Events",
        "deploymentName": "Testing Events 1.0.1",
        "deploymentVersion": "1.0.1",
        "deploymentProject": "Docker Test",
        "remediationAction": "N/A",
        "ciBackLink": "N/A",
        "source": "ADO",
        "ADO Build Number": "22-11-19",
        "Git commit": "N/A"
    }
}

 

 

Setting the Tag to the release does indeed target only that release.

benjamin_mullan_3-1668725140908.png

 

 

Thanks for such a comprehensive reply.
I am not able to implement this with just using kubernetes env variables even after having a discussion with support.

The only way I can get to see a deployment event is through implementing the EVENT v2 API in our CI/CD and I have followed your steps.

My question would be now after implementing it? How do I see how is the new deployment working comparing with the last one? I cant find much resources regarding how to actually take advantage of this beside a deployment event marker being there.

No problem! I thought it was worth going through again as I had the same problems as you did (Expectation vs Reality)

My intention was only ever to notify of a release in Dynatrace so I could see if it was the root cause of any deviation. I'm in a position where if there is a release and no problem created after it then the release is no worse (within the anomaly thresholds for alerting) Its trust based 🙂

The service it self can be filtered by "Release Version" on the services pane. So you should be able to scope all the telemetry by that. There is also a Compare changes over time. button on the services you could open two of these and scope to each release respectively? I'm not sure if there's a one stop shop to get where you are going of if this even helps.

I'm not sure at a glance if this release tag is available at the analysis view or the Data explorer query engine to SPLIT upon, you can filter but not split. This is where a Custom Metric might be used to add this dimension. I would have hoped it was built in but it seems like this is not the case, yet.

benjamin_mullan_0-1669236528596.png

Others may well have better approaches to analysis on service versions over time and while running in parallel (A/B). This is not something I have used in the past but that's how I would do it with my current understanding of this system.

Featured Posts