Dashboarding
Dynatrace dashboards, notebooks, and data explorer explained.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Release Monitoring – Focus on Kubernetes deployments

LeoBaracho
Visitor

Dear friends,

I am building a dashboard version based on the "Release Comparison Insights" dashboard, but focusing on a slightly simpler approach and strictly within the Kubernetes context, using metrics and logs. I used the standard labels from the documentation (https://docs.dynatrace.com/docs/shortlink/version-detection-latest#kubernetes-labels), but I am having trouble retrieving OOM and container restart metrics for the deployment pods.

I would appreciate your help in understanding what is needed to populate the container restart and OOM metrics. Ideally, I would like to display both metrics in the same tile, though separate tiles are acceptable if that isn't possible. I am also open to other improvements, provided they remain within the scope of Kubernetes metrics/logs (I plan to add traces at a later stage).

All the DQL queries were created with the help of Assist, so I am not sure if they were created in the most optimal way; I believe there is room for fine-tuning or refinement.

Regards,

2 REPLIES 2

LeoBaracho
Visitor

LeoBaracho_1-1792008541552.png

 

 

Geri_Puskas
Dynatrace Contributor
Dynatrace Contributor

Thanks for sharing the dashboard. I tested it against a workload with two releases, one OOM-killing and one healthy.

A few glitches in it:

  1. The “previous release” lookup was unreliable. It picked one ReplicaSet (limit 1) and ranked versions by Smartscape first_seen, which isn’t the ReplicaSet creation time. A version with several ReplicaSets was only partly matched, and the “previous” version could be the wrong one.
  2. Build tiles: they used undefined variables ($latest_build, $previous_build).
  3. CPU Throttle % reached 1,000–2,000%: cpu_throttled is millicores of unmet demand, so dividing it by the limit isn't a percentage.

What's changed:

  • Versions are ranked by deployment.kubernetes.io/revision, and all ReplicaSets of a version are matched.
  • Build tiles are now Revision tiles.
  • CPU throttling is 100 * throttled / (usage + throttled), which stays between 0 and 100%.

Patched JSON attached. If you'd rather see a flat 0 on healthy releases than "No records", that needs a zero-fill in the query. And these are just guesstimated ... as I don't 100% know what you wanted. But hopefully gives some ideas for further enhancements.

Featured Posts