cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Kubernetes Number of EndPoints Custom Metric

Hi everyone,

I am currently trying to create a metric event which is creating a custom alert if a Kubernetes service has no valid EndPoints (hence #EP in frontend SVC == 0). I can see the configured ports for this service but I don't see how I can alert on the absence of EndPoints. Anyone any suggestions?

marina_pollehn_0-1689249354982.png

 

A Dynatrace Professional nerd working for Eviden
3 REPLIES 3

My alternative approach was to use the connectivity measure of the process - unfortunately the connectivity % is not shown there - also after restarting the containers. Anyone any idea? 😉

 

marina_pollehn_0-1689253239724.pngmarina_pollehn_1-1689253262530.png

 

A Dynatrace Professional nerd working for Eviden

florian_g
Dynatrace Mentor
Dynatrace Mentor

Hi @marina_pollehn - by no valid endpoint you mean that there is currently no pod in ready state for this service? Meaning there's no pod accepting traffic.
fyi @stefan_penner 

One does not simply run a container...

Hi Florian,

We've just had a meeting about this and the idea is that we want to be informed IF a Pod looses it's endpoint. 
By default the service is configured in the Kubernetes deployment. So this alert will be a check that nobody pushed wrong configurations.

For us: a Kubernetes service is using TCP 80 to guide traffic towards our frontend and backend pods. So the idea now is to use the Data Explorer and relationships in it to do a pod count:

builtin:kubernetes.pods:splitBy("dt.entity.cloud_application"):sort(value(auto,descending)):filter(in("dt.entity.cloud_application", entitySelector("type(~"CLOUD_APPLICATION~"),toRelationship.isKubernetesSvcOfCa(type(~"KUBERNETES_SERVICE~"),entityName.contains(~"SERVICE_NAME~"))"))):Avg():Default(0)

If this number drops to 0, all relationships between the pod and it's supporting service are lost, meaning the internal endpoint is not available anymore. Tomorrow we'll start testing it and hope this suits for us.

KR. 
Michiel
 

#Performance matter!

Featured Posts