Kubernetes operator: ActiveGates are missing a liveness probe

[Enrico_F]

We noticed that the pods of operator-controlled ActiveGate stateful sets are missing a liveness probe in their manifest - there is only a readiness probe defined.

This has led to multiple outages due to e.g. thread pool exhaustion which had to be resolved by manually killing the pods - not good.

Is there a deeper reason for this?? Or is this worthy of an RFE?

[Enrico_F]

Related PR for the Dynatrace operator:

feat: add ActiveGate liveness probe by realtimetodie · Pull Request #5391 · Dynatrace/dynatrace-oper...

[Michal_Gebacki]

Hello, @Enrico_F!

Is the issue you described a few months ago still relevant?

Thank you for any feedback!

[Enrico_F]

Hi @Michal_Gebacki 

We did not have AG outages due to unresponsive routing and k8s-monitoring services since mid-September, but we increased the mem limits after that event, so that might have helped.

However, I would consider having a liveness probe for the operator-controlled AG containers as best-practice, so in that sense I see this RFE as still very much relevant to prevent future outages (unless there were some improvements made for the AG images that would make this unnecessary).

HTH

[Michal_Gebacki]

Hello, @Enrico_F!

Just for clarification - have you created a separated, related RFE in product ideas subforum or should we transfer this particular topic into a RFE and move it to the product ideas subforum?

BR

[Enrico_F]

No there is no separate RFE (I mistakenly assumed I had posted this in the Ideas forum - sorry).

Since the PR feat: add ActiveGate liveness probe by realtimetodie · Pull Request #5391 · Dynatrace/dynatrace-oper... has been accepted and merged I think there is no need for an RFE anymore as I assume it will be part of a future operator release, can you confirm?

[Michal_Gebacki]

The link you're sharing is for a GitHub environment (and sadly, for me is not working). Can you specify about what product idea are you talking about? Is it the one posted in the Dynatrace Community?

[Enrico_F]

The link to the PR is public and points to the official Dynatrace Operator repository on github.com. If it's not working you should probably talk to someone internally.

Like I said, there is no product idea for this yet.