# Avoiding host unavailable alerts for GCP preemptible VMs?

**URL:** <https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648>\
**Category:** Container platforms\
**Tags:** google-cloud-platform, kubernetes\
**Created:** [October 23, 2020, 8:27am UTC](https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648 "2020-10-23T08:27:05Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![r\_weber](https://dub1.discourse-cdn.com/dynatrace/user_avatar/community.dynatrace.com/r_weber/32/24884_2.png) [@r\_weber](https://community.dynatrace.com/u/r_weber)\
**Post date:** [October 23, 2020, 8:27am UTC](https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648/1 "2020-10-23T08:27:05Z")

</div>

Hello,

I was wondering if anyone has worked with [Googles Preemptible](https://cloud.google.com/preemptible-vms/) nodes. These are basically discounted resources that have a max lifespan of 24hours bur can be terminated at any time.

This is quite useful to use with GKE (K8s) as K8s takes care of rescheduling workloads.

However when using with Dynatrace when these hosts get killed they will create host unavailable alerts.  
In the case of a K8s cluster this should be handled differently and not alerted.

First I thought of just creating a customized anomaly detection based on tagging these preemptible nodes, but that is not possible in Dynatrace.  
Then I thought of creating an alerting profile that **excludes** these tagged hosts but that is also not possible (see here: [https://community.dynatrace.com/idea/241247/view.html](https://community.dynatrace.com/idea/241247/view.html))

Then I looked a bit deeper into Google’s documentation and found that when a preemptible node is being shut down GCP sends a ACPI G2 Soft Off signal that should be captured by a user script to ensure a graceful shutdown of the services running on a preemptible host.

For me this seems to be the logical step, let the Dyantrace agent detect this G2 Soft Off signal and react on it by performing a graceful shutdown. This would then not lead to an alert in Dynatrace. Is this something that the oneagent operator can be enhanced with?  
I will create an RFE if no other solution exists as of yet.

Reinhard

---

<div class="post-metadata">

**Author:** ![adam\_gardner](https://dub1.discourse-cdn.com/dynatrace/user_avatar/community.dynatrace.com/adam_gardner/32/20087_2.png) [@adam\_gardner](https://community.dynatrace.com/u/adam_gardner)\
**Post date:** [October 27, 2020, 11:10am UTC](https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648/2 "2020-10-27T11:10:29Z")

</div>

I’d raise this as an RFE too, but in the short term, if your script can catch the G2 Soft Off signal, you can always use this:

```auto
POST to https://{environmentid}/api/v1/events
with a MARKED_FOR_TERMINATION event

```

---

<div class="post-metadata">

**Author:** ![r\_weber](https://dub1.discourse-cdn.com/dynatrace/user_avatar/community.dynatrace.com/r_weber/32/24884_2.png) [@r\_weber](https://community.dynatrace.com/u/r_weber)\
**Post date:** [October 27, 2020, 2:03pm UTC](https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648/3 "2020-10-27T14:03:59Z")

</div>

Not sure though what the marked for termination event would do on the problem detection AI engine then. If that ensures that the shutdown is seen as a graceful one then great - only have to ensure the DT event API is accessible.

I’d rather send this event to the oneagent on the host directly instead of the tenant event API.

---

<div class="post-metadata">

**Author:** ![khoros\_anonymous](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@khoros\_anonymous](https://community.dynatrace.com/u/khoros_anonymous)\
**Post date:** [October 27, 2020, 4:15pm UTC](https://community.dynatrace.com/t/avoiding-host-unavailable-alerts-for-gcp-preemptible-vms/20648/4 "2020-10-27T16:15:01Z")

</div>

Yeah, that’s the idea, avoid raising a problem for that host.

In my old org was used when sending a termination for short life cycle EC2 instances and a lambda executed the calls to the API in such events.
