topic Re: Restore one sigle node in a 3-node managed cluster in Dynatrace Managed Q&A

Restore one sigle node in a 3-node managed cluster

DamianG — Mon, 10 Feb 2025 13:12:36 GMT

Hi Community,

I would like to know how to restore a single node in a 3-node managed cluster. Note: I have automatic backup enabled for the cluster.

In the Dynatrace documentation we can find instructions to restore the whole cluster (https://docs.dynatrace.com/managed/managed-cluster/operation/back-up-and-restore-a-cluster#restore-from-backup), but there are no details about how to restore a single node (or two) nodes of the cluster. In addition, I would like to know if it is possible to restore one node from a snapshot.

Possible scenario --> OS upgrade from RHEL 8.6 to RHEL9.4 in one of the nodes of the cluster. Steps to follow:

Take an snapshot of the node.
The OS upgrade of the node starts and it fails.
Restore from snapshot --> is this possible? if not, what to do?

Many thanks in advance.

Re: Restore one sigle node in a 3-node managed cluster

erh_inetum — Mon, 10 Feb 2025 05:33:53 GMT

Hi Damián,

Restore from snapshot is not supported. Support team told us and here you have another comment about it.

About restoring only one node it not recommended by Dynatrace because restoring less nodes than the original backup you risk losing the cluster configuration

Hope it helps.

Regards,

Elena.

Re: Restore one sigle node in a 3-node managed cluster

DamianG — Mon, 10 Feb 2025 07:36:02 GMT

Hi Elena,

Thank you for your comments.

What I meant in my post is that, in case only one node (of three) fails, I could restore only that node from snapshot because the other two nodes are still up and running without issues.

In addittion, in case the snapshot is not supported to restore one or more nodes, the restore procedure is based on a full restore of the cluster and I only want to restore one node because the other 2 nodes are already running. Does this mean that if one node fails (of three) I have to restore the whole cluster?

Thank you.

Re: Restore one sigle node in a 3-node managed cluster

erh_inetum — Mon, 10 Feb 2025 13:41:34 GMT

Hi Damián,

> What I meant in my post is that, in case only one node (of three) fails, I could restore only that node from snapshot because the other two nodes are still up and running without issues.

As far as I know it's not supported restoring nodes from snapshot. But I don´t know it would work. I've never tested it to be honest

> In addittion, in case the snapshot is not supported to restore one or more nodes, the restore procedure is based on a full restore of the cluster and I only want to restore one node because the other 2 nodes are already running. Does this mean that if one node fails (of three) I have to restore the whole cluster?

As it's said in the documentation, it seems it's possible restoring less nodes than the original cluster but it's not recommended because you risk losing the cluster configuration.

I hope have clarifying something

Maybe someone can provide a better answer than me.

Regards,

Elena.

Re: Restore one sigle node in a 3-node managed cluster

DamianG — Thu, 13 Feb 2025 14:08:15 GMT

Thank you Elena.

Following this thread. Do we have any restore procedure for ActiveGates? I couldn't find it in the documentation.

Thank you.

Re: Restore one sigle node in a 3-node managed cluster

erh_inetum — Thu, 13 Feb 2025 14:18:18 GMT

Welcome.

I think it's not possible to restore an AG. There is only a procedure to restore a cluster, not AGs:

https://community.dynatrace.com/t5/Dynatrace-Managed-Q-A/Restore-an-environment-from-backup-on-a-Managed-cluster/td-p/197659

Try to restart AG processes and if I doesn´t work, try to reinstall the AG.

Hope it helps.

Regards,

Elena.

Re: Restore one sigle node in a 3-node managed cluster

DamianG — Wed, 12 Mar 2025 09:33:45 GMT

Thank you Elena!!

In the other hand, a solution to restore only one node (provided by Dynatrace Support) could be the following, depending on the scenario:

Replace one node that is working: add a new node, disable OneAgent traffic on the affected node, and wait around 35 days (the default retention period for transaction storage).
That way, you'll be ready to replace the impacted node and won't lose any data.
Replace a node that is not working: remove it from the Managed Cluster completely and add a new node.

This is valid to such scenarios to avoid a full cluster restore.

Regards,

Damián

Re: Restore one sigle node in a 3-node managed cluster

erh_inetum — Wed, 12 Mar 2025 10:01:54 GMT

Hi Damián,

Thanks a lot for the information. It's very valuable.

Have a nice day.

Regards,

Elena.