topic Re: state of nodes in Alerting

state of nodes

alangdima — Thu, 11 Jun 2020 19:14:20 GMT

Hi everyone, is there any way to validate the state (started,stopped,failed,restarted) from the api nodes of the applications?

I'm asking this because when some nodes are turned off or stopped, Davis doesn't give any alert.

Thanks a lot, regards!!

Re: state of nodes

dave_mauney — Thu, 11 Jun 2020 19:30:36 GMT

This would be a good use case for Synthetic HTTP Monitors, assuming the APIs are HTTP-based.

Re: state of nodes

dave_mauney — Thu, 11 Jun 2020 19:31:54 GMT

Also, if you have or can create a health check API you can use it to send events to Dynatrace via our Events API.

Re: state of nodes

Radoslaw_Szulgo — Fri, 12 Jun 2020 08:31:54 GMT

You can use REST /rest/state (no auth) to check the server's status - e.g. RUNNING - this can be used for Synthetic checks easily.

More info you can find under:

/api/v1.0/onpremise/cluster

example response:

[
  {
    "id": 1,
    "clusterMemberAddress": "1.1.2.2:5701",
    "operationState": "RUNNING",
    "buildVersion": "1.194.0",
    "osInfo": "Platform: Linux, Version: 5.3.0-1023-aws, Architecture: amd64, Processors: 16",
    "jvmInfo": "VM: OpenJDK 64-Bit Server VM, Version: 1.8.0_252, Vendor: AdoptOpenJDK, Memory [maxMemory=17408M, initHeap=17408M, maxHeap=17408M, usedMeta=12M, committedMeta=12M, totalPhysicalMemory=62851M, freePhysicalMemory=40939M]",
    "dnsEntryPointUris": [],
    "restServiceRootUris": [
      "https://ip-1-1-2-2.eu-west-1.compute.internal:8021/api/v1.0",
      "https://1.1.2.2:8021/api/v1.0"
    ],
    "communicationUris": [
      "http://ip-1-1-2-2.eu-west-1.compute.internal:8020/communication",
      "http://1.1.2.2:8020/communication"
    ]
  },

for each node. You can also find there "operationState" field. States can be:

OFFLINE(0),                // initial - server not running
STARTUP(1),                   // startup is in progress
STARTUP_CANCELED(2),         // startup was canceled due to some severe problem
RUNNING(3),                   // fully operational, accepts all data, fully visible
RUNNING_FORSAKEN(4),         // fully operational but invisible for ALL Agents, no paths should be sent to the server. configurable via config.properties
SHUTDOWN_PHASED_OUT(5),          // prior to shutdown ==> still operate messages but is not visible to agents anymore
SHUTDOWN(6),               // shutdown is in progress
// special modes
EMERGENCY(7),              // some emergency situation (e.g. low memory) - may happen during startup/shutdown or normal operation
STARTUP_SUSPENDED(8),        // basic startup finished but not connected to cluster or database, no tenants started
DATABASE_DISCONNECTED(9),     // database got disconnected during runtime, similar to READ_ONLY mode but without config requests
UNDEFINED(10),             // not known

SHUTDOWN_IMMINENT(11);       // state before SHUTDOWN_PHASED_OUT, rest api still works in this state, even if server was (is) in emergency

Re: state of nodes

dave_mauney — Fri, 12 Jun 2020 12:48:19 GMT

I assumed the question was about application API enpoints...but maybe this is a better take!

Re: state of nodes

mark_pundurs — Mon, 23 Nov 2020 14:56:46 GMT

@Radoslaw S. what is the complete URL for "/api/v1.0/onpremise/cluster"? Thanks!

Re: state of nodes

Radoslaw_Szulgo — Mon, 23 Nov 2020 15:54:34 GMT

You prefix that with your cluster node IP or cluster URL.