Hi @Ellery, If you are on SaaS as well as the new Platform - you could possibly leverage a workflow to generate an alert for the process that need a restart:
Here is an example of the 2 tasks that I have in a workflow that can possibly help alert on process that need to be restarted, one thing to preface is there since there is ingest and querying of data - there are costs associated with just running this and is really just meant to serve as an example.
I will upload a copy of the Workflow json so you can upload it into your environment.
1. DQL Query tile: to fetch a list of hosts - the results from this is to be used in a subsequent task
2. The second task is going to use a JavaScript action: The code fetches information about process group instances associated with a given hostname from Dynatrace. It checks the monitoring states of these instances, and if any state indicates a need for restart (e.g., contains “restart”), it logs information about the processes and ingests an event into Dynatrace indicating the need for restart, including the reason for restart as a property.
import { queryExecutionClient } from '@dynatrace-sdk/client-query';
import { monitoredEntitiesMonitoringStateClient } from "@dynatrace-sdk/client-classic-environment-v2";
import { actionExecution } from "@dynatrace-sdk/automation-utils";
import { EventIngest, EventIngestEventType, eventsClient } from '@dynatrace-sdk/client-classic-environment-v2';
async function getProcessInformation(hostname) {
const timeout = 60;
const query = `
fetch dt.entity.process_group_instance
| lookup [fetch dt.entity.host], sourceField:belongs_to[dt.entity.host], lookupField:id
| filter lookup.entity.name == "${hostname}"
`;
const response = await queryExecutionClient.queryExecute({
body: {
query,
requestTimeoutMilliseconds: timeout * 1000,
fetchTimeoutSeconds: timeout,
},
});
return response.result.records || [];
}
export default async function ({ action_execution_id }) {
try {
const pageSize = 500;
const results = [];
const actionExe = await actionExecution(action_execution_id);
const hostname = actionExe.loopItem.hostname_variable["entity.name"];
const processInfoList = await getProcessInformation(hostname);
for (const processInfo of processInfoList) {
const config = {
entitySelector: `type("PROCESS_GROUP_INSTANCE"),entityId("${processInfo.id || ''}")`,
nextPageKey: undefined,
pageSize,
};
const monitoringStatesData = await monitoredEntitiesMonitoringStateClient.getStates(config);
if (!monitoringStatesData || !monitoringStatesData.monitoringStates || monitoringStatesData.monitoringStates.length === 0) {
continue;
}
const entityName = processInfo['entity.name'] || '';
for (const state of monitoringStatesData.monitoringStates) {
if (state.state.includes("restart")) {
const result = {
processId: processInfo.id || '',
processName: entityName,
monitoringState: state.state || '',
severity: state.severity || '',
};
results.push(result);
const processRestartEvent: EventIngest = {
eventType: EventIngestEventType.CustomAlert,
title: 'Process restart needed',
entitySelector: `type("PROCESS_GROUP_INSTANCE"),entityId("${processInfo.id || ''}")`,
properties: {
'reason.for.restart': state.state || ''
},
};
await eventsClient.createEvent({ body: processRestartEvent });
break;
}
}
}
console.log(results);
return results;
} catch (error) {
console.error("Error:", error);
return "Failed to fetch process information and monitoring states";
}
}
This is how that would look like in Dynatrace
If all that you are trying to do is have this data displayed on a dashboard you can upload the dashboard json attached and see if that would work for your use case.
ProcessRestartInfo - Workflow and Dashboard.zip (3.1 KB)