topic Re: Monitoring state API for process groups (/api/v2/monitoringstate) in Dynatrace API

Monitoring state API for process groups (/api/v2/monitoringstate)

Julius_Loman — Mon, 12 Jun 2023 08:56:26 GMT

Dynatrace 1.261 introduced new API /api/v2/monitoringstate for querying status of monitoring for process group instances. At first sight this looks as a replacement for legacy process group API v1.

It seems this API is not yet in the documentation , @jaroslaw_orlows can you give a look?

I'd like to ask folks using the API already how can you use it on a larger scale?

I ran into two issues severly limiting usage of this API:

API does not support pagination and the response is limited to 500 entities. How do you use it? Querying entities indivudually is not really an effective way of API usage.
API does not seem to support filtering for monitoring state itself - typically I need to query processes where there an issue (like severity equals warning). There is just the healthState(UNHEALTHY) - which does not reflect the monitoring state.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

ChadTurner — Thu, 08 Jun 2023 16:42:57 GMT

im right there with you on this @Julius_Loman. Have you made any headway with this?

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Julius_Loman — Thu, 08 Jun 2023 17:17:38 GMT

@ChadTurner no, there is no progress I'm aware of. In the current state is not useful for us, we are still using the legacy v1 smartscape API for process groups (mostly detecting processes requiring restarts).

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

ChadTurner — Thu, 08 Jun 2023 17:27:18 GMT

Yeah I tend to fall back on the Legacy methods even when they are "Deprecated" lol

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Julius_Loman — Mon, 18 Sep 2023 09:18:49 GMT

Looks like some of my wishes sometimes make it to the product 😁
In 1.275 there is finally pagination supported as described in release notes. Still waiting for filtering on the monitoring states

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

MichalC — Fri, 19 Apr 2024 13:58:26 GMT

Hi,

I'd like to get a list of all process groups with status "restart_required". I supposed it should be entitySelector like this for monitoringState API: type("PROCESS_GROUP_INSTANCE"),healthState("HEALTHY"),state("restart_required") but there is response:

{
"error": {
"code": 400,
"message": "Constraints violated.",
"constraintViolations": [
{
"path": "entitySelector",
"message": "Scope: state. Cannot deserialize value: \"restart_required\"",
"parameterLocation": "QUERY",
"location": null
}
]
}
}

We need such option for filtering to replace depracated API: .../api/v1/entity/infrastructure/processes?restartRequired=true&includeDetails=true.

Please help.

Michal

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Julius_Loman — Fri, 19 Apr 2024 21:03:35 GMT

That's what I mentioned above. That API does not allow filtering on the severity or state value. You can only query by entitySelector and then iterate the output, which is far from being effective.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Julius_Loman — Wed, 29 May 2024 13:01:52 GMT

Hello @florian_AIgner or other Dynatraces responsible for this API.

There are still issues with using this API, please improve it:

entitySelector does not allow filtering on last seen time, despite the PGI does have this property. It's not flexible to loop through all process group (even non-deep monitored ones) which are inactive for years. There are no other options to specify the timerange.
entitySelector does not allow filtering on severity or injection state
Response should be optionally enriched with other related entities and properties - such as Process group display name, Host ID, Host name, and Management zones name, ...
Please add this to the documentation. This endpoint is not listed in https://docs.dynatrace.com/docs/shortlink/api-entities-v2 or anywhere else in the documentation

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

MichalC — Wed, 05 Jun 2024 12:51:00 GMT

Hi,

Dynatrace Product Managers please improve this API, we need improvements too.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

kurejko — Tue, 23 Jul 2024 14:29:19 GMT

Agree with Julius, some filtering and response enrichment would be very helpful here.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

ItamarMM — Thu, 08 Aug 2024 14:17:26 GMT

Please, the situation is so absurd, I don't think it is normal that we must have to do invididual calls over thousands of processes just to obtain a list of what we need to restart...

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

MichalC — Tue, 01 Oct 2024 11:37:16 GMT

Using this API we need to iterate through tousands of process instances at present to get a list PODs to restart. It's far from green coding and it's to slow to use it during our deployments. Dynatrace please improve it quickly.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

HeadScratcher — Thu, 12 Dec 2024 15:30:13 GMT

Also interested in using this API effectively...

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

lukegoyer — Thu, 19 Jun 2025 13:11:46 GMT

I'm running into this issue currently across a number of k8s clusters. Is there a way to identify the entities with DQL currently? I'm thinking I could create a workflow to restart the applications.

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

jerry_lobenstei — Thu, 19 Jun 2025 16:44:14 GMT

I run the API via a Notebook-(Dynatrace SaaS) that will pull the PGI STATE for all process or I can change to just the desired PGI's. Import the Notebook, Accept and Run.

* This function will run in the DYNATRACE JavaScript runtime via a Notebook.
* For information visit https://dt-url.net/functions-help

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

wiwinleung — Thu, 16 Oct 2025 05:54:50 GMT

Hi jerry_lobenstei

Thanks for your notebook and it is very useful.

However, I run into a problem of only returning 1000 records regardless of how I increase the maxResultsRecords or other parameters.

Anyone has idea of increasing the records?

Many thanks!

Wiwin

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

jerry_lobenstei — Fri, 17 Oct 2025 14:35:12 GMT

Default Limit:
If no limit command is explicitly specified in a DQL query, Dynatrace applies a default limit of 1,000 records. This default limit is automatically appended as the last line of the query.

Explicitly Setting the Limit:
You can explicitly define the maximum number of records to be returned by adding | limit <number> to your DQL query. For example:

fetch logs
| filter loglevel == "ERROR"
| limit 5000

Utilize options to limit the scanned amount of data
Particularly when dealing with logs, a large amount of data is ingested in short periods. Even with a narrow query time range, a query can fetch a considerable amount of data.
To stop reading data after a specified amount, use the scanLimitGBytes fetch command optional parameter. To scan a larger amount of data, also allowing timeframes to be set.
fetch logs, scanLimitGBytes:100

fetch logs, from:-2h, scanLimitGBytes:100

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Kekashri — Fri, 19 Dec 2025 16:49:53 GMT

Hi jerry_lobenstei

Thanks for sharing your approach.

We have a similar use case where we want to set up alerts for all services that were previously active but are now in an inactive state due to some issue and require a pod restart.

I tried implementing this approach and modified the query by adding namespace and management zone filters. I also updated this field:
const twentyMinutesAgo = new Date(now.getTime() - 72 * 60 * 60 * 1000);

However, I am only seeing 14 records, whereas there should be more in the last 72 hours. Additionally, if I do not use the namespace filter, the data returned is only for a single namespace instead of all namespaces.

Could you please help me figure out what might be causing this issue?

/* * This function will run in the DYNATRACE JavaScript runtime. * For information visit https://dt-url.net/functions-help */ import { monitoredEntitiesMonitoringStateClient } from "@dynatrace-sdk/client-classic-environment-v2"; import { queryExecutionClient } from "@dynatrace-sdk/client-query"; // **** Dynatrace API **** async function fetchAllMonitoringStates(pgID) { const pgID_string = `${pgID.map(id => `"${id}"`).join(",")}`; let nextPageKey = null; const monitoringStates = []; do { const params = nextPageKey ? { nextPageKey } : { entitySelector: `entityId(${pgID_string})`}; const result = await monitoredEntitiesMonitoringStateClient.getStates(params); monitoringStates.push(...result.monitoringStates); nextPageKey = result.nextPageKey; } while (nextPageKey); return monitoringStates } export default async function () { const now = new Date(); const twentyMinutesAgo = new Date(now.getTime() - 72 * 60 * 1000); const formatDate = (date) => date.toISOString(); const $dt_timeframe_from = formatDate(twentyMinutesAgo); const $dt_timeframe_to = formatDate(now); // **** DQL Query **** const dqlQuery = ` fetch dt.entity.process_group_instance | fieldsAdd host.id=belongs_to[dt.entity.host], pg_id=instance_of[dt.entity.process_group], cgi_id=(belongs_to[dt.entity.container_group_instance]) | lookup [fetch dt.entity.host], sourceField:host.id, lookupField:id, fields:{host.name=entity.name, host.hostGroupName=hostGroupName} | lookup [fetch dt.entity.process_group], sourceField:pg_id, lookupField:id, fields:{process.name=entity.name} | lookup [fetch dt.entity.container_group_instance | fieldsAdd pod_id=belongs_to[dt.entity.cloud_application_instance], namespace_id=belongs_to[dt.entity.cloud_application_namespace], node_id=belongs_to[dt.entity.host] | lookup [fetch dt.entity.cloud_application_instance], sourceField:pod_id, lookupField:id, fields:{pod_name=entity.name} | lookup [fetch dt.entity.cloud_application_namespace], sourceField:namespace_id, lookupField:id, fields:{namespace=entity.name} | lookup [fetch dt.entity.host], sourceField:node_id, lookupField:id, fields:{node_name=entity.name}], sourceField:cgi_id, lookupField:id, fields:{pod_name, namespace, node_name} | fields managementZones, host.id, host.name, host.hostGroupName, process.name, pod_name, namespace, node_name, id | sort process.name desc | filter namespace == "test-v2" | filter host.hostGroupName == "TEST.PROD" // | limit 5000 `; // **** Execution Performance Limits **** // const queryExecution = await queryExecutionClient.queryExecute({ body: { query: dqlQuery } }); const queryExecution = await queryExecutionClient.queryExecute({ body: { query: dqlQuery, fetchTimeoutSeconds:5000, defalutScanLimitsGbytes: 50, maxResultsBytes: 1000000, maxResultsRecords: 3000000 } }); let dqlResults = []; while (true) { const queryPollResult = await queryExecutionClient.queryPoll({ requestToken: queryExecution.requestToken }); if (queryPollResult.state !== "RUNNING") { dqlResults = queryPollResult.result.records; break; } } let pgID = dqlResults.map(item => item.id); const processAllChunks = async () => { const allMonitoringStates = []; for (let i = 0; i < pgID.length; i += 50) { const chunk = pgID.slice(i, i + 50); const chunkStates = await fetchAllMonitoringStates(chunk); allMonitoringStates.push(...chunkStates); } return allMonitoringStates; }; const monitoringStates = await processAllChunks(); const mergedResults = dqlResults.map(dqlItem => { const match = monitoringStates.find(apiItem => apiItem.entityId === dqlItem.id); return match ? { ...dqlItem, severity: match.severity, state: match.state } : dqlItem; }); const filteredResults = mergedResults.filter(item => item.state && item.state.toLowerCase().includes("restart") ); return filteredResults; }

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

gasperetta — Wed, 25 Feb 2026 10:32:57 GMT

thank you so much for this.

Is possible filter for state, i needed only the state "restart_required_outdated_agent_injected"

thanks for reply

topic Re: Monitoring state API for process groups (/api/v2/monitoringstate) in Dynatrace API

Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Utilize options to limit the scanned amount of data

Re: Monitoring state API for process groups (/api/v2/monitoringstate)

Re: Monitoring state API for process groups (/api/v2/monitoringstate)