topic Re: Data freshness for events and problems in Automations

Data freshness for events and problems

henk_stobbe — Mon, 22 Jul 2024 14:39:52 GMT

Hello,

Who can confirm below?

When using a "event" or "problem" trigger in a workflow, why does it take 5 minutes for you can retrieve by DQL:

Events belonging to a Problem

Problem belonging to an Event

For my workflow needed to wait 3x 1:59 minutes to make it work (timeout per task is 2:00 min)k

KR Henk

Re: Data freshness for events and problems

christian_kreuz — Thu, 25 Jul 2024 08:55:25 GMT

Hi,

there is a slight delay between when Workflows are triggered (by an event), and when data is available in Grail to be fetched.

Though I've never seen a delay above 30 seconds. Could you share the DQL Statements that you're using to fetch those events / problems? Are you using the DQL Action, or writing JavaScript Code?

Re: Data freshness for events and problems

henk_stobbe — Mon, 02 Sep 2024 12:24:59 GMT

Hello Christian,

This is the trigger:

Sending you the top/important part of the JS task:

import { execution } from '@dynatrace-sdk/automation-utils';

import { queryExecutionClient } from '@dynatrace-sdk/client-query';

import { eventsClient } from "@dynatrace-sdk/client-classic-environment-v2";

//Before this task already 2 tasks have been active with the same delay

setTimeout(() => {

console.log("1 minute has passed");

}, 119000);

export default async function ({ execution_id }) {

try {

// Get the current execution

const ex = await execution(execution_id);

// Extract event data

const data = ex.event();

const url = "https :// events.pagerduty.com/v2/enqueue";

// Initialize payload and custom_details if not present

data.payload = data.payload || {};

data.payload.custom_details = data.payload.custom_details || {};

const eventIds = data['dt.davis.event_ids'];

let allTags = {};

// Fetch event details and add them directly to custom_details

try {

for (const eventId of eventIds) {

const eventQuery = `fetch dt.davis.events | filter event.id == "${eventId}"`;

//console.log("**********",eventQuery);

const eventResponse = await queryExecutionClient.queryExecute({

body: {

query: eventQuery,

requestTimeoutMilliseconds: 60000,

fetchTimeoutSeconds: 60000

}

});

Retrieving the events will fail if if you do not wait for min 5 minutes.

(can share full script but than I need to sanitize first (-;)

KR Henk

Re: Data freshness for events and problems

christian_kreuz — Mon, 02 Sep 2024 12:21:20 GMT

A wait time of 10 seconds (let's make it 15 seconds to be on the safe side) should be enough. In your code example, I believe you're missing the await statement.

export default async function ({ execution_id }) { console.log('Starting to sleep'); const milliseconds = 15000; // sleep time await new Promise(resolve => setTimeout(resolve, milliseconds)); console.log('Done sleeping'); }

Can you verify that the execution of the step that should sleep takes the expected amount of time, e.g., 15 seconds as in my example?

Re: Data freshness for events and problems

henk_stobbe — Thu, 25 Jul 2024 13:45:54 GMT

Hello Christian,

This will make your day, I have now a 20s wait,

No problems!

Thanks for you help and guidance!

KR Henk