Execute scheduled workflow only after the previous run has finished

[jegron]

Hi,

Is it possible to trigger a workflow only after the previous one has fully completed?

Right now, I’m facing an issue where a new run starts before the previous one finishes, causing it to reprocess the same data.

Julien

[dania]

Hi, each task within a workflow waits for the previous one to finish before continuing to the next. What might be happening is that the workflow is skipping tasks when an error occurs.
To avoid this, you can choose one of the available options to ensure that a task only proceeds after the previous one has completed successfully.

Alternatively, you can configure settings such as increasing the wait time between tasks or enabling retries a certain number of times.

These options help guarantee that a task only executes once the preceding one has run as expected.

[jegron]

Hi,

Thanks for help.

My issue isn’t related to individual task execution — tasks are running as expected.

The problem lies in the workflow execution itself: if the workflow doesn’t complete within a given interval (e.g., 1 minute), a new instance starts while the previous one is still running, leading to overlapping executions and chaotic behavior.

How can I ensure that only one workflow instance runs at a time and prevent this kind of overlap?

[dania]

Hi, what’s the event trigger for that workflow? You might need to define a more specific trigger to avoid overlapping with instances from other executions.

[jegron]

I tested using a scheduled event, but it turned out to be unreliable due to multiple parallel executions.

After that, I tried with an event trigger:

dt.automation_engine.workflow.id == "XXX" and
event.kind == "WORKFLOW_EVENT" and event.provider == "AUTOMATION_ENGINE" and event.type =="WORKFLOW_EXECUTION"
and dt.automation_engine.state.is_final == true
and dt.automation_engine.is_draft == false

What I was hoping for was to start the process manually once, and then let the loop continue running on its own. However, it doesn’t work that way. Is there some hidden limitation I’m missing?

[dania]

I’d recommend triggering the process on demand using the API. This way, you can set a wait time limit for example, one minute or more to help ensure that overlapping executions don’t happen again.

https://docs.dynatrace.com/docs/shortlink/workflows-trigger#on-demand-trigger

[johanoreel · Accepted solution]

Hi,

I'd like to suggest you an alternative approach. You could set up a "Time interval trigger" to have your workflow execute every X minutes (endlessly) and connect it to a DQL task that will essentially work as a gatekeeper. This DQL query checks the current state of the last execution. If it is still running, you could abort the newly started execution, by putting a condition on the next task (only run if the last execution ended). This DQL query could take the following form:

fetch dt.system.events
| filter dt.automation_engine.workflow.id == "XXX" // The id of the workflow you are working on
and event.type =="WORKFLOW_EXECUTION"
and dt.automation_engine.is_draft == false
| sort timestamp desc // From more recent to oldest
| limit 2 // Only keep the last two execution (the newly started one, and the previous one)
| summarize last_execution_ended = takeLast(dt.automation_engine.state) != "RUNNING" // Could be RUNNING, SUCCESS or ERROR

Of course, extra filters can be put in place, and the result can be presented differently.
But essentially, this DQL query will return "true" if the last execution ended and "false" if it is still running.

I hope this helps.

Kind regards,
Johan

[jegron]

Thanks a lot for the suggestion — it works in my case (though I’m using takeFirst instead of takeLast 😁).

There might be an edge case to handle if one of the two previously started workflows is still running.
For example:

• 9:00 → Still Running
• 9:02 → Autostop
• 9:04 → Running…

[johanoreel]

Taking into account your edge case, you could try to filter on all currently running executions and stop the workflow if you detect at least one:

fetch dt.system.events
| filter dt.automation_engine.workflow.id == "XXX" // The id of the workflow you are working on
and event.type =="WORKFLOW_EXECUTION" // It is a workflow execution
and dt.automation_engine.is_draft == false // It is not a draft
and dt.automation_engine.state == "RUNNING" // The workflow is currently running
| summarize count() // Count the running workflow executions

[jegron]

This request always count a running execution, since it runs every minute 😁

I built this one, and it seems to be working well. ✌️

fetch dt.system.events, from:-1h
| filter dt.automation_engine.workflow.id == "xxx"
and event.type =="WORKFLOW_EXECUTION"
and dt.automation_engine.is_draft == false
| sort timestamp desc
| summarize { collectArray(dt.automation_engine.state), count() }, by:{start_time, dt.automation_engine.workflow_execution.id}
| fieldsAdd execution_ended = if(`count()`> 1, true, else: false )
| filter `collectArray(dt.automation_engine.state)` == array("RUNNING")