Does the order in processing step in pipeline matters?

[y_buccellato]

Good afternoon community,

in relation to Pipelines and Processing phase I'm not able to understand if processing rule will work all at the same time or if they will work in order by first matcher

So for example: what happen if I put the more generic processing step on top of the list and then I put the most specifc at the bottom?

Regards

[Peter_Youssef]

Hi @y_buccellato 

• Processing rules work in order based on their position in the list (top to bottom).
• If you place a more generic rule on top and a specific one at the bottom, the generic rule may prevent the specific one from being applied.
• The Order of course matters, so for specificity, place the most specific rules higher in the list.

BR,

Peter

[Peter_Youssef · Accepted solution]

Hello @y_buccellato 

Referring to Dynatrace documentation resources, PFA simple illustration:

• The order of processing rules is important and matters.
• The processing rules do not run in parallel or at the same time.
• They are applied sequentially based on the order in which they appear in the list, starting from the top.

Practically, How it works:

1. Order of Execution: The processing rules are executed as per the listed order, from top to bottom. This means the first rule in the list is applied first, followed by the second, and so on.

2. Matcher Specificity: Each rule in the pipeline can have matchers (conditions) that determine which events or data they apply to. If a rule at the top of the list has a more generic matcher, it could potentially match a broader range of data than a rule further down the list that has a more specific matcher.

3. Impact of Order: If you place a more generic rule at the top and a more specific rule further down, As a result:

   • The generic rule may apply first to a broader set of events or data, potentially including the events that would have been matched by the more specific rule.
   • The specific rule further down may not be triggered because the data has already been processed by the more generic rule. This can result in unexpected outcomes where the specific matching conditions are not applied as intended.

4. Tip: For ensuring the more specific rules take precedence, it's advised to keep the most specific rules higher in the list and the more generic ones lower down. This way, the more specific rules will match and process the relevant events first, and only if necessary will the more generic rules apply.

References:

• https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline
• https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/concepts/processing
• https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/getting-started/tutorial-co...

Logs:

• https://docs.dynatrace.com/docs/analyze-explore-automate/logs
• https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-openpipeline
• https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-processing

Hoping it adds value.

BR,

Peter