Posthumously change event properties

[ewoelkerling]

Hi All, 

    I was hoping you could help me understand if something’s possible in Dynatrace – haven’t found the way so far, but not sure if it’s me or it just can’t be done..

The use-case:

• I’m creating metric based events based on a threshold breach (CUSTOM_ALERT) and then post-processing these events to update a downstream system.
• I’m filtering for these events using a eventType(CUSTOM_ALERT) and a custom dt.event.source property setting in the events (eg. DXC_EVENT) to specifically target the events that I’d like to see
• Events may or may not be ‘open’ during the processing, happy to adjust the 'timout' if that's useful
• I’d like to adjust the dt.event.source property setting once I’ve completed processing so that they no longer appear in information returned by the filtered query (ie. Change dt.event.source to DXC_EVENT_PROCESSED)
• FYI: These metric events are excluded from processing by DAVIS and are not rolled up into other events.

The thinking is I can retain state of which of these I have processed and which I have not simply by changing the dt.event.source property of the event, processing only the ones that appear (and making ‘processed’ ones no longer appear).  I’ve tried a few different ways using the API but am having no luck.  Any ideas if this is possible? 

All thoughts appreciated! 

[adam_gardner · Accepted solution]

Some assumptions from my side:

1) You cannot rely on the timeout as in your case, it really means nothing as the downstream system may not have processed the event yet.
2) You have the ability to send supplementary events into DT based on the results of your processing (eg. processing successful or processing failed).

Do these events create problems in Dynatrace? Is that how your downstream system is notified or are you periodically polling the API for new CUSTOM_INFO problems?

I wouldn't change the source and I can't find a way to modify a problem after it has occurred, but if you can get access to the raw JSON of the "original" problem then you know it's ID. So then when you're done processing (be that a day or a week later) you can fire in a second event which contains the ID of the original.

Then use DQL to do the diff and say "show me events grouped by problem ID and where problem IDs have an 'open' but not a 'close'".

I thikn this would be a great usecase for business events so tagging @KlausEnzenhofer here for his expertise.

[ewoelkerling]

Hi Adam, 

  Thanks for the thoughts - These do create problems in Dynatrace, and yes I'm polling the API periodically for new problem/event records matching the signature.  Given the API query I definitely have access the JSON so Interesting suggestion - I'll take a peek at it. 

Ultimately, I was hoping to keep it simple by modifying the event in some detectable/filterable way - that way I can just run one query which shows me anything outstanding to be processed, and then excludes anything already processed so to speak.  

In terms of the problem ID - I'm not seeing it in the JSON (only the eventId) - would it happen to be the correlationId field?  

Thanks for the help!