Our end goal is to have some way of automating the changing of incident states so that we can utilize smart alerting without accidentally missing an incident and thus missing an alert.
I have been looking into the REST interface where we can manipulate incidents but as I am relatively new to actually implementing something like this I had some questions.
My other idea if this proves difficult is to just have a scheduled report emailed out that shows if there are any unconfirmed incidents. The downside to this would be it is still possible to miss something if all actions are manual and so this is more of a fall back.
Solved! Go to Solution.
You can't change the state of multiple incidents at the same time. The two calls that change the state of incidents take a specific incident id as a mandatory parameter.
Yes, using a plugin running on a schedule is a reasonable way of proceeding. I like using the Generic Execution Plugin and writing whatever I want in shell / PowerShell. That way, I can debug the script outside of Dynatrace and then just drop the script into the plugin. Inside the script, you can call the REST interface via curl. One obvious caveat about this technique: the plugin runs in the Collector with whatever privileges the Collector process has.
Well, someone else might have a better idea ...Two more caveats about the doc page at
If you go down this path and you end up running curl on Windows, reach out to me at email@example.com and I'll send you what I have. I'm sure you're better at this than me, but getting the authentication and escaping right in curl on Windows had me pulling my hair out for a day or so.
Thanks Graeme, that helps a lot. Now I'm thinking about using the REST incident interface or just querying a dashlet to get a list of incident ids where the incident is unconfirmed (and maybe old if I can figure it out) and then looping through that and making REST calls to change the status to confirmed probably at a system profile level at first.
Did you end up writing a shell script or something to do what you described above? I'm looking for ways to periodically confirm closed incidents as well and think this is probably the best approach, but I wanted to know if you had any success with it.
Hey Jake - we basically decided against implementing it focusing on the design of incidents to limit the amount of spam (making them less sensitive as an example) but we did have an early prototype working where we actually used jmeter to pull the list of open incidents and cycle through them setting them as confirmed. Once installed on a collector if I remember correctly we were able to schedule this though we never moved forward with it. I don't believe I have much if anything from that but I can say that it is feasible, and actually a plugin would probably be the best option for that.