Has anyone come across a 'Process unavailable' type problem being created for a process stopping but then when viewing the problem card you have something like 'State of process xyz is unknown'. The problem never actually closes out automatically even after the process is started back. Even manually closing the problem will cause another problem to be created for the exact issue. It takes several times closing out this problem to prevent it from coming back.
What I have seen is that this occurs when you enable process group monitoring for a process group then change the hostgroup of the agent and restarting the process from the process group. I understand that hostgroups are typically something you try to steer away from changing but it is sometimes unavoidable and it seems by doing so Dynatrace loses its mind a little with its process group.
Below are the steps I took. After my original post I was able to follow these steps to get the same type problem created
- Had 2 hosts with the same hostgroup. Process group was reflecting correctly and showing both processes as running
- Changed the hostgroup on just 1 host via oneagent ctl
- Stopped the process on the 1 host with the changed hostgroup
- Received an open problem card that was worded correctly (Process xyz on host xyz has been shut down)
- Started process on host that previously had the process shutdown
- I waited some time to see if the above problem would close out, it never did after 20 mins so I stopped the same process on the same host, trying to get the state of unknown problem
- Never received problem of "state of process is unknown"
- Started the process again almost 10 mins later after stopping it
- The above original problem never closed automatically. I expected this being that the hostgroup had changed and a new process group was created. Even after 20+ hours it was still open, I manually closed it
- New problem opened with text of "State of process xyz is unknown" immediately after closing the original problem