cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

This product reached the end of support date on March 31, 2021.

Incident End time wrong alert generated

ashutosh_singh1
Organizer

Dear Team,

We are facing issues with "Application process unavailable(Unexpected)" end time incident alert. What exactly is happening, Incident has occurred and alert generated for same once agent connection was lost and it was again connected within 2-3 minutes, But that incident was not ended at same time. it end after 3 days and sent incident alert after 3 days. Due to that customer got wrong notification.

Could you please confirm why is it happened and any work around for same?

Regards and Thanks,
Ashutosh Kumar Singh

6 REPLIES 6

JamesKitson
Dynatrace Leader
Dynatrace Leader

Check your agents overview and see if you can find an "indexed" agent (e.g. myagent[1]), you'll need to select the option to display offline agents. By default if an agent with the same name connects to the server it will get an indexed name and this can occur if an agent is restarted or reconnects while the server is still disconnecting that same agent with the result that it will see the same agent as a new connection and if there is an incident it will stay open. It could also be some other small change with the agent / agent name that caused it to see a new agent connection as opposed to a return of the previous one, more info would need to be looked at.

After three days is I believe the default for when it will automatically end incidents based on an agent being disconnected which would explain why that end/email occurred then.

James

Hi James,

Thanks for your response...!!

Agent was indexed as you shown in example. But it is not closed at same time and it is taking default time interval to close incident and send mail for same, that is why customers are getting wrong notification and escalating.

Is it any possible solution for same through it will close automatically once connected?

Regards,
Ashutosh

Hi Ashutosh, I think what you probably do is manually confirm the incident. Also can you highlight the use case to the incident ending for agent disconnect. AFAIK from experience, the agent disconnection should be handled on priority. And most of the cases the incident start alerting is sufficient.

However I do agree that once the original agent gets connected the fallback alert should be ended.

Regards, Rajesh

There is a debug flag you can add to the server that will prevent this indexing from occurring: https://community.dynatrace.com/community/display/DOCDT65/Unique+Agent+Names+with+Index

By default the value is true so if you add this you would want to set it as false.

-Dcom.dynatrace.diagnostics.collector.agentcenter.unique_agent_names_with_index=false

You used to have to add it to collectors, but more recent versions (including 6.5) you can add it to the server. We have found making this change prevents those open-ended process unavailable incidents from occurring and has made them much more usable.

James

ashutosh_singh1
Organizer

Hi James,

Thanks for valuable support....!!

I have one doubt with above mentioned property. Does this property can affect the agent connection. I mean to say, if above property will change into false and agent will unexpected unavailable and again try to connect, in that case is it possibility to refuse agent connection with same name.

Regards,
Ashutosh

No that won't be an issue, if you look at the agent names in the agents overview you'll see they add some additional info including the PID to the name as what the server sees so it will be able to tell these apart without the index. We've had this in place at a pretty big deployment for a over a year now probably and that has not been an issue. All it will do is remove that indexing mechanism.