Is there a way to alert when transaction's response time is abnormally faster, for web application a faster response time is a good indicator, however looking at the response time of our payment app (not web app) abnormal fast response time indicates an issue needs attention.
I was suggested to use Custom metrics events, however different services will need different thresholds within the same app.
I don’t think there is any other option. What I can suggest is try to create dedicated metric for each service and setup thresholds. It may cos some work but I don’t think there is any other option right now.
Why not use the custom event for alerting feature to simply define a threshold for response time 'below' x ms for a selected service key request or web application key user actions.
Yes this is an option, however different times of day will have different response time, so auto alert on an anomaly (positive or negative) would be ideal. Also we have multiple services within our app, I have started going down the path of custom events.