cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

API rate limiting for automatied SRG validation?

r_weber
DynaMight Champion
DynaMight Champion

Has anyone encountered this?

I have four (similar) site reliability guardians, each guardian has 18 objectives, all based on DQL queries analyzing traces.
All four guardians are executed periodically by a single workflow (in parallel):

r_weber_0-1712311691741.png

The workflow often fails because it is hitting an API rate limit (apparently too many DQL queries are made in a short time).

This is a problem because the Site Reliability Guardian becomes unreliable 🙂
I thought of creating one workflow per guardian, but that is associated with additional cost (every workflow costs!) and it would duplicate the actions that I'd like to take after the SRG validation steps.

So maybe that API rate limit should be increased for workflows?

kR

Certified Dynatrace Master, Dynatrace Partner - 360Performance.net
2 REPLIES 2

ChadTurner
DynaMight Legend
DynaMight Legend

@r_weber - have you reached out to the Account Rep in terms of seeing about getting your limit increased? If so, where they able to raise it for you? 

-Chad

christian_kreuz
Dynatrace Advisor
Dynatrace Advisor

When having a lot of Guardians and evaluations, or generally a lot of load on Grail, SRG used to run into this problem.

With SRG 1.17.0, the team has "Increased Stability and Reliability in Heavy-Load Environments". Please give it a try!

 

Featured Posts