I have the following warning:
Server cache limit has-been exceeded.
There are 50000 Servers in the cache limit is September to 50000 (50000).
This happened after adding static url with url parameter part groups
and this is not plotting any data
Central Analysis Server
version : 220.127.116.11
Any suggestions to solve
thanks for your suggestions
Solved! Go to Solution.
The limit you hit is set for performance reasons. Depending on your hardware, you might have some capacity left and you can tweak this. But the best way to do this is to go through you configuration and validate that you don't have things monitored/configured that you don't explicitly need.
You should also check your "Server IP Range". This is done by going to the advanced properties and changing those HTTP://YOURSERVER.COM/ATSCON and selecting "Advanced Properties Editor" and then specifying the "Accepted Server IP address range".
If this doesn't help and you DO have extra capacity in your hardware then you go to HTTP://YOURSERVER.COM/ATSCON and select "Capacity Settings Editor" and slightly increase the server settings.
If you add to much, you run the risk of slowing down the server database to much by having more data than it can manage.
Server limit also includes distinct urls, so that explains why it happened after reconfiguration of url parameter. Most likely there are too many values of this parameter resulting in thousands different urls populating the database. 50k is a lot, so i dont advise increasing this limit, rather take a look at how dynamic is the parameter you configured and tweak its definition to result in more aggressive aggregation of parameter values to single operations.
I also seem to be running into a repeated issue of exceeding the server cache limit. Today I just exceeded 30000 and had to bump it to 40000... The weird thing is that this is a very small RUM environment. It is just monitoring a single cluster of an SAP application. I would say there are no more than 15 servers this AMD is receiving traffic on (12.1, soon to be 12.2).
Could the SAP GUI traffic it is analyzing at the moment really be creating this many unique URLs and causing it to continue to hit this limit? Any guidance would be great... Thanks.
Hi Adam, thanks for the reply. Is this what you were referring to?
|Advanced DB Statistics||Raw sessions||246298|
|Advanced DB Statistics||Manual sites|
|Advanced DB Statistics||Users|
|Advanced DB Statistics||File defined sites|
|Advanced DB Statistics||Applications|
|Advanced DB Statistics||CIDR sites|
|Advanced DB Statistics||AS sites|
|Advanced DB Statistics||Predefined sites|
|Advanced DB Statistics||Servers|
|Advanced DB Statistics||URLs on servers|
|Advanced DB Statistics||Services (incl URLs)|
|Advanced DB Statistics||Unique URLs|
|Advanced DB Statistics||Services (excl URLs)|
|Advanced DB Statistics||Aggregated Sessions (AI)|
|Advanced DB Statistics||Aggregated Sessions (AS+AL)|
|Advanced DB Statistics||Server samples|
|Advanced DB Statistics||Network samples|
|Advanced DB Statistics||Server aggregated samples|
|Advanced DB Statistics||Usage samples|
|Advanced DB Statistics||Network aggregated samples|
|Advanced DB Statistics||Last sample retrieved|
reecently I get these errors quite often as well. I told the customer that we would need to go through the software service and check if we can get rid of any configured regular expression URL monitors. I was told they would need all the URLs which are configured there. Customer wanted to know what the impact of this warning/error is (Server cache limit exceeded). What is the impact? Performance data is lost? Would be great to get some insight in what the result of the error is... I couldnt find anything in the documentation.
Which DCRUM version are you running ? We got those errors at a time when we used 12.1.2 (I guess this was18.104.22.168). After the error was triggered no data - at least for our defined software services that use the HTTP Analyzer - was processed.
We got a patch for 12.1.2 and the advertising that the patch will be included in 12.1.3 (APMODCRUM-12273). Never checked it because meanwhile we migrated to 12.2.1 where the fix is included for sure.
Thanks for your answer, interesting point. I am facing it at a customer having version 22.214.171.124 (CAS).
I still have data shown in the start page of the CAS. However, I don't know if any data might be lost.
Unfortunately I can't remember how the start page looked in our environment. We regularly don't use the start page and instead assign custom made reports to the CAS users as personal starting pages. As far as I can remember no Real User monitoring data was processed. In any case 126.96.36.199 (that should be DCRUM 12.1 + SP1) is a version prior to the version we ran into the problem with.Therefore there is a chance that the error is there, too.
The patch I got from Compuware to solve the issue APMODCRUM-12273 was named "cas_is_not_saving_data_when_server_cache_limit_exceeds.jar". I assume you should have a look if maybe this patch is installed in your customer's environment but contact Compuware anyway to clarify if you suffer from data loss.
When server cache limit is exceeded, several things happen:
When a new data file (5-min zdata package) with new servers/URLs is processed by the CAS and cache limit will be exceeded in the middle of it - servers are processed first, then URLs are processed, so chances are that servers will be recorded and some (randomly chosen) URLs may be ignored.
Summarizing, if you have an existing server already known to CAS and you expanded number of URLs monitored to the extent that cache has been exceeded - you should still be getting complete measurements for the server level and complete measurements for some URLs (those that have been known to the CAS already plus, perhaps, some of the newly added URLs). Remaining URLs data will be rolled up into All Other Operations.
Please note that in the upcoming 12.3 DC RUM release we redesigned the way how auto-learning of servers is performed on CAS, so you will always be able to discover all software services on the network and report on them on the software service level plus aggregate of servers to a location level ("servers from location abc"), and then drill down to the list of serves that comprise the server location.
So what you are saying is that in the case of "Server cache limit ,,,," the total bytes for a server will be correctly recorded but the details for all URL's might be skewed?
So then the data for the URL's that can''t be added in detail will be collected under "All other" ?
So I ran into the same issue. I tried to configure monitored url's by rewriting the url as an 'URL as regular expression'.
Can I 'clear' the 'server-cache' somehow ? Since it is a quite fresh installation, I could restart capturing with a 'fresh' cache ?
Disscussed with Support I got the response that since we are running with the default value for the server cache, it could be possible to increase this if we had enough resources available, which looking at the system, they say we don't.
"The CAS on it's own should have 32GB RAM, this system only have 16GB with only 7GB assigned to the CAS"
So if the CAS was given more resources we could easily up the server cache limit, but that as it appears our only option is probably going to be to reduce the amount of data in the CAS by means of aggregation perhaps.
We have a customer that now hits the 50k Server Cache limit. The "Advanced DB Statistics" are below.
Obviously it's the URLs on servers, and unique URLs that are filling mostly the case.
In use are HTTP, SSL but also SAP decodes. In what way can we quickly get insight in which software services are generating the most URLs in use. And what the URLs are?
|File defined sites||0|
|Services (incl URLs)||28472|
|Services (excl URLs)||173|
|URLs on servers||49650|
|Aggregated Sessions (AI)||111|
|Aggregated Sessions (AS+AL)||30389|
|Server aggregated samples||2995216|
|Network aggregated samples||2995216|
I ran that script, indeed. Pointed me somewhat in the right direction. But to determine what URLs actually are filling up one needs to dive deeper in the results. What is the definition of sessions? It is not the URL entries in the Cache right? As I see it those are te end user sessions connecting to the software server/URL.
It is with a query like for instance below that gives a certain insight regarding the task generating a lot of URLs:
select task,count(*) as tel from (
select url,alias,task,service from dbo.rtmurl
where url like 'https://portal.customer.com%'
) as sub
GROUP BY task
ORDER BY tel desc
The output of your script:
|Software Service||Number of sessions|
|Operation name||Number of sessions|
|All other operations||208781|
What would here be the next approach?
Indeed this script is not answer which URLs are filling server cache (that is the most often reason) but only a direction which Software Services have the most sessions (that is usually related to number of URLs) so we know which one should be investigated in DMI by making very simple report with Software Service and Operation dimensions and Operations metric.
As an alternative you can remove top directive from SQL that queries for URLs.
Both here and in "DMI approach" you will be looking for unique URLs that has very small (usually just 1) operation/session. This used to happen when monitored application uses non standard cut separators, incorrect URL monitoring regexes that introduces unique/temporary/1-operation URLs.
And next step is fixing the configuration or if configuration is OK and it's just a matter of number of non-unique operations increasing server cache limit if CAS health permits.
Thank you, Adam. This now has become a good reference in the approach to attack the Server Cache Limit exceeded phenomenon.
Indeed with the simple DMI the URLs can further be investigated.
In our case we needed to fix an application with the right regexp that was generating 30K URLs, because an unique identifier was enclosed in the URL.
One question remains, that I think I have seen asked on the forum somewhere else. Can we somehow delete the server cache entries?
Just read this thread, am running 12.3.2 with same issues. Server cache keep increasing during the new year. From the SQL console result --I found most of sessions belongs to "Other TCP Proto" and "Other UDP Proto" while the largest contributor of "Operation name" is <empty>, yes, it consumes over 1 million session but the column name show nothing. Is there a way to decrease the server cache? Ideas are welcome. Many thanks.
|Advanced DB Statistics||Raw sessions|
|Advanced DB Statistics||Manual sites||25|
|Advanced DB Statistics||AS sites||0|
|Advanced DB Statistics||File defined sites||0|
|Advanced DB Statistics||CIDR sites||43544|
|Advanced DB Statistics||Predefined sites||1|
|Advanced DB Statistics||Users||6416|
|Advanced DB Statistics||Applications||3380|
|Advanced DB Statistics||Servers||338879|
|Advanced DB Statistics||URLs on servers||403325|
|Advanced DB Statistics||Unique URLs||31541|
|Advanced DB Statistics||Services (incl URLs)||65867|
|Advanced DB Statistics||Services (excl URLs)||26897|
|Advanced DB Statistics||Aggregated Sessions (AS+AL)||1484995|
|Advanced DB Statistics||Aggregated Sessions (AI)||133075|
|Advanced DB Statistics||Server samples||41635511|
|Advanced DB Statistics||Network samples||54095004|
|Advanced DB Statistics||Server aggregated samples||42795493|
|Advanced DB Statistics||Network aggregated samples||59443316|
|Advanced DB Statistics||Last sample retrieved||1/19/16 11:35|
|Advanced DB Statistics||Usage samples||77911072|
Referencing this post is some good background info on sever cache limit exceeded.
However, I do have a question about this posting
now that I have this information, what do I do with it? How do I use it to resolve the cache limit.
I apologize for asking a question that is probably pretty obvious to most here.
Thanks and God bless,
The question is very good, as I'm not sure if there is any topic speaking about what to do next ...
I would encourage to create new topic like "Server cache limit - how to resolve?" so we coudl group all answers there and give it a better visibility than page X in this thread ...