Is it possible to re-process the zdata files which in past time stamp? I disable the data-feed of a AMD (say 11:00am) and enable back after 6 hours (say at 5:00pm), from the CAS report I only see data since 5:00pm, it seems it didn't process the zdata between 11:00am to 5:00pm, I check those zdata files are still on the AMD.
Much appreciate for any comment/suggestion.
CAS can process data only if it's newer that latest interval already processed by CAS.
This means that it's not possible to "fill" any "data gaps".
But if you have disabled data processing at 11am and then started it on 5pm, then CAs should start processing 11:05 data, then 11:10, 11:15, etc ...
The only problem could be that if you have skipped 11am-5pm data either in CAS or deleted on the AMD.
Please share output of http://your-CAs/rtmdebug screen.
If the report server is reading from multiple AMDs and you disabled one of them, your only option to process the effectively skipped data files is to shutdown the report server and restore the database from a backup taken before you disabled the datafeed, then let the report server reprocess everything from all AMDs to catch up again.
If this is the case so what will happen if we have only CAS
server and that goes down.
According to documents AMD can keep data for the 420 Minutes or
7 Days, so I thought that whenever we will bring back our CAS server before 7
Days the data will be reprocessed again automatically without any gap in the
monitored data. It means my understanding was wrong.
Can you please clear my concept
in the case of failure of reporting server?
Erik, yes I'm using multiple AMD. I thought enable/disable data feed is per AMD so it should process the "time-gap" zdata when data feed is enabled back. In my case, if I don't disable the data feed of problematic AMD, it caused CAS not process the normal AMD's zdata and the user interface will show "delay in performance data processing is x hours .....". If problem happen again, should I delete the problematic AMD from the CAS config instead of disable data feed? Or the product design won't trace back those old zdata even AMD connect back to CAS?
Badly I don't have any DB backup.
The AMD does buffer the data, but that is only for a CAS down
situation; zdata is a synchronous data type, so whatever zdata files are
available from all data sources when the CAS processes the data are
merged into a single data and then processed by the CAS. Once that data
set is processed, there is no "going back" short of restoring the
database to a backup taken before the data was processed.
enable/disable is per data feed, but disabling a data feed excludes
that data from processing entirely. Deleting the data feed would have
the same effect.
How is this AMD "problematic"?
Normally a delay in data processing like you are describing is caused by
overloading the CAS, and you should consider adding CAS hosts as secondary
nodes to better distribute the data load or reconfigure the Software
Services to reduce the data load.
If you have an AMD that is important to be in the CAS data set, but is problematic, you can mark the AMD as 'Priority' in the RUM Console (CAS Data Source configuration), the CAS will not move forward with processing until the priority AMD(s) are back, this will result in processing delays (no processing occurs) if a priority AMD is unavailable.
Once the priority AMD is back, the CAS will continue on form where it left off. Leaving you with a complete data set from all AMDs, but with some delays to catch up on.
Does this setting make any difference if a CAS is using an AMD and Synthetic data as two data sources? Or is it only between multiple AMDs? For example in a situation where the connection between the AMD and CAS is disrupted, the Synthetic data keeps coming in and the AMD is still monitoring.
Thanks Guys your valuable responses. To consolidate the above, it seems the issue I met is related to product design: one processing time stamp for al AMDs.
Actually I have two AMDs and both rtm are running fine, the only "problematic" component is the unstable network connection between the regional AMD. The heartbeat between AMD and CAS lost then lead to both AMD stop processing data (over an hour), thus I disable datafeed of the regional AMD to let the local AMD process data as usual.....so after datafeed enabled back on the regional AMD , CAS only process the time stamp base on local AMD.
Well, if the above theory is true, my next question will be "how to overcome"? What if CAS can separate the processing time stamp of individual AMD? ^^
@Sylvian Lam Hypothetically - If you want to have the data from both AMD's ASAP, and you don't want any wait for the other while you have a shaky network, you would need 2 CAS.
If you do like Chris is suggesting above, you'll have delayed processing as soon as you have any network problems, but I still think that is the best approach.
In that scenario your options are