We are in the process of creating a high availability system for one of our customers. Essentially we have a setup where one of the DT servers is in an idle state but ready to go if the active DT server fails. We need to ensure that the passive DT server always has the right configuration. Currently we run the dynatrace-migration.jar on a nightly basis but this backups all of the Dynatrace configuration and can be in excess of 1GB. We want to be in a position where we can run the sync every x minutes. I am wondering if there are some parameters we can set when calling the dynatrace-migration.jar which will only backup the key files like profiles/dashboards etc etc rather then everything or is our only option to create our own xcopy type job which could check to see if the file has recently changed and then copy over if required.
If Xcopy is our only option do we have a list of files/folders that we should be checking for changes
using the migration tool is the preferred way, as it is complex to backup all required config files and not too many. Unfortunately not all files to backup are in /conf/ and not everything in /conf/ should be backed up.
You can start by opening the .dtma file with a zip tool and check what takes up so much space.
Typically two things require the most space in the migration archive:
1) Collector class cache
2) agentres files
As you have a production server, you should likely use a server-only installation and have the collector on separate machines. If you have a collector move it to another machine and use the -noclasscache flag to ignore those files with the migration tool. Starting with 7.1, for linux there will also be a server-only installer that will not install the collector files onto the file system to get rid of a few more files there.
For the files in the server/lib/agentres-* and collector/lib/agentres-* : these are not configuration files, but they come along with each update - however they are still an important part of the installation, which is why they are backed up - if missing they can cause agent crashes.
Starting with 7.1 through the story JLT-186154 you can keep the number of the agentres files by updating agents, removing then the old updates and on restart the server will delete the unneeded files - which will drastically reduce the dtma files if you installed a lot of updates.
For older versions, you can:
0) Make a FULL backup with the migration tool before you do anything.
1) For each server/lib/agentres-X.jar where X is a version, make sure that no agent with version X is connected and no one with such a version will ever connect again (offline agents).
2) Shut down server
3) Remove the server/lib/agentres-X.jar
Do NOT remove the server/lib/agentres.
Safer is of course to wait for 7.1 (see also EAP).
Thomas, thanks for the response. In our environment we have the collectors on different servers. In our situation the question really is do we have to run a full migration backup every x minutes as surely most configuration files wouldnt change once initially copied over? Therefore looking for alternatives which reduces the size of the file but ensures that the standby server has the correct/most up to date configuration at the same time. Once the standby server has the server/lib/agentres-* then surely we wouldnt need to keep copying this over?.
Our ultimate aim is to run some kind of differental type backup every 5 minutes but we still run a full backup once per day or week
The agentres files will actually never change, new agentres files come along as you install new updates. So for full correctness, you could take full backups after installing updates (need server restarts anyways).
In general regarding configuration files, this depends also on your users, when a user makes a changes the config files change (note that here "config" also includes things like dashboards, web dashboards etc).
Another idea would be a request for enhancement for the migration tool to skip those (see other forum).
I've put in the ticket JLT-206033 for the migration tool to allow the exclusion of the agentres - but cannot promise any delivery dates here.
For now, I would say yes, there is no other satisfying option that I can think of.
a new version of the migration tool was just released: 10.1.0.1004, which contains the parameter "-agentres no". Together with "-noclasscache" this should now exlude all larger files and run very quickly and could work for your use case.
Note that you should run the full backup still after each update installation.
For others reading this: should not for a normal upgrade/migration scenario.