The good news is that some of this is already handled out of box.
Alerting uses the DAVIS baselining to learn your traffic data, so you shouldn't have to worry about false positives. It'll know that your traffic probably goes down overnight (and performance gets worse as bots skew the data), it'll know if you have weekend traffic drops, etc.
For data retention, you should have 35 days on SaaS. There is no cost for retaining the DEM data itself; capture is based on amount of billable sessions.
One of the most important initial configuration steps is setting up key user actions and naming rules. KUAs will give longer data retention and customized alerting thresholds for your key actions, and naming rules will help clean up your data. URLs (such as API calls) with unique values in them (eg. parameters, IDs, timestamps, etc) will get picked up as unique actions -- using naming rules gives you the flexibility to group them if needed.
Session/action properties are also very useful to set up. If something exists in a CSS selector (like an error message) or a JS var (eg. various pieces of metadata), you can associate them with the action or session.
