Haraldur - You can simply uninstall on the agents you want to disappear. They will age out in 3 days.
You can also reset the data being used for Problem notification. Go into an entity such as a Service, click Edit, and then Anomaly detection. You can reset the reference period data using the button at the bottom
