That's interesting since I'm running a browser test for about a year and yesterday the monitored website had an outage (for several hours) and Dynatrace did not notice the outage at all. The service resulted in HTTP code 503 (with valid HTML output showing an error page). I can see it in the run analysis, that Dynatrace got the error page, but no problem was generated and there is not an outage in the availability graph.
I've opened a support case (SUP-8685). My further investigation of synthetic runs revealed that although the waterfall displayed in Dynatrace reflected the error page which actually was actually server, the screenshot did not. So this really seems to be an issue in the Dynatrace synthetic engine.
Also, if Dynatrace detects a 200 response from any of the requests in an action, a single 500 response among many 200 responses will NOT trigger an error. Phillipp may be correct about the "main document" response, but I know for a fact that other errors can slip by unless you directly check the response code.