can we setup service now and pagerduty for security problem notifications?
Yes, this is supported. Check documentation:
Send Dynatrace notifications to ServiceNow
ServiceNow
Send Dynatrace notifications to PagerDuty