topic Re: Zscalar to Dynatrace Integration in Automations

Zscalar to Dynatrace Integration

MarwanC — Fri, 30 May 2025 07:58:29 GMT

I am trying to do an integration from Zscalar to Dynatrace which sends a specific payload which needs to be transformed to Dynatrace specific event payload to be ingested to Dynatrace. Is there a Webhook proxy in Dynatrace to do this transformation even if it is in the form of an extension. I appreciate your support on this urgent requirement.

Re: Zscalar to Dynatrace Integration

AntonPineiro — Mon, 26 May 2025 10:49:20 GMT

Hi,

You can ingest metrics or events in differents ways.

Best regards

Re: Zscalar to Dynatrace Integration

MarwanC — Mon, 26 May 2025 14:41:01 GMT

Thank you for the feedback Antonio, your feedback is very generic. The Zscaler send webhooks call and its payload is different than DT. A transformation is required. Any other comments on this?

Re: Zscalar to Dynatrace Integration

AntonPineiro — Mon, 26 May 2025 20:53:23 GMT

Hi,

It would depends which information you need to ingest, or if you are using Grail or Managed... I recommend you checking OpenPipeline where you can transform data before it is ingested in Dynatrace SaaS server.

Best regards

Re: Zscalar to Dynatrace Integration

MarwanC — Wed, 28 May 2025 11:30:55 GMT

None of the ingest worked for the this kind of interface, we get always webhook configuration error. Is there a webhook proxy in DT. If not id OpenPipeline the only solution for this.

Re: Zscalar to Dynatrace Integration

MarwanC — Wed, 28 May 2025 16:48:46 GMT

Bear in mind we are only sending events at the moment from Zscalar to Dynatrace

Re: Zscalar to Dynatrace Integration

Akhil-Jayendran — Wed, 28 May 2025 20:44:35 GMT

Hi @MarwanC ,

If you have syslog enabled on Zscaler, you might consider integrating those syslogs into Dynatrace. https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-ingestion/lma-log-investion-syslog

for Event only ingestion then may be ingest event via API should help ? [ you might adjust your playload accoding to DT ]
https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-api/environment-api/events-v2/post-event

BR,

Re: Zscalar to Dynatrace Integration

marco_irmer — Wed, 28 May 2025 21:43:40 GMT

We have had a similar challenge in our environment, where there was a sending entity whose payload could not be made to fit within the constraints of Dynatrace's available API endpoints (for logs or events). After many conversations with our Dynatrace account team, we had to admit defeat. Our eventual solution was to implement an intermediate (and custom) OpenTelemetry collector to receive the incoming data using a compatible receiver and then push them into Dynatrace using the OTLP Log exporter.

Re: Zscalar to Dynatrace Integration

Julius_Loman — Thu, 29 May 2025 20:14:05 GMT

Addionally two options not yet mentioned (if the source system can only emit data and call webhook)

Create a Dynatrace Extension (Python) which will listen on a webhook and send the data (event/logs/metrics... ). This will work for both Dynatrace Platform and Dynatrace Managed
Create an app function and call it externally. This might not be the most cost-effective solution if the webhook is called frequently.

Anyway, be sure to double-check and explore the standard options already mentioned. If it's log / event data, I'd use the standard Log ingest API and OpenPipeline to process it.

Re: Zscalar to Dynatrace Integration

MarwanC — Fri, 30 May 2025 13:03:28 GMT

Thanks Julius, for summary of the options available, OPTION ONE: is to create a complete extension for this simple event mapping from Zscaler to Dynatrace, what is the simplest extension documentation/code that you recommend to achieve this. Is this a long shot to achieve a simple interface? OPTION TWO: seems to be interesting, an app function will accept the event payload from Zscaler in its format and then transform it to a Dynatrace payload format event, the documentation refers to doing authentication by ourselves if we want to expose a function to the outside what is meant by this, is this app function written in python like in the same concept as a lambda function here?. OPTION 3: seems to be the most viable if I look at this but I am currently struggling with OpenPipeline concept: at the entrance and when it ends up in Dynatrace in the area of event, if I understand this, the Log ingest API is called, then we design a transformer/processor (that is clear clear) to map the event coming from the different payload to the Dynatrace Payload, the last part is confusing it ends up in grail and then what, where do we see it in Dynatrace, is only visible and extracted in a notebook using DQL, if you can kindly clarify those three options mentioned even though from high level and no hand on it will be much appreciated. The webhook interface is a standard interface in the industry, and ought to be supported out of the box in Dynatrace, the mapping is the only input required as an input - I do not wish to re-invent the wheel here and struggle with difficult solution to maintain. I appreciate any feedback on this.

Re: Zscalar to Dynatrace Integration

MarwanC — Fri, 30 May 2025 13:29:43 GMT

Thanks Marco, I fear I am dreading to the same fate at the end, "OpenTelemetry collector to receive the incoming data using a compatible receiver and then push them into Dynatrace using the OTLP Log exporter" Can you clarify the last part - OLTP log exporter? is that Dynatrace interface does it use the Dynatrace API? Thanks kindly for your reply.

Re: Zscalar to Dynatrace Integration

Julius_Loman — Fri, 30 May 2025 21:23:36 GMT

Option 1 (Extension) - I don't have anything prepared at the moment, but I'd choose this path if this has to be a reusable thing and has to work with Dynatrace Managed (Onpremise). Especially if you need to combine it with other data.

Option 2 - I'd choose this if you really need to write your own webhook and it won't be called frequently. Most likely, you don't need to choose this path at all.

Option 3 - Ingest using API (preferred). You call the log ingest API, construct the payload and headers so that the event gets ingested. Then with OpenPipeline, you can process the data (transform it, create a metric, set storage bucket, etc.). Since you sent it as log, it can be seen in the Logs app, or Notebooks/Dashboard or in any other app querying the logs in Grail such as Security Investgator.
If you are on Dynatrace Managed, you can still use this option, just it does not have Openpipeline, but you have other processing options.

Option 4 (bizevents, not yet mentioned, noting it for reference) is similar to Option 3, just the data has slightly different characteristics and purpose.

If you need some help on that, can you please elaborate more on the constraints of the webhook configuration on the zscaler side and provide same sample of the payload?

Re: Zscalar to Dynatrace Integration

MarwanC — Sat, 31 May 2025 09:13:53 GMT

Thank you Julius, your recommendations make sense, I already made the decision to use OpenPipeline, it seems to fit the solution, and the filtered data can be presented on dashboards or pushed to an another Davis Event which is the intention, but I am still long way from achieving that (the pipeline is easy to construct just using rename field will do the trick). The issue that I am having at the moment is basic, that Zscaler do not publish their payloads anywhere and we sort of trying to guess what kind of data is coming through to the logs. We wrote a simple python to push a sample Zscaler payload which we found somewhere on the internet and the ingest is going through giving response code 204 indicating a success? but we can not see the entry in the logs or the logs and event apps, any idea what search criteria based on this payload data ingested that we can search on, we checked the exact date but we can not find the entry. strange... any feedback is much appreciated. I am stuck here now.

[2025-05-31 11:00:22] Sending raw Zscaler payload...
Sending payload to: https://DT_URL
Headers: {'Authorization': 'Api-Token REMOVED', 'Content-Type': 'application/json'}
Payload: {
  "eventType": "SECURITY_ALERT",
  "timestamp": "2025-05-31T23:00:00Z",
  "source": "Zscaler",
  "alertDetails": {
    "severity": "HIGH",
    "category": "Malware",
    "description": "Suspicious activity detected",
    "affectedUser": "user@example.com",
    "affectedDevice": "MY_LAPTOP",
    "ipAddress": "192.168.1.100",
    "location": "London, UK"
  },
  "actionTaken": "Blocked",
  "correlationId": "a770faffe1384991"
}
Response status code: 204
Response body:

Re: Zscalar to Dynatrace Integration

MarwanC — Sat, 31 May 2025 09:30:50 GMT

We also used curl using the API part of DT, unable to find the entry, maybe I am not searching well 😞

curl -X 'POST' \  'https://DT_URL/api/v2/logs/ingest' \  -H 'accept: application/json; charset=utf-8' \  -H 'Authorization: Api-Token OUR_TOKEN' \  -H 'Content-Type: application/json' \  -d '{  "eventType": "SECURITY_ALERT",  "timestamp": "2025-05-31T23:00:00Z",  "source": "Zscaler",  "alertDetails": {    "severity": "HIGH",    "category": "Malware",    "description": "Suspicious activity detected",    "affectedUser": "user@example.com",    "affectedDevice": "L51W0V7",    "ipAddress": "192.168.1.100",    "location": "London, UK"  },  "actionTaken": "Blocked",  "correlationId": "a770faffe1384991"}'

Output:

Code

Details

204

Response headers

 access-control-allow-origin: https://URL_DT  access-control-expose-headers: *  cache-control: no-store,no-cache  date: Sat,31 May 2025 09:25:11 GMT  dynatrace-response-source: Cluster  pragma: no-cache  server: ruxit gateway  strict-transport-security: max-age=31536000;includeSubDomains  traceresponse: 00-d6fb8d25873473cd5eaad7bf8d84cb92-b781e801e0f0df41-01  vary: Origin  x-dt-tracestate: ec907795-f23c233@dt  x-robots-tag: noindex

Re: Zscalar to Dynatrace Integration

Julius_Loman — Sat, 31 May 2025 14:40:15 GMT

@MarwanC 204 response code means the data has been successfully ingested.

You can lookup the log entry for example by the API token it was ingested with (just the public part):

fetch logs | filter matchesValue(dt.auth.origin, "dt0c01.JUR2SVT6KBWHDJ7JWERHQ5UB")

If you still can't see it, please verify:

If the entry had not been removed in any OpenPipeline or in the classic pipeline by some processing rules
If you have permissions to query data from the log bucket where the was stored by OpenPipeline or classic pipeline

Actually, it's not important to know the payloads in advance; that's where the power of Grail comes in. You can just ingest it (maybe easily enrich it by adding log.source or some other data) and you can parse the content on query.

This is the curl example of yours in my environment (I just pointed it to my environment and used my API token. I executed the curl twice, that's why you can see two records here.

Re: Zscalar to Dynatrace Integration

MarwanC — Sat, 31 May 2025 16:31:47 GMT

Indeed thank you, I knew my data were ingested but did not search in an efficient way - thank you

Just a silly question, how did you change the filter box to be able to run DQL, mine is always forcing a filter, I used a Notebook to do the search (see screenshots) next step is now to build the pipeline - thanks very much for your feedback.

Re: Zscalar to Dynatrace Integration

MarwanC — Sat, 31 May 2025 16:32:47 GMT

Re: Zscalar to Dynatrace Integration

Julius_Loman — Sat, 31 May 2025 20:15:51 GMT

@MarwanC switch to DQL input in the logs app is hidden a little bit, I'd give it a more prominent space in the app (FYI @michal_franczak )

Re: Zscalar to Dynatrace Integration

MarwanC — Mon, 02 Jun 2025 07:40:36 GMT

Thanks for the reply. Indeed I agree may be the switch option should be at the top. As I mentioned that the simple ingest API call I conducted is either done from a curl or python program and this seems to work but when we do the actual test on the Zscaler intself and providing the the logs ingest call and the token: I am getting the following error: Failed : Invalid Webhook Config {"RESPONSE_MESSAGE":"The webhook configuration is not valid. Check the values of the configuration fields if they meet requirements."} - what is meant by the webhook configuration. Your feedback is much appreciated.

Re: Zscalar to Dynatrace Integration

Julius_Loman — Mon, 02 Jun 2025 07:47:24 GMT

@MarwanC that seems to be an issue on the zscaler side - not accepting the configuration you provided. Unfortunately, I don't have access to any zscaler environment. Can you share the configuration you are trying to do in zscaler?