Zscaler to Dynatrace Integration

Thanks once more Julius, the only screenshot I can share from Zscaler is whe we define the configuratin for DT and its beared token, I attach the screen shot below, I do not see any data ingested in DT after putting the API call/DT Toekn - this is needed to see what kind of Payload it is ingested int he content as we did before, once I know this payload the next step is to define the OpenPipeline for the integration but I am stuck in the first step which is essential before I can proceed. Do I need to change the strategy and use something else? See attached screenshot from Zscaler.

Test Webhook gives:

Failed : Invalid Webhook Config

{“RESPONSE_MESSAGE”:“The webhook configuration is not valid. Check the values of the configuration fields if they meet requirements.”}

@MarwanC you have the URL and the bearer token empty. Can you share the values you are defining there? Blur the middle part of the token and hostname of the URL.

We removed them for security reasons. But trust me they were put correctly.

One idea which spring in mind. If Zscaler send XML out would it still needed to say something to the Dynatrace API?

Also what is the difference of Access Token and Bearer Token, is it possible that Zscaler uses a differen format when injecting the header to send to Dynatrace? e.g; Authorization: Bearer <your_token_here>

@MarwanC

Based on your screenshot, you have 3 options to authorize. Basic, Bearer, OAuth. You were trying to send that with api-token authentication, which is a different method.

Dynatrace support Oauth2 - OAuth clients — Dynatrace Docs, what options do you have for oauth2 authentication on the zscaler side?

Thanks Julius for your useful input as usual, Indeed I noticed just in the last tests only that the GUI input is always forcing to add the Bearer in front of the DT token, what does it mean here for our API call to ingest it, the link you provided is not working for me. I also include the screenshot from Zscaler again to show that UI supports the three types, can the ingest call used with any of the three and  needs to be done to configure this?

does it mean we have been using the wrong token all the time?

Configuring Webhooks | Zscaler

After reading through the docs, I believe you need to use the value

api-token dt0c01.N3BJLL…<rest of your api-token>

Maybe just add it (don’t click on test webhook).

To summarise:

  1. Log Ingest API requires API token, It’s not bearer token (Authorization: Bearer <token>) , but the header is Authorization: api-token <token>
  2. Dynatrace supports OAuth, but not for the log ingest api

Just an update, we have added in the Zscaler UI, in the token field api-token <token> but upon saving is always coming up as  Bearer api-token <token> - We are generating alerts if there is anything being ingested. We also created an OAuth2 client and we see that the permission of ingest are all available, are you sure that OAuth client is not supported to ingest logs using this OAuth Bearer token?

the docs of

The docs that we follwed for OAuth client creation:  OAuth clients — Dynatrace Docs

The scope we will use in the OAuth is as follows: We are hoping the ingest event one for the pipeline at least it works: “scope”:“openpipeline:events.sdlc.custom:ingest openpipeline:events.sdlc:ingest openpipeline:security.events.custom:ingest openpipeline:security.events:ingest openpipeline:events.custom:ingest openpipeline:events:ingest storage:logs:read storage:events:read storage:events:write storage:metrics:read storage:logs:write”,“token_type”:“Bearer”

Julius, I can confirm that after hard work to try to create an OAuth client and a token and giving all the relevant scope to the token, the log ingest is not accepting the Bearer token to inject logs or event in open pipelines, despite adding the following scopes which are available during the OAuth client creation

The scopeopenpipeline:events.sdlc.custom:ingestopenpipeline:events.sdlc:ingestopenpipeline:security.events.custom:ingestopenpipeline:security.events:ingestopenpipeline:events.custom:ingestopenpipeline:events:ingeststorage:logs:readstorage:events:readstorage:events:writestorage:metrics:readThe ingest is always giving {“error”:{“code”:401,“message”:“Access token is invalid.”}}This brings back to think about an alternative solutionThe reason is that Zscaler is always forcing to append Bearer string to its token which means that the Authorisation of Dynatrace is breaking; Dynatrace is expecting Api-Token for this work.If Zscaler do not support the Api-Token string before the token, this will being be to think of an alternative solution.Any fresh thinking on how to proceed now to ingest this simple Zscaler event into Dynatrace.

Yes, you are right, currently

  • Dynatrace needs api-token authorization header for log ingest api (except when sending it locally on a host monitored by OneAgent)
  • ZScaler GUI is not flexbile enough to allow your custom authorization header and enforces bearer string

If you have an option to run a “proxy” between zscaler and Dynatrace, this could be the easiest method - just rewriting authorization headers. If not, I’d probably go with creating a Dynatrace app function which will consume log from zscaler and send it as log event internally. This won’t require any additional infrastructure. 
When called externally app functions can be authenticated using Platform bearer tokens.

Julius, thanks once more, I am glad we are in the same page. My next venture is to try to get a Dynatrace app this will be easier than an external web proxy using lambda function or open telemetry collection. The issue you can get me into speed how I can write quickly his function as the deadline is approaching to deliver this integration. Where will this app be hosted on the SaaS server or one of our Active Gate and I guess it is written python which I favour. You support is much appreciated.

Julius, I would also raise the concern that if OAuth is working in Dynatrace to support Bearer token to ingest logs than the task can be easily resolved by using pipelines. Can we put this issue to the product team to see why they discontinued the support for OAuth for ingest API, the scope are all there and the token is generated but the API does not support. How can escalate resolving this as it seems to be the easiest solution in this integration. Can you help to raise a Product change feature to the product team? on the community or direct via the support line. I have a ticket for this as well.

Julius, on the subject OAuth client, currently it does not support ingest of logs into grail , what about fetching data e.g. api/v1/entity/infrastructure/hosts what scope I need to assign to get a full extract of the hosts, using the OAuth client, thanks for your support.