User session exports to Elasticsearch 8

[ydr]

Hello,

We trying to use "user Session export" with  Elasticsearch 8 referring to  User session export feature:

https://www.dynatrace.com/support/help/shortlink/session-export-http

We got automaticly the 400 error when we try the export test.
After some searches, we found that Elasticsearch 8 no longer use mapping types. So if we send a sample export data like this :

{ "index" : { "_index" : "my-index", "_type" : "_doc", "_id" : "umsaywsjuo-744377345-1622107543233" } }
{"tenantId":"umsaywsjuo","userSessionId":"744377345","startTime":1622107543233,"endTime":1622107578205,"duration":34972,"internalUserId":"744377345","userType":"SYNTHETIC","applicationType":"WEB_APPLICATION","bounce":false,"newUser":true,"userActionCount":12,"totalErrorCount":5,"totalLicenseCreditCount":0,"matchingConversionGoalsCount":0,"..................................................

Elasticsearch reject the data because --> "_type" : "_doc"

If we remove "_type":"_doc" from the data, so the Data are accepted by the server.

All data automaticly sent by Dynatrace include : "_type":"_doc"

So is there any solution to remove "_type":"_doc" ?

Thanks for your help.

[mgome · Accepted solution]

You could send the output through logstash, remove the field and then output to elasticsearch.  

filter {
  mutate {
    remove_field => [ "_type" ]
  }
}

[Candra]

Since I just got the same problem, I'll share an update for the solution that I found
As explained in the doc below, you just need to keep the field "Type of documents in the Elasticsearch index" in the setting empty. Elastic 8 will not reject it anymore
https://docs.dynatrace.com/docs/platform-modules/digital-experience/session-segmentation/export-sess...