topic User session exports to Elasticsearch 8 in Real User Monitoring

User session exports to Elasticsearch 8

ydr — Tue, 22 Feb 2022 08:28:53 GMT

Hello,

We trying to use "user Session export" with Elasticsearch 8 referring to User session export feature:

https://www.dynatrace.com/support/help/shortlink/session-export-http

We got automaticly the 400 error when we try the export test.
After some searches, we found that Elasticsearch 8 no longer use mapping types. So if we send a sample export data like this :

{ "index" : { "_index" : "my-index", "_type" : "_doc", "_id" : "umsaywsjuo-744377345-1622107543233" } }
{"tenantId":"umsaywsjuo","userSessionId":"744377345","startTime":1622107543233,"endTime":1622107578205,"duration":34972,"internalUserId":"744377345","userType":"SYNTHETIC","applicationType":"WEB_APPLICATION","bounce":false,"newUser":true,"userActionCount":12,"totalErrorCount":5,"totalLicenseCreditCount":0,"matchingConversionGoalsCount":0,"..................................................

Elasticsearch reject the data because --> "_type" : "_doc"

If we remove "_type":"_doc" from the data, so the Data are accepted by the server.

All data automaticly sent by Dynatrace include : "_type":"_doc"

So is there any solution to remove "_type":"_doc" ?

Thanks for your help.

Re: User session exports to Elasticsearch 8

mgome — Mon, 21 Feb 2022 18:17:04 GMT

You could send the output through logstash, remove the field and then output to elasticsearch.

filter {
mutate {
remove_field => [ "_type" ]
}
}

Re: User session exports to Elasticsearch 8

Candra — Mon, 11 Dec 2023 17:23:42 GMT

Since I just got the same problem, I'll share an update for the solution that I found
As explained in the doc below, you just need to keep the field "Type of documents in the Elasticsearch index" in the setting empty. Elastic 8 will not reject it anymore
https://docs.dynatrace.com/docs/platform-modules/digital-experience/session-segmentation/export-session-data