topic Correlate two log sources in DQL

Correlate two log sources

wellpplava — Fri, 02 Feb 2024 13:06:03 GMT

Guys, would this be possible?

When a user registers on our website or app, a log line with their ID is generated. We were able to find out whether it was through the website or app because of the service that generated the log.

If the registration is successful, a new log line is generated with the ID, but we cannot know whether it came from the website or the APP.

Would it be possible to use the ID to correlate the two log lines and count how many registrations were made successfully through the SITE and APP and store it in a metric?

Re: Correlate two log sources

sinisa_zubic — Fri, 02 Feb 2024 13:16:42 GMT

Hi @wellpplava

Yes this is possible. I will try to mock the DQL for that use case

fetch logs | filter log.source == "<source hp" or log.source == "<source2>" // either you extract the id in the processing pipeline or on read // I will assume that you do it in the processing pipeline and have it available in the field id | summarize takeAny(log.source), by: { id }

If you extract the ID on read, then please share an example log line from both log sources.

Best,
Sini

Re: Correlate two log sources

wellpplava — Fri, 02 Feb 2024 14:07:40 GMT

I identify whether it is a website or an app through this filter and this line:

matchesPhrase(content, "preRegisterCustomer - Starting pre-register customer") and matchesValue(dt.entity.cloud_application, "CLOUD_APPLICATION-YYY")

"message": "preRegisterCustomer - Starting pre-register customer ID_NUMBER""

I identify whether it is a website or an app through this filter and this line:

matchesPhrase(content, "setFinalReturnCustomerState") AND matchesValue(dt.entity.cloud_application, "CLOUD_APPLICATION-XXXX")

"message": "setFinalReturnCustomerState - Customer ID_NUMBER is Active",

Can I have this information in a metric? Maybe creating two metrics? Or a single metric with dimensions?

Re: Correlate two log sources

sinisa_zubic — Mon, 05 Feb 2024 13:15:25 GMT

you can do both, capture that info in one metric or a single one with dimensions. for that you don't really need DQL. it is just a configuration of (1) log processing rules and (2) Log metrics extraction.

Go to settings -> Log for configuring processing and log metric extraction

Re: Correlate two log sources

rudy_h — Thu, 22 Feb 2024 01:21:53 GMT

How about Log traces extraction? Can we create span/traces via settings?