topic Re: retrieve detected log sources given a host group in DQL

retrieve detected log sources given a host group

imrj99 — Fri, 01 Aug 2025 06:59:43 GMT

which DQL query lets you retrieve all discovered log sources (ingested or not) for a given host group?

I am able to see these log sources in logsourcestate under dt.entity.process_group_instance, but not able to filter them, it always comes up null...

Re: retrieve detected log sources given a host group

angelalanis — Fri, 01 Aug 2025 17:22:00 GMT

Hi, does this query help you?
You can filter by host group name and you get a list of the unique log sources.

Re: retrieve detected log sources given a host group

imrj99 — Fri, 01 Aug 2025 17:53:13 GMT

unfortunately no....it only retrieves those that there is log data for (ie. already being ingested), but not for all detected log sources, even if not ingested..

using the query below I can find all the detected log sources in "dt.entity.process_group_instance" under "logSourceState", but they each come back as columns, which makes it very hard to filter, not sure why this is

fetch dt.entity.process_group_instance, from:now() - 5m
| fieldsAdd logSourceState = logSourceState
| filter isNotNull(logSourceState)
| limit 200

Re: retrieve detected log sources given a host group

angelalanis — Fri, 01 Aug 2025 18:42:22 GMT

I see what you mean with coming back as columns, I found this workaround, hopefully it helps you.

fetch dt.entity.host | fields id, logSourceState, hostGroupName | filter matchesValue(hostGroupName, "extensions") | filterOut isNull(logSourceState) | fieldsAdd sLogSourceState = toString(logSourceState) | fieldsAdd sLogSourceState = substring(sLogSourceState, from:1, to: stringLength(sLogSourceState)-1) | fieldsAdd sLogSourceState = splitString(sLogSourceState,",") | expand sLogSourceState | fieldsAdd sLogSourceState=trim(sLogSourceState) | parse sLogSourceState, "'\"' LD:file '\":\"storageStatus:' LD:state '\"'" | fields id, file, state

Re: retrieve detected log sources given a host group

imrj99 — Fri, 01 Aug 2025 19:28:00 GMT

it didnt, got no results...... but i think I see where you are going, I changed the 1st line to dt.entity.proceess_group_instance and commented out lines 2,3 and I do get all the log sources in a row format!

I guess from this I need to filter for specific hosts by joining which process group instance runs in which host group? so close!

fetch dt.entity.process_group_instance
//| fields id, logSourceState, hostGroupName
//| filter matchesValue(hostGroupName, "HOSTGROUPNAME")
| filterOut isNull(logSourceState)
| fieldsAdd sLogSourceState = toString(logSourceState)
| fieldsAdd sLogSourceState = substring(sLogSourceState, from:1, to: stringLength(sLogSourceState)-1)
| fieldsAdd sLogSourceState = splitString(sLogSourceState,",")
| expand sLogSourceState
| fieldsAdd sLogSourceState=trim(sLogSourceState)
| parse sLogSourceState, "'\"' LD:file '\":\"storageStatus:' LD:state '\"'"
| fields id, file, state

Re: retrieve detected log sources given a host group

TracingMySanity — Fri, 01 Aug 2025 21:14:59 GMT

Modified your query a bit to get the hostGroup entity attribute using the `belongs_to` relationship from the process_group_instance

fetch dt.entity.process_group_instance | filterOut isNull(logSourceState) | fieldsAdd sLogSourceState = toString(logSourceState) | fieldsAdd sLogSourceState = substring(sLogSourceState, from:1, to: stringLength(sLogSourceState)-1) | fieldsAdd sLogSourceState = splitString(sLogSourceState,",") | expand sLogSourceState | fieldsAdd sLogSourceState=trim(sLogSourceState) | parse sLogSourceState, "'\"' LD:file '\":\"storageStatus:' LD:state '\"'" | fieldsAdd host=belongs_to[dt.entity.host] | fieldsAdd hostGroup=entityAttr(host, type:"dt.entity.host", "hostGroupName") | fields id, file, state, hostGroup

Re: retrieve detected log sources given a host group

imrj99 — Fri, 01 Aug 2025 22:00:38 GMT

awesome! i just need to be able to filter on a specific host group now, i think i could figure that out (maybe!!) lol

Re: retrieve detected log sources given a host group

marusla — Wed, 03 Sep 2025 17:36:19 GMT

Hello, i have the same message no log for process instance, i try the dql but still is empty, which option i have to enable for see process instance logs, thank you