cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Is it possible to add multiple fields as sourceField for lookup to work in DQL?

virdavindersing
Participant

Here's the query

fetch spans
| filter dt.system.bucket == "default_spans"
AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
AND NOT (matchesPhrase(endpoint.name, "*liveness*") or matchesPhrase(endpoint.name, "*health*") or matchesPhrase(endpoint.name, "*readiness*"))
AND http.response.status_code>399
// | fieldsAdd is_failed = http.response.status_code>399
| fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)
| fieldsAdd traces = concat("[DT Link](https://{environmentid}.apps.dynatrace.com/ui/apps/dynatrace.distributedtracing/explorer?filter=dt.host_group.id+in+%28",cluster,"%29+AND+k8s.namespace.name+%3D+",k8s.namespace.name,"+AND+k8s.workload.name+%3D+",dt.kubernetes.workload.name,"+AND+endpoint.name+%3D+",endpoint.name,"+AND+http.response.status_code+%3E+399&tf=now-2h%3Bnow)")
| summarize errorcount=count(), by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, http.response.status_code, traces}
| lookup [ fetch spans
| filter dt.system.bucket == "default_spans"
AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
AND NOT in(dt.kubernetes.workload.name, {"collab-datamart-content-proxy"})
AND NOT (matchesPhrase(endpoint.name, "*liveness*") or matchesPhrase(endpoint.name, "*health*") or matchesPhrase(endpoint.name, "*readiness*"))
AND isNotNull(http.response.status_code)
// | fieldsAdd is_failed = http.response.status_code>399
| fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)

| summarize all=count(), by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name}],
source: {cluster: cluster, k8s.namespace.name: k8s.namespace.name, dt.kubernetes.workload.name: dt.kubernetes.workload.name, endpoint.name: endpoint.name},
fields: {all},
joinType: "inner"
| fields cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, http.response.status_code, errorcount, all, traces

5 REPLIES 5

marco_irmer
Champion

Hi there. You may wish to add some additional explanation/context, in addition to the DQL query, to your post in order to help community members understand what you are trying to accomplish and what challenge you are facing.

virdavindersing
Participant

sure I am trying to building a table like below but it should have the "failure rate" column for the endpoint as well

virdavindersing_0-1753378198333.png

 

virdavindersing
Participant

does it help from context perspective ?

virdavindersing
Participant

Basically I need to have the error count based on all the entities (cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, http.response.status_code, errorcount, traces)

At the same time, I need to add a column for failure rate based on errorcount/"all requests count" as well 
I couldn't add the failure rate based on http.response.status_code together with error count

In lookup it is possible with a trick, but in this case it ok just to use join to get this effect. I think this query will give you what you need:

fetch spans
| filter dt.system.bucket == "default_spans"
  AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
  AND NOT (matchesPhrase(endpoint.name, "*liveness*") 
    or matchesPhrase(endpoint.name, "*health*") 
    or matchesPhrase(endpoint.name, "*readiness*"))
  AND http.response.status_code>399
| fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)
| fieldsAdd traces = concat("[DT Link](https://{environmentid}.apps.dynatrace.com/ui/apps/dynatrace.distributedtracing/explorer?filter=dt.host_group.id+in+%28",cluster,"%29+AND+k8s.namespace.name+%3D+",k8s.namespace.name,"+AND+k8s.workload.name+%3D+",dt.kubernetes.workload.name,"+AND+endpoint.name+%3D+",endpoint.name,"+AND+http.response.status_code+%3E+399&tf=now-2h%3Bnow)")
| summarize errorcount=count(), 
  by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, http.response.status_code, traces}

| join 
[ fetch spans
  | filter dt.system.bucket == "default_spans"
  AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
  AND NOT in(dt.kubernetes.workload.name, {"collab-datamart-content-proxy"})
  AND NOT (matchesPhrase(endpoint.name, "*liveness*") 
    or matchesPhrase(endpoint.name, "*health*") 
    or matchesPhrase(endpoint.name, "*readiness*"))
  AND isNotNull(http.response.status_code)
  | fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)
  | summarize all=count(), 
    by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name}
],
on: {cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name}, 
fields: {all}, kind:inner

| fields cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, 
  http.response.status_code, errorcount, 100.0*errorcount/all, traces

 

In this case subquery will always produce record for record in main query, so it is safe to use inner join.

lookup command allows only single fields to be user, but you can construct a record containing all needed fields to be used for comparison. Important thing is that on both sides fields need to be names the same and in be in same order.

Such query should produce same results:

fetch spans
| filter dt.system.bucket == "default_spans"
  AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
  AND NOT (matchesPhrase(endpoint.name, "*liveness*") 
    or matchesPhrase(endpoint.name, "*health*") 
    or matchesPhrase(endpoint.name, "*readiness*"))
  AND http.response.status_code>399
| fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)
| fieldsAdd traces = concat("[DT Link](https://{environmentid}.apps.dynatrace.com/ui/apps/dynatrace.distributedtracing/explorer?filter=dt.host_group.id+in+%28",cluster,"%29+AND+k8s.namespace.name+%3D+",k8s.namespace.name,"+AND+k8s.workload.name+%3D+",dt.kubernetes.workload.name,"+AND+endpoint.name+%3D+",endpoint.name,"+AND+http.response.status_code+%3E+399&tf=now-2h%3Bnow)")
| summarize errorcount=count(), 
  by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, http.response.status_code, traces}
| fieldsAdd key = record(cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name)

| lookup 
[ fetch spans
  | filter dt.system.bucket == "default_spans"
  AND matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_name")
  AND NOT in(dt.kubernetes.workload.name, {"collab-datamart-content-proxy"})
  AND NOT (matchesPhrase(endpoint.name, "*liveness*") 
    or matchesPhrase(endpoint.name, "*health*") 
    or matchesPhrase(endpoint.name, "*readiness*"))
  AND isNotNull(http.response.status_code)
  | fieldsAdd cluster=entityName(dt.entity.kubernetes_cluster)
  | summarize all=count(), 
    by:{cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name}
  | fieldsAdd key = record(cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name)
], sourceField:key, lookupField:key, fields: {all}

| fields cluster, k8s.namespace.name, dt.kubernetes.workload.name, endpoint.name, 
  http.response.status_code, errorcount, 100.0*errorcount/all, traces

 

Featured Posts