Radu
June 28, 2022, 9:39am
1
Use case : add dimension filters to an Extension 2.0 YAML so that unwanted data can be filtered out. This can be implemented in the “dimensions” section of each data source definition (under “group” or “subgroup”). This post will use “snmp ” as the data source, but the implementation is the same across all other data sources .
Generic concept and syntax :
You can add a filter to a dimension in one of two ways:
filter: const:<filter-definition>
const: must be followed by a literal filter definition<filter-definition> follows this syntax:
$eq(<str>) – check if <str> matches what you’re filtering$prefix(…) – must begin with …$suffix(…) - must end with …$contains(…) – must contain …$and(<expr1>, <expr2>) – can be used to chain two or more of the above expressions with AND operator$or(<expr1>, <expr2>) – can be used to chain two or more of the above expressions with OR operator$not(<expr>) – can be used to negate an expression
filter: var:<var-id>
Example 1 - I’m collecting a sensor.value metric (oid 1.0.0.0.0.3) from a table of sensors of my snmp device. The table exposes a dimension called sensor.type (oid 1.0.0.0.0.2) which is an integer value for the type of sensor. I know sensor.type 3 is a temperature sensor, so I only want to collect the value of this type of sensor. I will have to use filters to only select this dimension value.YAML definition:
snmp:<br> - group: metrics<br> interval:<br> minutes: 1<br> subgroups:<br> - subgroup: sensor-data<br> table: true<br> dimensions:<br> - key: sensor.type<br> value: oid:1.0.0.0.0.2<br> filter: const:$eq(3)<br> metrics:<br> - key: sensor.value<br> value: oid:1.0.0.0.0.3
Example 2 - The F5 device has a lot of Traffic Profiles that may not be needed for monitoring. I want users to be able to filter out Traffic Profiles that they don’t want monitored. I must create a variable where users can enter syntax to filter out this data.
YAML definition:
vars:<br> id: profile-name-filter<br> type: pattern<br> displayName: Pattern to filter which Profies are monitored<br><br>snmp:<br> - group: traffic-profiles<br> interval:<br> minutes: 1<br> subgroups:<br> - subgroup: profile-metrics<br> table: true<br> dimensions:<br> - key: profile.name<br> value: oid:2.0.0.0.0.1<br> filter: var:profile-name-filter<br> metrics:<br> - key: some.profile.metric<br> value: oid:2.0.0.0.0.8
Note: the variable field in the Monitoring configuration will take the same syntax as the literal filter definition explained earlier. The users see this:
8 Likes
tarjei
June 28, 2022, 11:24am
2
So this filtering should work?
(It doesnt)
- <span>subgroup</span>: ResourceGroup<br> <span>wmiNamespace</span>: <span>"root\\mscluster"<br></span> <span>query</span>: SELECT State,Name FROM MSCluster_ResourceGroup WHERE Name!='Available Storage'<br> <span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresourcegroup.state<br> <span>value</span>: column:State<br> <span>dimensions</span>:<br> - <span>key</span>: resourcegroup.name<br> <span>value</span>: Name<br> <span>filter</span>: const:$contains("Clu")
Radu
June 28, 2022, 11:27am
3
Please remove the quotes around Clu.
Radu
June 28, 2022, 11:31am
4
And also remember that value extraction in WMI always happens from Columns.
You correct dimension definition is:
- key: resourcegroup.name<br> value: column:Name<br> filter: const:$contains(Clu)
tarjei
June 28, 2022, 11:50am
5
It doesn’t work.
Should I open a support case then?
tarjei
June 28, 2022, 11:51am
6
This is the entire yaml:
<span>name</span>: <span>"custom:com.kanari.extension.wmi-cl"<br></span><span>version</span>: <span>"1.0.48"<br></span><span>minDynatraceVersion</span>: <span>"1.225"<br></span><span>author</span>:<br> <span>name</span>: <span>"Kanari AS"<br></span><span><br></span><span>vars</span>:<br> - <span>id</span>: resourceGroupFilter<br> <span>displayName</span>: Resource Group Filter<br> <span>type</span>: pattern<br> - <span>id</span>: resourceFilter<br> <span>displayName</span>: Resource Filter<br> <span>type</span>: pattern<br> - <span>id</span>: nodeFilter<br> <span>displayName</span>: Node Filter<br> <span>type</span>: pattern<br> - <span>id</span>: networkinterfaceFilter<br> <span>displayName</span>: Network Interface Filter<br> <span>type</span>: pattern<br><br><span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresourcegroup.state<br> <span>metadata</span>:<br> <span>displayName</span>: Cluster ResourceGroup State<br> <span>description</span>: State of the cluster resource group<br> <span>unit</span>: Count<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresource.state<br> <span>metadata</span>:<br> <span>displayName</span>: Cluster Resource State<br> <span>description</span>: State of the cluster resource<br> <span>unit</span>: Count<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusternode.state<br> <span>metadata</span>:<br> <span>displayName</span>: Cluster Node State<br> <span>description</span>: State of the cluster node<br> <span>unit</span>: Count<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusternetworkinterface.state<br> <span>metadata</span>:<br> <span>displayName</span>: Cluster Network Interface State<br> <span>description</span>: State of the cluster network interface<br> <span>unit</span>: Count<br><br><br><br><span>wmi</span>:<br> - <span>group</span>: Host<br> <span>interval</span>: 1m<br> <span>subgroups</span>:<br> - <span>subgroup</span>: Resource<br> <span>wmiNamespace</span>: <span>"root\\mscluster"<br></span> <span>query</span>: SELECT State,Name FROM MSCluster_Resource<br> <span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresource.state<br> <span>value</span>: column:State<br> <span>dimensions</span>:<br> - <span>key</span>: resource.name<br> <span>value</span>: column:Name<br><br> - <span>subgroup</span>: Node<br> <span>wmiNamespace</span>: <span>"root\\mscluster"<br></span> <span>query</span>: SELECT State,Name FROM MSCluster_Node<br> <span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusternode.state<br> <span>value</span>: column:State<br> <span>dimensions</span>:<br> - <span>key</span>: node.name<br> <span>value</span>: column:Name<br><br> - <span>subgroup</span>: ResourceGroup<br> <span>wmiNamespace</span>: <span>"root\\mscluster"<br></span> <span>query</span>: SELECT State,Name FROM MSCluster_ResourceGroup WHERE Name!='Available Storage'<br> <span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresourcegroup.state<br> <span>value</span>: column:State<br> <span>dimensions</span>:<br> - <span>key</span>: resourcegroup.name<br> <span>value</span>: column:Name<br> <span>filter</span>: var:resourceGroupFilter<br> - <span>subgroup</span>: Node<br> <span>wmiNamespace</span>: <span>"root\\mscluster"<br></span> <span>query</span>: SELECT State,Name FROM MSCluster_NetworkInterface<br> <span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusternetworkinterface.state<br> <span>value</span>: column:State<br> <span>dimensions</span>:<br> - <span>key</span>: networkinterface.name<br> <span>value</span>: column:Name<br><br><span>#vars:<br></span><span># - id: resourceGroupFilter<br></span><span># displayName: Pattern matching Resource Groups which should be ignored<br></span><span># type: pattern<br></span><span><br></span><span>dashboards</span>:<br> - <span>path</span>: <span>"dashboards/windows_failover_clustering.json"<br></span><span><br></span><span>alerts</span>:<br> - <span>path</span>: <span>"alerts/node_high.json"<br></span> - <span>path</span>: <span>"alerts/node_low.json"<br></span> - <span>path</span>: <span>"alerts/cluster_resource_high.json"<br></span> - <span>path</span>: <span>"alerts/cluster_resource_low.json"<br></span> - <span>path</span>: <span>"alerts/cluster_resourcegroup_high.json"<br></span> - <span>path</span>: <span>"alerts/cluster_resourcegroup_low.json"<br></span> - <span>path</span>: <span>"alerts/nic_low.json"<br></span><span><br></span><span>screens</span>:<br> - <span>entityType</span>: wmi:com_kanari_extension_wmi_cluster_instance<br> <span>detailsSettings</span>:<br> <span>staticContent</span>:<br> <span>showProblems</span>: true<br> <span>showProperties</span>: true<br> <span>showTags</span>: true<br> <span>showGlobalFilter</span>: true<br> <span>showAddTag</span>: true<br> <span>layout</span>:<br> <span>autoGenerate</span>: false<br> <span>cards</span>:<br> - <span>type</span>: <span>"CHART_GROUP"<br></span> <span>key</span>: <span>"host-clusterresources"<br></span> <span>propertiesCard</span>:<br> <span>properties</span>:<br> - <span>type</span>: RELATION<br> <span>relation</span>:<br> <span>entitySelectorTemplate</span>: type(host), toRelationships.runsOn($(entityConditions))<br> <span>displayName</span>: Host<br> <span>chartsCards</span>:<br> - <span>key</span>: <span>"host-clusterresources"<br></span> <span>displayName</span>: <span>"Cluster Resources"<br></span> <span>numberOfVisibleCharts</span>: 3<br> <span>charts</span>:<br> - <span>displayName</span>: <span>"Cluster ResourceGroup State"<br></span> <span>metrics</span>:<br> - <span>metricSelector</span>: <span>"com.kanari.extension.wmi-cl.host.clusterresourcegroup.state:SplitBy(\"resourcegroup.name\")"<br></span> - <span>displayName</span>: <span>"Cluster Resource State"<br></span> <span>metrics</span>:<br> - <span>metricSelector</span>: <span>"com.kanari.extension.wmi-cl.host.clusterresource.state:SplitBy(\"resource.name\")"<br></span> - <span>displayName</span>: <span>"Cluster Network Interface State"<br></span> <span>metrics</span>:<br> - <span>metricSelector</span>: <span>"com.kanari.extension.wmi-cl.host.clusternetworkinterface.state:SplitBy(\"networkinterface.name\")"<br></span><span># entitiesListCards:<br></span><span># - key: "cluster-nic-list"<br></span><span># displayName: "Network Interfaces List"<br></span><span># entitySelectorTemplate: "type(wmi:com_kanari_extension_wmi_cluster_network_interface),fromRelationships.runsOn($(entityConditions))"<br></span><span># displayCharts: false<br></span><span># displayIcons: true<br></span><span># enableDetailsExpandability: tru<br></span><span>topology</span>:<br> <span>types</span>:<br> - <span>name</span>: wmi:com_kanari_extension_wmi_cluster_instance<br> <span>displayName</span>: Cluster Instance<br> <span>enabled</span>: true<br> <span>rules</span>:<br> - <span>idPattern</span>: wmi_cluster_instance_{dt.entity.host}<br> <span>sources</span>:<br> - <span>sourceType</span>: Metrics<br> <span>condition</span>: $prefix(com.kanari.extension.wmi-cl.host)<br> <span>attributes</span>: []<br> <span>requiredDimensions</span>: []<br> <span>instanceNamePattern</span>: Cluster Instance on {dt.entity.host}<br> - <span>idPattern</span>: wmi_cluster_instance_{host}<br> <span>sources</span>:<br> - <span>sourceType</span>: Metrics<br> <span>condition</span>: $prefix(com.kanari.extension.wmi-cl.host)<br> <span>attributes</span>: [ ]<br> <span>requiredDimensions</span>: [ ]<br> <span>instanceNamePattern</span>: Cluster Instance on {host}<br> - <span>name</span>: wmi:com_kanari_extension_wmi_cluster_network_interface<br> <span>displayName</span>: Cluster Network Interface<br> <span>enabled</span>: true<br> <span>rules</span>:<br> - <span>idPattern</span>: wmi_cluster_network_interface_{dt.entity.host}<br> <span>sources</span>:<br> - <span>sourceType</span>: Metrics<br> <span>condition</span>: $prefix(com.kanari.extension.wmi-cl.host.clusternetworkinterface)<br> <span>attributes</span>: [ ]<br> <span>requiredDimensions</span>: [ ]<br> <span>instanceNamePattern</span>: Cluster Network Interface on {dt.entity.host}
Radu
July 1, 2022, 8:16am
7
Hi @tarjei
After some discussions I found out that for WMI, due to performance reasons, the filtering within the dimensions definition is disabled. Instead, filtering should be applied directly within the query. You can still use variables to expose the filter as a configuration parameter.
This means that for WMI, the filtering takes WQL syntax and not the syntax I documented eariler.
The example from your extension would be:
<span>query</span>: SELECT State,Name FROM MSCluster_ResourceGroup WHERE Name!='Available Storage' var:resourceGroupFilter<br><span>metrics</span>:<br> - <span>key</span>: com.kanari.extension.wmi-cl.host.clusterresourcegroup.state<br> <span>value</span>: column:State<br><span>dimensions</span>:<br> - <span>key</span>: resourcegroup.name<br> <span>value</span>: column:Name
Where resourceGroupFilter is a variable defined as before. Within the monitoring configuration you will have to add the additional syntax. For example, “Name starts with Clu” would be “AND Name LIKE ‘Clu%’”
Could you try this approach and let me know if it fixes your issue?
Ok,
Thank you!
Would you happen to know when the documentation for ext 2.0 will get the desired TLC?
tarjei
July 1, 2022, 11:20am
9
I tried it, but it did not work.
Radu
July 1, 2022, 11:34am
10
Could you try one final variation please. Change the query to
<span>query</span>: SELECT State,Name FROM MSCluster_ResourceGroup WHERE var:resourceGroupFilter
And add the whole Name filter in the var field of the UI.
tarjei
July 1, 2022, 12:12pm
11
Hi!
I have now tried multiple variants of this, but it doesnt work.
tarjei
February 7, 2023, 2:11pm
12
Hi @Radu this does not work. Is this because I have to use a specific schema version?
1 Like
tarjei
February 7, 2023, 2:19pm
13
Hi!
I see that in Dynatrace IIS Extension, there is dimension filtering. That plugin is using 1.245 schema.
Microsoft IIS | Dynatrace Hub
Radu
February 7, 2023, 2:38pm
14
Please open a support ticket so we can troubleshoot this. The query looks correct but there may be other things happening that we cannot tell from the YAML alone.
tarjei
February 7, 2023, 4:10pm
15
Hi!
I have an ongoing discussion for this, will make them open a support case for it. Thank you!
jegron
December 5, 2023, 10:45am
16
Hi @Radu !
I believe there might not be any documentation available on this subject currently. Would it be possible to reach out to the documentation team for further information on this?
Additionally, I’m looking to filter only interfaces specifically with the prefix ‘#’ :
vars: - id: if_alias type: text displayName: Pattern to filter capturing of Interfaces by Interface Alias
dimensions: - key: if.alias value: oid:1.3.6.1.2.1.31.1.1.1.18 filter: var:if_alias
I tried this syntax :
It works but I also obtain all the interfaces which do not have the if alias filled in. How can I exclude them?
I will try this. But you do understand my frustration with this?