topic Re: Inject Custom Metadata into host record, without oneagentctl? in Extensions

Inject Custom Metadata into host record, without oneagentctl

ewoelkerling — Tue, 07 Feb 2023 12:04:15 GMT

Hi All,

We are monitoring an environment with a number of legacy windows servers, and are picking up details for these using the Remote Windows Host Monitoring extension via WMI. Whilst this part works fine, in order to facilitate a number of downstream integrations we need to augment the data for the created host records with additional information (eg. Serial Number).

For hosts that have oneagents installed, this information is being provided on the hosts themselves and is appearing in the customHostMetadata are for each of the host entries, however this is obviously not viable for hosts that do not have oneagent installed.

Our intention was to implement this augmentation using an ActiveGate extension to detect these 'remote' hosts, then query WMI for the missing details and augment in Dyna, however I have searched high and low on the API and through the SDK, but unfortunately don't seem to be able to find the appropriate methods/examples to do this either way.

Is anyone able to offer some guidance please, or even better example code of how this can be done in this context?

All help would be greatly appreciated! Thanks!

Re: Inject Custom Metadata into host record, without oneagentctl?

Julius_Loman — Mon, 30 Jan 2023 07:36:25 GMT

Since you are using the Extension 2.0, I believe the only method is to extend the existing topology model for the extension. You can do that in the Settings > Generic topology model for the wmi:com_dynatrace_extension_host-observability_instance type.

You need to add another source of data (I'd choose either events or logs, depending on which fits you best) and create appropriate attribute extraction rule. You need to send the data regularly to keep the properties "alive". So sending an event with the device name and the serial number (and other properties you also need).

However, any future updates of this extension from the HUB will overwrite your topology modifications.

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Mon, 30 Jan 2023 07:51:07 GMT

HI Julius - thanks for the info, I'll give it a try. FYI - we're not necessarily tied to Extension 2.0 (apart from the Remote Windows part) - the active gate extension can use V1 if there's appropriate methods/capability - does that change your thoughts?

Re: Inject Custom Metadata into host record, without oneagentctl?

Julius_Loman — Mon, 30 Jan 2023 08:35:29 GMT

I'd stick with Extension 2.0 - as it definitely provides mode value along with the universal analysis screens. With Extension 1.0 it was possible to add metadata by using SmartScape API and updating the custom device,, but definitely I'd recommend to stay on Extension 2.0 version you have now.

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Mon, 30 Jan 2023 21:02:06 GMT

Thanks Julius - very much appreciate the insight & assistance - thanks again!

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Wed, 01 Feb 2023 06:39:37 GMT

HI Julius - given your excellent answer before, I'd like to pose a follow on question (I didn't make it a new forum question as it relies on the context above)..

I've tested your suggestion and it works really well and has no problems, however I've run into a downstream issue..

In order to feed in the appropriate information (SerialNumber in this case) I setup the appropriate rules in the custom topology, then quickly created a V2 WMI extension to retrieve the serial via WMI. Whilst the WMI extension is reporting as running OK, no metrics are coming through. When I investigate the ActiveGate on which it's running, it's throwing a MINT error (7336][err][2023-02-01 06:33:01.965Z] [ds:wmi] [error] [EEC_CONNECTION_ERROR] HTTP error, dropping MINT packet (1 lines, 255 bytes): [HTTP Status Code = 400 - Bad Request (http://127.0.0.1:14599/mint/4962476800840004540)] EEC REST API call failed [status code=30]).

Investigating further, I believe that it's probably due to the WMI query (SELECT * FROM Win32_BIOS) as there's nothing in this that can return a 'metric' (they're all text fields - see below for example output).

Ultimately, the metric line that works for manual ingestion is like "com.dynatrace.extension.host-observability.info,,host.serialnumber=xyzqr,device.name=ip-10-0-0-214.ec2.internal 100" - which I believe works as it's a proper metric send.. whereas the V2 extension WMI I've defined is:

wmi:
- group: augments
interval:
minutes: 1
dimensions:
- key: host
value: this:device.host
subgroups:
- subgroup: BIOS
query: SELECT * FROM Win32_BIOS
metrics:
- key: com.dynatrace.extension.host-observability.info
value: column:Version
dimensions:
- key: host.serialnumber
value: column:SerialNumber

WMI Output:

SMBIOSBIOSVersion :
Manufacturer :
Name : T95 Ver. 01.11.00
SerialNumber : CND2303LMY
Version : HPQOEM - 0

Obviously a mismatch.. any suggestions on a workaround that might allow me to turn this into a proper metric send, but still have the dimension I need for augmentation?

Re: Inject Custom Metadata into host record, without oneagentctl?

Julius_Loman — Wed, 01 Feb 2023 07:39:39 GMT

As you have chosen to use metric as the data source, then it must definitely contain a number, your Version output is not a number.
So your WMI query must return (any) numeric number. Any chance you can do some join with another WMI table and just return some random number? It can be any random number.

Re: Inject Custom Metadata into host record, without oneagentctl?

Mike_L — Wed, 01 Feb 2023 07:44:30 GMT

You can’t join tables in WQL, so that won’t work alas. There is an easier solution though which is to use a constant for the value of the metric. You’d do that by setting the value as const:1 for example.

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Wed, 01 Feb 2023 21:08:25 GMT

Thanks guys - although I should have mentioned, I tried that and ran into what looks like a schema constraint problem and/or a mint-metric issue: (apologies for the lack of detail on the previous)

For example: If I run the following plugin configuration:

wmi: - group: augments interval: minutes: 1 dimensions: - key: host value: this:device.host subgroups: - subgroup: BIOS query: SELECT * FROM Win32_BIOS metrics: - key: com.dynatrace.extension.host-observability.info value: 100 dimensions: - key: host.serialnumber value: column:SerialNumber

I get an error on the plugin upload step:

raise dtcliutils.ExtensionValidationError(response.text) dtcli.utils.ExtensionValidationError: {"error":{"code":400,"message":"Extension validation error","constraintViolations":[{"path":"$.wmi[0].subgroups[0].metrics[0].value","message":"$.wmi[0].subgroups[0].metrics[0].value: integer found, string expected","parameterLocation":"PAYLOAD_BODY","location":null}]}}

If I alter it so that the metric is quoted, but still an integer like below:

wmi: - group: augments interval: minutes: 1 dimensions: - key: host value: this:device.host subgroups: - subgroup: BIOS query: SELECT * FROM Win32_BIOS metrics: - key: com.dynatrace.extension.host-observability.info value: "100" dimensions: - key: host.serialnumber value: column:SerialNumber

It uploads OK, but throws a different error when trying to run on the ActiveGate

Failed to assign monitoring configuration to ActiveGate. Reason: GENERIC_ERROR:Status GENERIC_ERROR (27) returned from 1 agents (10.0.179.98: Cannot gather mint metrics from query BIOS: Fault retrieving metric com.dynatrace.extension.host-observability.info: Unknown name. [status code=27] [status code=27]);

which leaves me stuck..

Any other thoughts on how to get around this one? Personally I'm not a huge fan of the schema constraint, although I can understand possible thoughts behind it..

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Wed, 01 Feb 2023 21:21:34 GMT

Apologies, read Mike's post more carefully and added the const:.. but unfortunately I'm still throwing an error..

So running the following configuration the plugin now uploads:

wmi: - group: augments interval: minutes: 1 dimensions: - key: host value: this:device.host subgroups: - subgroup: BIOS query: SELECT * FROM Win32_BIOS metrics: - key: com.dynatrace.extension.host-observability.info value: const:100 dimensions: - key: host.serialnumber value: column:SerialNumber

but throws the following:

Any ideas? Metric definition perhaps?

Re: Inject Custom Metadata into host record, without oneagentctl?

Mike_L — Wed, 01 Feb 2023 22:45:28 GMT

It might be that the WMI datasource forgot to implement the const function. Can you open a support ticket?

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Wed, 01 Feb 2023 23:01:31 GMT

Can do and have.. case #130044 created. Will update this forum as things progress. Thanks!

Re: Inject Custom Metadata into host record, without oneagentctl?

Mike_L — Thu, 02 Feb 2023 11:07:37 GMT

Thanks, I name dropped some R&D people in that support ticket who might be able to help out.

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Thu, 02 Feb 2023 20:43:36 GMT

Thanks Mike - much appreciated. Please take an extra donut out of petty cash 😁

Re: Inject Custom Metadata into host record, without oneagentctl?

ewoelkerling — Mon, 06 Feb 2023 21:55:08 GMT

FYI - For anyone reading this - the CONST: modifier isn't necessary for the BIOS Serial retrieval - there are a number of hidden metrics available in the WMI branch for Win32_BIOS that can be used.

I have addressed this issue, and am now adding the serial_number as a dimension using the following query & configuration:

- subgroup: bios wmiNamespace: root\cimv2 query: SELECT SerialNumber, SMBiosMajorVersion FROM Win32_BIOS dimensions: - key: serial_number value: column:SerialNumber metrics: - key: com.dynatrace.extension.host-observability.info.bios.biosversion value: column:SMBiosMajorVersion type: gauge featureSet: standard

Would still be nice to get the const: element sorted, but at least there's a workaround 😀

Re: Inject Custom Metadata into host record, without oneagentctl?

Ana_Kuzmenchuk — Tue, 07 Feb 2023 11:46:22 GMT

Hi @ewoelkerling thank you very much for sharing a workaround with the Community! Much appreciated 😊 Happy you were able to find a solution!