Handle Custom Monitored Entities via Metrics APIv2

[36Krazyfists]

So, I want to send some metrics from one of our other monitoring tools about Network internet facing Interfaces that live on some switches in our datacenters using the Dynatrace Ingest Metrics API v2. 

Each interface entity would have some properties like Name, ID, IP Address, Transmit/Receive bandwidth, and maybe even some custom properties like ISP Name and Circuit ID.  Each interface lives on a network switch that I would also want to create as new entities in Dynatrace with properties like Name, Model Number, and IP Address and custom properties like Location and Device Role (i.e. Core Switch, Access Switch, DataCenter Switch, etc...).

From what I've pieced together via the Metrics Ingestion Protocol, the Monitored Entities API, and the Topology Model documentation, it seems like I have to send all those property values for the entities as dimensions with each metric payload and then you have rules in Settings > Topology model > Generic types that parse those dimensions out into Entity ID, Entity Name, and Entity Attribute fields. 

If that's correct, that seems weird and unnecessary to me.  It seems more logical to have my script that is sending the metrics to the API to check and see if the entities I'm sending metrics about already exist and, if they don't, to send all of the above properties and values as dimensions to have the new entities created and, if the entities do already exist, just send the Interface Name (or some unique ID value) and metric for this timestamp.  Do we really have to send all of those static properties with each and every metric payload?  Those properties will pretty much never change once set, and the documentation talks about how metrics and entities are two separate things, but entity dimensions are shown as always being sent with the metrics payloads.  It just doesn't make sense.

Also, apparently Entities in Dynatrace can have Metadata.  Is that the same thing as the attribute values set by the Topology model rules?  If not, how do I set the metadata for an entity created by custom metrics created via the metrics v2 ingest API?

[mark_bley]

1. No need to declare a topology, you can just send metric lines with dimensions and use the built-in visualization tools or alerting options to create tickets.

2. Topology helps to visualize data, organize it in logical groups in Dynatrace.

Define relationships as well, create Unified analysis screens and display data as needed.

ping,hostname=some.device,ipaddresses=10.0.0.1,environment=test 1

[{
    "schemaId": "builtin:monitoredentities.generic.type",
    "value": {
      "enabled": true,
      "name": "device:ping",
      "displayName": "Ping Device",
      "createdBy": "mark",
      "rules": [{
          "idPattern": "{hostname}",
          "instanceNamePattern": "{hostname}",
          "iconPattern": null,
          "sources": [{
              "sourceType": "Metrics",
              "condition": "$eq(ping)"
            }
          ],
          "requiredDimensions": [],
          "attributes": [{
              "key": "environment",
              "displayName": "Environment",
              "pattern": "{environment}"
            }, {
              "key": "ipaddresses",
              "displayName": "IP_addresses",
              "pattern": "{ipaddresses}"
            }
          ],
          "role": null
        }
      ]
    }
  }
]

 If you do not send dimensions, Dynatrace won't be able to attach the datapoint to any entity, it will be as the 1. case.

The couple of extra dimensions you send are to keep the entities up to date, worth a couple of bytes, your network should be able to handle this.

As you see from above the attributes is the metadata you ask for, displayed on every entity individually and extracted, from the metric line.

[36Krazyfists]

Wow, so that really is the way it works?  You send all the same static properties with every payload, every minute?  What if you're sending several dozen metrics about an entity every minute?   Sure, of course the network can handle it, it just seems so... inefficient and unnecessary.  The properties of entities rarely if ever change, so sending those values with every metric just seems really weird to me.... But ok.  I guess it makes writing the script easier, so that's good news.

Thanks for your reply.

[36Krazyfists]

Oh, and I'm not talking about just sending a couple extra dimensions. 

For example, for a network interface, I have many properties:

Name, ID, description, ip address, interface type (like internet, uplink, server, etc...), receive bandwidth, transmit bandwidth, ISP Phone number (if it's an internet interface), and probably others I am not thinking of. 

I need to send all that with every metric I send?  I'm gonna be sending at least 8 or 10 metrics every minute per interface, so I've got to send all those properties and values 8 or 10 times every minute (times the number of interfaces I am sending)?  Is that really how it works?   

Sorry to keep asking, but I just feel like I'm not understanding something because that seems so wrong to me...

Why can't I just send all that the first time and then, from that point on, just send the interface id and the metrics?  None of those other properties are going to ever change and, if they do, I can send the ones that do change another time to update them.

[mark_bley · Accepted solution]

There is a way, but it will have effects downstream once you want to filter by a specific dimension, I really would not recommend this.

Also please keep in mind, Dynatrace is not supposed to be your CMDB, if an entity stops receiving data it will just slowly fade out of existence (it is not used important anymore), so this is one of the reasons you will need to keep the entity alive.

You can send all the data at once, e.g.

ping00,hostname=some.device 1
ping01,hostname=some.device 1
ping02,hostname=some.device 1
ping03,hostname=some.device 1
ping04,hostname=some.device 1
ping,hostname=some.device,ipaddresses=10.0.0.1,environment=test 1

The entity will receive all metrics, and the dimensions will also be shown.

But again if you would like to create an alert based on a metric missing this dimension it will not work.

Below metric with dimensions and one of the metrics without them