How to figure out "Deletion of legacy Kubernetes container metrics in Dynatrace version 1.292"

[Kenny_Gillette]

Team,

Dynatrace release notes this week for 1.286 came out.  They mentioned that "Deletion of legacy Kubernetes container metrics in Dynatrace version 1.292" will happen, which is about 3 months away.  How best review our many tenants for this without manually looking?  We have too many dashboards/metrics to go through everything manually.

Release notes: https://docs.dynatrace.com/docs/whats-new/release-notes/saas/sprint-286

Deletion of legacy Kubernetes container metrics in Dynatrace version 1.292

Infrastructure Observability | Kubernetes

deprecated

In Dynatrace version 1.292, the following metric keys will be deleted, and access to their historical data will no longer be possible. If you are using any of these metrics, please migrate to the What to use instead metrics, which contain the same data and can be used to access the data.

Containers, CPU

Deprecated metric What to use instead

builtin:containers.cpu.throttledMilliCores.legacy Name: [Deprecated] Containers: CPU throttling, mCores Description: CPU throttling per container in millicores. Deprecated metric for historical data access only. Unit: Millicores Aggregations: auto, avg, max, min	builtin:containers.cpu.throttledMilliCores
builtin:containers.cpu.usageMilliCores.legacy Name: [Deprecated] Containers: CPU usage, mCores Description: CPU usage per container in millicores. Deprecated metric for historical data access only. Unit: Millicores Aggregations: auto, avg, max, min	builtin:containers.cpu.usageMilliCores
builtin:containers.cpu.usagePercent.legacy Name: [Deprecated] Containers: CPU usage, % of limit Description: Percent CPU usage per container relative to CPU resource limit. Logical cores are used if CPU limit is not set. Deprecated metric for historical data access only. Unit: Percent (%) Aggregations: auto, avg	builtin:containers.cpu.usagePercent
builtin:containers.cpu.throttlingRatio Name: [Deprecated] Containers: CPU throttling ratio Description: CPU resource limit per container. Unit: Ratio Aggregations: auto, avg, max, min	There is no replacement for the throttling ratio, but it can be calculated on the fly from other metrics.

Memory

Deprecated metric What to use instead

builtin:containers.memory.residentSetBytes.legacy Name: [Deprecated] Containers: Memory usage, bytes Description: Resident set size (Unix) or private working set size (Windows) per container in bytes. Deprecated metric for historical data access only. Unit: Byte Aggregations: auto, avg, max, min	builtin:containers.memory.residentSetBytes
builtin:containers.memory.usagePercent.legacy Name: [Deprecated] Containers: Memory usage, % of limit Description: Resident set size (Unix) or private working set size (Windows) per container in percent relative to container memory limit. If no limit is set, this equals total physical memory. Deprecated metric for historical data access only. Unit: Percent (%) Aggregations: auto, avg	builtin:containers.memory.usagePercent

[rgarzon1]

For the dashboards you can use something like this, i will. 

Spoiler

import requests
import time
import os
import json
#  ###############################################  #
#  #                                             #  #
#  #  Set the following variables to your        #  #
#  #  environment                                #  #
#  #                                             #  #
#  ###############################################  #
# Set the following variables to your environment
tenant = '{tenantId}'
dynatraceTenant = '{0}.live.dynatrace.com'.format(tenant)
dynatraceTokenEncrypt = '{apiToken}'
dynatraceToken = dynatraceTokenEncrypt
#  ###############################################  #
#  #                                             #  #
#  #  Do not modify below this line              #  #
#  #                                             #  #
#  ###############################################  #
epochtime = int(time.time())
namefile = "{0}-dashboards-{1}".format(tenant, epochtime)
rootFolder = os.path.dirname(os.path.abspath(__file__))
outputFile = "{0}\{1}.json".format(rootFolder,namefile)
 
dashboardWithDeprecateKeys = {}

#  ###############################################  #
#  #                   Logic                     #  #
#  ###############################################  #
# get the list of dashboards
url = "https://" + dynatraceTenant + "/api/config/v1/dashboards"
headers = {
    'accept': 'application/json; charset=utf-8',
    'Authorization': 'Api-Token ' + dynatraceToken,
    'Content-Type': 'application/json; charset=utf-8'
}
response = requests.get(url, headers=headers)
response.raise_for_status()
# first item in the list
exitafter = 0
arrayDeprecateKeys = [
                        "builtin:containers.cpu.throttledMilliCores.legacy",
                        "builtin:containers.cpu.usageMilliCores.legacy",
                        "builtin:containers.cpu.usagePercent.legacy",
                        "builtin:containers.cpu.throttlingRatio",
                        "builtin:containers.memory.residentSetBytes.legacy",
                        "builtin:containers.memory.usagePercent.legacy"
                    ]
for dashboard in response.json()["dashboards"]:
    if exitafter == 1:
        break
    url = "https://" + dynatraceTenant + "/api/config/v1/dashboards/" + dashboard["id"]
    response = requests.get(url, headers=headers)
    response.raise_for_status()
    # check if the dashboard has any deprecated keys text
    for key in arrayDeprecateKeys:
        # print("Checking for deprecated key: " + key)
        if key in response.text:
            print("Dashboard URL: " + url)
            dashboardWithDeprecateKeys[dashboard["id"]] = {
                "name": dashboard["name"],
                "url": url
            }
            if "deprecatedKeys" in dashboardWithDeprecateKeys[dashboard["id"]]:
                dashboardWithDeprecateKeys[dashboard["id"]]["deprecatedKeys"].append(key)
            else:
                dashboardWithDeprecateKeys[dashboard["id"]]["deprecatedKeys"] = [key]

        else:
            print("No deprecated keys found:" + key)
    # exitafter = exitafter + 1

with open(outputFile, 'w') as outfile:
    json.dump(dashboardWithDeprecateKeys, outfile, indent=4)
    print("File saved: " + outputFile)
    print("Total dashboards with deprecated keys: " + str(len(dashboardWithDeprecateKeys)))

I know that could be better but for now this will create a Json file with all dashboards and theirs Deprecate Keys

Hope it helps

[florian_g · Accepted solution]

thanks for bringing this up - we will integrate this into this tool: https://metricaudit.services.dynatrace.com/
The tool checks all your dashboards, alerts, and SLOs for known (to be) deprecated metrics and allows you to export the information in csv for further analysis.
I'll come back asap here to tell you when it's available.

[Kenny_Gillette]

Ah thanks.  I forgot about this tool and this would be great for this to be in this tool

[florian_g]

Just got confirmed that these metrics are already integrated into the tool 🙂 
(Also working on getting it mentioned in the release notes) thx again @Kenny_Gillette 

[Kenny_Gillette]

Perfect.  Thanks for response!