OneAgent Plugin for browser process metrics

[rseibert1]

I am trying to make my first python extension. I am having trouble with the below plugin.py (keeps getting KeyError: 'results_builder'. I have not had much luck finding examples of what I want to do. Not really finding any doc out there on the ruxit.api  I have easily built a similar script that uses the metrics API to ingest but trying to convert that to Oneagent Extension has been challenging. You can see I have been trying several iterations of invoking the self.results_builder*.

import psutil

from ruxit.api.base_plugin import BasePlugin

from ruxit.api.data import PluginMeasurement

import logging

import os

# Set up logging for Windows

log_file_path = os.path.join(os.getenv('PROGRAMDATA'), 'dynatrace', 'custom_browser_metrics.log')

logging.basicConfig(level=logging.DEBUG, filename=log_file_path, filemode='w')

logger = logging.getLogger()

class PythonPluginHost(BasePlugin😞

    def initialize(self, **kwargs😞

        logger.debug("Plugin initialized")

    def query(self, **kwargs😞

        #pgi = self.find_single_process_group(pgi_name('plugin_sdk.demo_app'))

        #pgi_id = pgi.group_instance_id

        #stats_url = "http://localhost:8769"

        #stats = requests.get(stats_url, timeout=10).json()

        results_builder = self.results_builder

        if not results_builder:

            logger.error("results_builder not found in kwargs")

            return

        browsers = ["chrome", "msedge"]

        for browser in browsers:

            metrics = get_process_metrics(browser)

            if metrics:

                cpu = metrics["CPU"]

                memory = metrics["Memory"]

                logger.debug(f"Sending metrics for {browser}: CPU={cpu}, Memory={memory}")

                #self.results_builder.absolute(key="custom.browser.cpu", value=cpu, dimensions={"browser": browser})

                self.results_builder.add_absolute_result(PluginMeasurement(key="custom.browser.cpu", value=cpu))

                #self.results_builder.absolute(key="custom.browser.{browser}.cpu", value=metrics["CPU"])

                #self.results_builder.absolute(key="custom.browser.memory_percent", value=memory, dimensions={"browser": browser})

                self.results_builder.add_absolute_result(PluginMeasurement(key="custom.browser.memory_percent", value=memory))

                #self.results_builder.absolute(key="custom.browser.{browser}.memory",value=metrics["Memory"])

            else:

                logger.debug(f"No metrics found for {browser}")

def get_process_metrics(process_name😞

    total_memory = psutil.virtual_memory().total # Total memory in bytes

    cpu = 0

    memory = 0

    for proc in psutil.process_iter(['name', 'cpu_percent', 'memory_info']):

        if proc.info['name'] and process_name.lower() in proc.info['name'].lower():

            cpu += proc.cpu_percent(interval=1) # Allow psutil to measure CPU usage over a short interval

            memory += proc.info['memory_info'].rss # Resident Set Size in bytes memory_percent = (memory / total_memory) * 100

    memory_percent = (memory / total_memory) * 100  # Convert bytes to percentage

    logger.debug(f"Process: {process_name}, CPU: {cpu}, Memory: {memory_percent}")

    return {"CPU": round(cpu, 1), "Memory": round(memory_percent, 1)} if cpu or memory else None

# If running as a script, instantiate and query

if __name__ == "__main__":

    plugin = PythonPluginHost()

    plugin.initialize()

    plugin.query()

The plugin.json file for this:

{

    "name": "custom.python.browser_metrics",

    "version": "1.0.0",

    "metricGroup": "custom_browser.metrics",

    "technologies": ["PYTHON"],

    "type": "python",

    "entity": "HOST",

    "metrics": [

      {

        "timeseries": {

        "key": "custom.browser.cpu",

        "displayname": "Browser CPU Usage",

        "unit": "Percent"

        }

      },

      {

        "timeseries": {

        "key": "custom.browser.memory_percent",

        "displayname": "Browser Memory Usage",

        "unit": "Percent"

        }

      }

    ],

    "interval": 60

  }

[Julius_Loman]

@rseibert1 First of all - and that's most important, your example uses Extensions 1.0 which will be removed from Dynatrace this October. 
Secondly, you are fetching regular process metrics (CPU/memory) which are available out of the box with OneAgent, also in Infrastructure mode. You might want to setup Declarative process grouping if your processes are "glued" together. If those metrics do not fit your use case or you want to have another data and this was just an example, have a look at Extension 2.0 , especially at WMI data source or at Python Data source.
For Python, I'd recommend starting with this observability clinic - A Practical Guide to Building Python Based Extensions with Dynatrace.

[rseibert1]

Thanks Julius for pointing me down the 2.0 path, I had looked into that earlier but for some reason I thought I could just convert to 2.0 later.

I have built a python extension in 2.0 format and it runs fine with the simulator, I am developing on a system that also has oneagent installed as well as the EEC enabled. When I upload and try to add a configuration for a specific host, I am lost on what to do for an endpoint as I want this to run locally so there is no endpoint/user/passwd configuration. Would I remove remote part of the activation from extension.yaml? Currently it has:

 activation:

    remote:

      path: activationSchema.json

    local:

      path: activationSchema.json

which came from default extension creation.

[Julius_Loman]

You need to have activation in the extension. If this is a OneAgent extension only, you can remove the remote part - this is valid only for extensions executed by the ActiveGate. The extension must have some activationSchema which includes the extension configuration - you can remove all the configuration options in your activationSchema.json (when you generate the project, it generates something for you). 

[Mike_L]

It indeed doesn't support an empty schema. Whenever we need that we just add one dropdown for logging to make it easier to change the logging of the extension if you run into issues.

[rseibert1]

I think if I am following you I can modify the activationSchema such that it will not require configuration after uploading?
In attempting to remove the config options I am getting the below error:
dtcli.utils.ExtensionValidationError: {"error":{"code":400,"message":"Activation schema error: Schema validation failed with Exception: 'No properties specified for complex type dynatrace.datasource.python:python_browser_metrics-localhost' in Schema 'template'."}}

The activationSchema.json is attached let me know what is missing?

[Julius_Loman]

TBH I'm not sure if the schema allows such an empty configuration. Unfortunately, there is no public doc for the activation schema. Maybe @Mike_L or @david_lopes can help here.

[duey_hoang · Accepted solution]

i normally leave the default alone, just add your own configuration after the default in activationschema. If doing local oneagent extension, add your logic in _main_.py. After deployed, they are on the configuration page, and you can enter garbage there as endpoint, and it just does not do anything.