topic Re: OneAgent Plugin for browser process metrics in Extensions

OneAgent Plugin for browser process metrics

rseibert1 — Wed, 31 Jul 2024 16:44:09 GMT

I am trying to make my first python extension. I am having trouble with the below plugin.py (keeps getting KeyError: 'results_builder'. I have not had much luck finding examples of what I want to do. Not really finding any doc out there on the ruxit.api I have easily built a similar script that uses the metrics API to ingest but trying to convert that to Oneagent Extension has been challenging. You can see I have been trying several iterations of invoking the self.results_builder*.

import psutil

from ruxit.api.base_plugin import BasePlugin

from ruxit.api.data import PluginMeasurement

import logging

import os

# Set up logging for Windows

log_file_path = os.path.join(os.getenv('PROGRAMDATA'), 'dynatrace', 'custom_browser_metrics.log')

logging.basicConfig(level=logging.DEBUG, filename=log_file_path, filemode='w')

logger = logging.getLogger()

class PythonPluginHost(BasePlugin😞

def initialize(self, **kwargs😞

logger.debug("Plugin initialized")

def query(self, **kwargs😞

#pgi = self.find_single_process_group(pgi_name('plugin_sdk.demo_app'))

#pgi_id = pgi.group_instance_id

#stats_url = "http://localhost:8769"

#stats = requests.get(stats_url, timeout=10).json()

results_builder = self.results_builder

if not results_builder:

logger.error("results_builder not found in kwargs")

return

browsers = ["chrome", "msedge"]

for browser in browsers:

metrics = get_process_metrics(browser)

if metrics:

cpu = metrics["CPU"]

memory = metrics["Memory"]

logger.debug(f"Sending metrics for {browser}: CPU={cpu}, Memory={memory}")

#self.results_builder.absolute(key="custom.browser.cpu", value=cpu, dimensions={"browser": browser})

self.results_builder.add_absolute_result(PluginMeasurement(key="custom.browser.cpu", value=cpu))

#self.results_builder.absolute(key="custom.browser.{browser}.cpu", value=metrics["CPU"])

#self.results_builder.absolute(key="custom.browser.memory_percent", value=memory, dimensions={"browser": browser})

self.results_builder.add_absolute_result(PluginMeasurement(key="custom.browser.memory_percent", value=memory))

#self.results_builder.absolute(key="custom.browser.{browser}.memory",value=metrics["Memory"])

else:

logger.debug(f"No metrics found for {browser}")

def get_process_metrics(process_name😞

total_memory = psutil.virtual_memory().total # Total memory in bytes

cpu = 0

memory = 0

for proc in psutil.process_iter(['name', 'cpu_percent', 'memory_info']):

if proc.info['name'] and process_name.lower() in proc.info['name'].lower():

cpu += proc.cpu_percent(interval=1) # Allow psutil to measure CPU usage over a short interval

memory += proc.info['memory_info'].rss # Resident Set Size in bytes memory_percent = (memory / total_memory) * 100

memory_percent = (memory / total_memory) * 100 # Convert bytes to percentage

logger.debug(f"Process: {process_name}, CPU: {cpu}, Memory: {memory_percent}")

return {"CPU": round(cpu, 1), "Memory": round(memory_percent, 1)} if cpu or memory else None

# If running as a script, instantiate and query

if __name__ == "__main__":

plugin = PythonPluginHost()

plugin.initialize()

plugin.query()

The plugin.json file for this:

{

"name": "custom.python.browser_metrics",

"version": "1.0.0",

"metricGroup": "custom_browser.metrics",

"technologies": ["PYTHON"],

"type": "python",

"entity": "HOST",

"metrics": [

{

"timeseries": {

"key": "custom.browser.cpu",

"displayname": "Browser CPU Usage",

"unit": "Percent"

}

{

"timeseries": {

"key": "custom.browser.memory_percent",

"displayname": "Browser Memory Usage",

"unit": "Percent"

}

"interval": 60

}

Re: OneAgent Plugin for browser process metrics

Julius_Loman — Wed, 31 Jul 2024 20:38:29 GMT

@rseibert1 First of all - and that's most important, your example uses Extensions 1.0 which will be removed from Dynatrace this October.
Secondly, you are fetching regular process metrics (CPU/memory) which are available out of the box with OneAgent, also in Infrastructure mode. You might want to setup Declarative process grouping if your processes are "glued" together. If those metrics do not fit your use case or you want to have another data and this was just an example, have a look at Extension 2.0 , especially at WMI data source or at Python Data source.
For Python, I'd recommend starting with this observability clinic - A Practical Guide to Building Python Based Extensions with Dynatrace.

Re: OneAgent Plugin for browser process metrics

rseibert1 — Tue, 06 Aug 2024 12:49:01 GMT

Thanks Julius for pointing me down the 2.0 path, I had looked into that earlier but for some reason I thought I could just convert to 2.0 later.

I have built a python extension in 2.0 format and it runs fine with the simulator, I am developing on a system that also has oneagent installed as well as the EEC enabled. When I upload and try to add a configuration for a specific host, I am lost on what to do for an endpoint as I want this to run locally so there is no endpoint/user/passwd configuration. Would I remove remote part of the activation from extension.yaml? Currently it has:

activation:

remote:

path: activationSchema.json

local:

path: activationSchema.json

which came from default extension creation.

Re: OneAgent Plugin for browser process metrics

Julius_Loman — Tue, 06 Aug 2024 14:29:35 GMT

You need to have activation in the extension. If this is a OneAgent extension only, you can remove the remote part - this is valid only for extensions executed by the ActiveGate. The extension must have some activationSchema which includes the extension configuration - you can remove all the configuration options in your activationSchema.json (when you generate the project, it generates something for you).

Re: OneAgent Plugin for browser process metrics

rseibert1 — Tue, 06 Aug 2024 20:06:23 GMT

I think if I am following you I can modify the activationSchema such that it will not require configuration after uploading?
In attempting to remove the config options I am getting the below error:
dtcli.utils.ExtensionValidationError: {"error":{"code":400,"message":"Activation schema error: Schema validation failed with Exception: 'No properties specified for complex type dynatrace.datasource.python:python_browser_metrics-localhost' in Schema 'template'."}}

The activationSchema.json is attached let me know what is missing?

Re: OneAgent Plugin for browser process metrics

Julius_Loman — Tue, 06 Aug 2024 21:41:17 GMT

TBH I'm not sure if the schema allows such an empty configuration. Unfortunately, there is no public doc for the activation schema. Maybe @Mike_L or @david_lopes can help here.

Re: OneAgent Plugin for browser process metrics

Mike_L — Thu, 15 Aug 2024 10:28:26 GMT

It indeed doesn't support an empty schema. Whenever we need that we just add one dropdown for logging to make it easier to change the logging of the extension if you run into issues.

Re: OneAgent Plugin for browser process metrics

duey_hoang — Wed, 21 Aug 2024 22:16:21 GMT

i normally leave the default alone, just add your own configuration after the default in activationschema. If doing local oneagent extension, add your logic in _main_.py. After deployed, they are on the configuration page, and you can enter garbage there as endpoint, and it just does not do anything.