Yes, the minimum resolution of the metric is 1 minute. But because the query method is called every 60.0x seconds, there is a minute where data cannot be collected.
To make it easier to understand, if the query method is called every 60.1 seconds, it will look like this.
Time when the query method was called
- 2024-06-17 08:25:59.876
- 2024-06-17 08:26:59.976
- 2024-06-17 08:28:00.076
- 2024-06-17 08:29:00.176
Looking at python3.log, it looks like this, and the data for 08:27 was actually missing.
Is it wrong to retrieve monitoring items from the OS in the query method and register metrics?
Your extension is manually reporting the timestamp for the metric, and that is where your issue comes from, but lets assume folks always wanted to manually report the timestamp for their metrics (not the case), instead of letting the SDK calculate it.
This scenario you describe never happened in the logs you shared, like I mentioned before extensions are not scheduled to run at 00:00.00 of the minute.
The real logs look like:
2024-06-20 15:14:30,168 [INFO] dynatrace_extension.sdk.extension (ThreadPoolExecutor-0_0): query START mysample:0.0.2.
2024-06-20 15:15:30,173 [INFO] dynatrace_extension.sdk.extension (ThreadPoolExecutor-0_0): query START mysample:0.0.2.
2024-06-20 15:16:30,184 [INFO] dynatrace_extension.sdk.extension (ThreadPoolExecutor-0_0): query START mysample:0.0.2.
So there is a "cushion" of several seconds before that scenario could ever happen, 30 seconds in this case.
For that scenario to happen at 4ms per minute the extension would need to run for 7500 cycles, so because you are manually setting the timestamp of your metric, you are responsible for reporting it every minute, you denied the SDK the option to adjust the metric timestamp.
Even in that worst case scenario, you will lose 1 datapoint every 7500 minutes, which is acceptable because you took the risk of setting your own timestamps.
If you didn't set the timestamp manually, the SDK calculates the metric timestamp based on the first time the method was scheduled, which guarantees that no datapoint will be lost, you can check the code here and here
In summary, remove the timestamp argument from your report_metric method and you will never lose a datapoint
The code comments even describe the same issue you have.
Thank you. I didn't know this mechanism.
> In summary, remove the timestamp argument from your report_metric method and you will never lose a datapoint
Following this advice, I modified the code as follows and ran it for about 7 hours.
def query(self):
# ### Some parts omitted ###
now = datetime.datetime.now()
self.report_metric(
key=metric_key,
value=now.second,
dimensions=dimensions,
# timestamp=now, # this line was commented out
)
The graph for this metric now looks like this:
However, if as you say, there is no data loss with this implementation, I will need to wait a few more hours until the start second changes from 59 to 00.
I am attaching the result of grepping this version of python3.log with START. If there are any other files that would be useful in investigating this, please let me know.
Hmm, I'm in trouble.
As David_lopes said, I removed the timestamp from the report_metirc argument and checked to see if any metrics were missing.
def query(self):
# ### Some parts omitted ###
now = datetime.datetime.now()
self.report_metric(
key=metric_key,
value=now.second,
dimensions=dimensions,
# timestamp=now, # this line was commented out
)
Finally, the moment when the millisecond changes from 9xx to 0xx has arrived, so take a look.
2024-06-21 17:50:59,902 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:51:59,926 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:52:59,956 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:53:59,985 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:55:00,015 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:56:00,023 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:57:00,053 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:58:00,081 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 17:59:00,111 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
2024-06-21 18:00:00,121 [INFO] dynatra ... utor-0_0): query START mysmp2:0.0.6.
As you can see from the logs, the plugin did not start at 17:54.
And when I checked DataExplorer, the data at 17:55 was missing.
If you didn't set the timestamp manually, the SDK calculates the metric timestamp based on the first time the method was scheduled, which guarantees that no datapoint will be lost.
Indeed, the metric timestamp appears to be calculated automatically, but ultimately one point of data was missing.
Is it inevitable that extensions developed using the SDK will have occasional defects? Or is there something wrong with the way I implement it?
Featured Posts
