topic Re: How do we ingest files into Dynatrace SAAS that has .json (extension)? in Open Q&A

How do we ingest files into Dynatrace SaaS that has .json?

karthikr — Thu, 07 Mar 2024 09:01:49 GMT

Team,

Can anyone please let me know how do we ingest files with .json extension onto Dynatrace SAAS environment? I am able to ingest files with .log extension but not files with .json extension.

Re: How do we ingest files into Dynatrace SAAS that has .json (extension)?

PacoPorro — Thu, 22 Feb 2024 13:44:21 GMT

Please check custom log source doc and pay special attention to the security rules .

Re: How do we ingest files into Dynatrace SAAS that has .json (extension)?

karthikr — Thu, 22 Feb 2024 14:19:21 GMT

Thanks @PacoPorro for your response. I believe I had gone through this already. In-fact I created .json file under this path /var/lib/dynatrace/oneagent/agent/config/logmodule to allow .json file to discover by OneAgent.

{
"@version": "1.0.0",
"allowed-log-paths-configuration": [
{
"directory-pattern": "/tmp/",
"file-pattern": "*.json",
"action": "INCLUDE"
}
]
}

I included the above content as the json file to override the security rules, but however after configuring Ingest rules and when trying to add json file path under Custom Log Source, I am seeing below warning message.

Provided value contains following potential issues:

not recognized as log, may not be allowed due to security rules

And when I look at the Host page for log sources, it says the following,

/tmp/example1.json (No log data available)

I am not sure what I am doing wrong with any of the configuration, please advise.

Re: How do we ingest files into Dynatrace SAAS that has .json (extension)?

PacoPorro — Thu, 22 Feb 2024 15:17:13 GMT

Could you please check in the Log Agent's own logs?

Re: How do we ingest files into Dynatrace SAAS that has .json (extension)?

PacoPorro — Thu, 22 Feb 2024 15:46:50 GMT

I just tested it in my lab and it works.

I created your rule

Agent restarted just in case.
Custom log source created. I ignored the security rule warning.

Added the ingest rule.

Re: How do we ingest files into Dynatrace SAAS that has .json (extension)?

karthikr — Thu, 22 Feb 2024 16:21:52 GMT

Thank you so much @PacoPorro for testing it so quickly. I believe the issue was related to the timestamp key/value pairs that I was using in the .json file, when I created new event with latest timestamp after enabling the log ingesting rules + custom log source path, it worked. I will continue to perform the testing. Thanks for all your help.