Ingesting log .net on Kubernetes

[Ellery]

Hello, 

Hi, my head is splitting trying to set up log ingestion. I'm working on Azure Kubernetes with Dockerized containers. This retrieves a log using log4net, which is created at the path: /app/logs/application.log. The log exists, 

# ls -la /app/logs
total 12
drwxr-xr-x 2 root root 4096 Mar  4 14:43 .
drwxr-xr-x 1 root root 4096 Mar  4 14:43 ..
-rw-r--r-- 1 root root 3204 Mar  4 15:43 application.log

this woks ok, 

# tail -f /app/logs/application.log
2024-03-04 14:43:03,232 [11] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=843e41f18ab7f063,trace_sampled=true] Application - Main is invoked
2024-03-04 14:43:04,944 [11] ERROR ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=843e41f18ab7f063,trace_sampled=true] [Firmar][Web firmadora] No se encontro la web firmante.
2024-03-04 14:43:05,136 [11] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=843e41f18ab7f063,trace_sampled=true] [Firmar][INICIO][17427955-1] Se firmaran 2
2024-03-04 14:43:09,545 [4] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=22958368528b0b5c,trace_sampled=true] [Proceso de firma][17427955-1] Tiempo total de ejecución
2024-03-04 14:43:09,650 [4] WARN  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=22958368528b0b5c,trace_sampled=true] [SGP][17427955-1][Info] Se prepara para enviar información a SGP
2024-03-04 14:43:09,732 [4] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=22958368528b0b5c,trace_sampled=true] [Firma][17427955-1] Documentos firmados, enviando a digitalización.
2024-03-04 14:43:09,839 [4] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=22958368528b0b5c,trace_sampled=true] [Firmar][FIN][17427955-1] Se completo el proceso de firma.
2024-03-04 14:43:09,843 [4] INFO  API_FirmaDigital2.Controllers.AdmisionController - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=22958368528b0b5c,trace_sampled=true] [Firmar][FIN][17427955-1]
2024-03-04 15:43:26,582 [29] ERROR ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=28c6324e98a572f0,trace_sampled=true] [Firmar][Web firmadora] No se encontro la web firmante.
2024-03-04 15:43:26,583 [29] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=28c6324e98a572f0,trace_sampled=true] [Firmar][INICIO][17427955-1] Se firmaran 2
2024-03-04 15:43:26,771 [30] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=b44df715bb86f809,trace_sampled=true] [Proceso de firma][17427955-1] Tiempo total de ejecución
2024-03-04 15:43:26,775 [30] WARN  ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=b44df715bb86f809,trace_sampled=true] [SGP][17427955-1][Info] Se prepara para enviar información a SGP
2024-03-04 15:43:26,776 [30] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=b44df715bb86f809,trace_sampled=true] [Firma][17427955-1] Documentos firmados, enviando a digitalización.
2024-03-04 15:43:26,856 [30] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=b44df715bb86f809,trace_sampled=true] [Firmar][FIN][17427955-1] Se completo el proceso de firma.
2024-03-04 15:43:26,856 [30] INFO  API_FirmaDigital2.Controllers.AdmisionController - [!dt trace_id=cc674496662e9ddfa3528b34e4f87dd0,span_id=b44df715bb86f809,trace_sampled=true] [Firmar][FIN][17427955-1]

and I've set up the rules in Dynatrace."

custom log

rules ingest

So, i have a configurated in a dynatrace but in .json :

cat /opt/dynatrace/oneagent/agent/conf/securityRulesLoganalytics.json

# cat /opt/dynatrace/oneagent/agent/conf/securityRulesLoganalytics.json
{
  "@version": "1.0.0",
  "allowed-log-paths-configuration": [
    {
       "directory-pattern":"/",
       "file-pattern":"*.pem",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"/.ssh/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"/.*/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"/",
       "file-pattern":".*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/etc/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/boot/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/proc/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/dev/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/bin/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/sbin/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
       "directory-pattern":"^/usr/**/",
       "file-pattern":"*",
       "action":"EXCLUDE"
    },
    {
      "directory-pattern": "/",
      "file-pattern": "*[-.\\_]log[-.\\_]*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/",
      "file-pattern": "*[-.\\_]log",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/",
      "file-pattern": "catalina.out*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/log/",
      "file-pattern": "*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/log/*/",
      "file-pattern": "*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/logs/",
      "file-pattern": "*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "/logs/*/",
      "file-pattern": "*",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "^/var/lib/docker/containers/*/",
      "file-pattern": "*.log",
      "action": "INCLUDE"
    },
    {
      "directory-pattern": "^/var/log/**/",
      "file-pattern": "*",
      "action": "INCLUDE"
    }
  ]
}

The configuration from the Dynatrace web interface is not reflected and it cannot find custom logs.

[natanael_mendes · Accepted solution]

i think "INICIO" is not on content.

2024-03-04 14:43:05,136 [11] INFO  ACHS.FirmaDigital.Api.Program - [!dt trace_id=ffa3e2491e89ce51e045d18ded80461b,span_id=843e41f18ab7f063,trace_sampled=true] [Firmar][INICIO][17427955-1] Se firmaran 2

in this case the content is "Se firmaran 2" inicio is a field on the log. search for "Se firmaran 2" and you gon find this log

[Ellery]

Don't work, i make the query, 

I thinks that logs dont ingest into dynatrace logs tables, how i can verify thas logs was ingested into dynatrace ?

[natanael_mendes]

you can filter by the "ingest" field

[Ellery]

What you means with field "ingest" thats field does't exist into my query.

[Suryanto_1]

Should you be using matchesPhrase instead ?

matchesPhrase(content, "error")

https://docs.dynatrace.com/docs/observe-and-explore/logs/lma-log-processing-matcher