I'm trying to write a custom log processing rule that takes a log file containing a JSON object on each log line and:
* parse some fields from the content and populate custom attributes
* override the content attribute
Here is my rule definition:
* Matcher: `matchesValue(k8s.deployment.name, "splash-cake-cronjob-bulkJob-*")`
* Processor definition
FIELDS_RENAME(contentJson:content)
| PARSE(contentJson,"JSON:parsedJson")
| FIELDS_ADD(
content: parsedJson["message"],
splapi.requesthash: parsedJson["requestHash"],
splapi.pid: parsedJson["pid"],
splapi.organizationid: parsedJson["orgId"],
salesforce.splashjobid: parsedJson["splashJobId"],
salesforce.status: parsedJson["record"]["Status"],
salesforce.id: parsedJson["record"]["Id"],
salesforce.contactid: parsedJson["record"]["ContactId"],
salesforce.leadid: parsedJson["record"]["LeadId"],
salesforce.campaignid: parsedJson["record"]["CampaignId"],
salesforce.error: parsedJson["record"]["sf__Error"],
loglevel: UPPER(STRING(parsedJson["loglevel"])),
)
| FIELDS_REMOVE(parsedJson)
Sample log:
{
"content": "{\"requestHash\":\"e3310d9e\",\"pid\":548,\"serviceName\":\"cake\",\"domain\":\"unspecified\",\"splashJobId\":\"854579\",\"orgId\":1315,\"record\":{\"sf__Id\":\"00vHp00002PoAtIIAV\",\"sf__Error\":\"DUPLICATE_VALUE:Duplicate id specified:00vAA11112PoAtIIAV:--\",\"Id\":\"00vAA11112PoAtIIAV\",\"CampaignId\":\"00vAA11112PoAtHHHH\",\"Status\":\"Added\",\"LeadId\":\"\",\"ContactId\":\"00vAA11112PoAtZZZZ\"},\"message\":\"package\\\\SomeIntegration\\\\BulkHandler\\\\Class::processResults Job id 854579(750Hp00001H08itIAB): Processing record\",\"timestamp\":\"2023-11-21 14:31:04\",\"loglevel\":\"info\"}",
"k8s.deployment.name": "splash-cake-cronjob-bulkJob-*"
}
My custom fields are being parsed properly, but I'm unable to override the content field. What should I change?
