topic Re: [Q] - How to mask IPv6 on ingestion in DQL

[Q] - How to mask IPv6 on ingestion

mark_forrester — Fri, 26 Jul 2024 11:40:42 GMT

Hi Community

I'm trying to mask a IPv6 address on log ingestion but struggling to get the commad right

I find the IPv6 in the logs using the following command
// detect ip
| parse content, "LD IPADDR:ip LD"
| fieldsAdd ip = if(ipIsPublic(ip), ip, else: null)

However when trying to mask using a similar command I get the below error, can anyone advise the right command to mask a IPv6 detected string in a log entry.

USING(INOUT content)
| PARSE (content,"LD IPADDR:ip LD")
| FIELDS_ADD(content: REPLACE_PATTERN(content, ip,"*** DT MASKING ***"))

Re: [Q] - How to mask IPv6 on ingestion

AntonPineiro — Fri, 26 Jul 2024 14:55:41 GMT

Hi,

Maybe, can you share some logs lines content where you want to extract that information?

Best regards

Re: [Q] - How to mask IPv6 on ingestion

DanielS — Fri, 26 Jul 2024 21:00:11 GMT

Hello @mark_forrester Hope it helps

Is not ingestion but does the same job

Re: [Q] - How to mask IPv6 on ingestion

mark_forrester — Mon, 29 Jul 2024 08:14:53 GMT

Here is an edited sample content output in DT @DanielS your example fails on this example as no matches found.

{ "log-type":"access", "domain":"@domain_name@", "time":"2024-07-29T08:03:53.435Z", "process":"582", "filename":"/var/www/html/rb_bf61376cao", "remoteIP":"2a00:2a00:6416:5d01:51bb:1d00:e019:92c1", "x-forwarded-for":"-", "host":"www.ourdomain.co.uk", "request":"/rb_bf61376cao", "query":"?type=js3&sn=v_4_srv_1_sn_89E82F0035A9458ED42E462151D39F6F_mvisitor_1724274919_msn_4-2D0_perc_100000_ol_0_mul_1_app-3Afc8a255d717e7ed8_1_rcs-3Acss_0&svrid=1&flavor=post&vi=PDHUTTNIVUULRVNDHPMLWCCVTKRBRRKR-0&modifiedSince=1722019465728&rf=https%3A%2F%2Fwww.ourdomain.co.uk%2Fpubdeals%3Futm_source%3Dsizzlapp%26utm_medium%3Dapp%23%2F&bp=3&app=fc8a255d717e7ed8&crc=3798105690&en=l3f8ysai&end=1", "method":"POST", "status":"200", "userAgent":"Mozilla/5.0 (Linux; Android 14; SM-A528B Build/UP1A.231005.007; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/126.0.6478.134 Mobile Safari/537.36", "referer":"https://www.ourdomain.co.uk/pubdeals?utm_source=sizzlapp&utm_medium=app" },

Re: [Q] - How to mask IPv6 on ingestion

DanielS — Wed, 31 Jul 2024 19:43:59 GMT

Hello @mark_forrester using this regex in regex101 works perfect with all examples:

(?:[a-fA-F0-9]{1,4}:){7}[a-fA-F0-9]{1,4}

and also in Dynatrace:

let me know how it goes.

Re: [Q] - How to mask IPv6 on ingestion

mark_forrester — Thu, 01 Aug 2024 07:33:07 GMT

Thank you this works a treat 😁