Data Observability, borrowing ideas from Software Observability, pertains to understanding an organization's full data lifecycle. It involves monitoring and managing the internal state of data systems from ingestion to storage and usage. It is about gaining insight into the data pipeline, understanding how data evolves, and identifying any issues that could compromise its integrity or reliability.
In this Observability Lab hosted by Andi Grabner @andreas_grabner , David Bruendl @DavidBruendl and Wolfgang Beer @wolfgang_beer explain how Dynatrace Davis Anomaly Detection can be applied to the 5 Pillars of Data Observability: Freshness, Distribution, Volume, Schema, and Lineage.
David and Wolfgang show us how Davis Anomaly Detection based on any data in Grail can be applied in Notebooks, Dashboards, Workflows, and in the recently introduced Davis Anomaly Detection App to automate the alerting on Data Observability issues!
Links discussed:
Introduction to Anomaly Detection based on DQL
Data Observability with Dynatrace: https://dynatr.ac/4b90o7U
Davis Anomaly Detection: https://dynatr.ac/4biQpN2
Blog on Data Observability: https://dynatr.ac/4b63JUW
Blog on the Importance of Data Quality: https://dynatr.ac/3WvxnP9
Chapter List:
00:00 - Introduction
00:50 - What you will be learning today
01:22 - Recap of Anomaly Detection with DQL session
04:00 - What is Data Observability?
05:03 - Data-driven Enterprises
06:06 - Example of Data Pipelines such as ETL
08:25 - 5 Pillars of Data Observability
12:02 - Live Demo of Data Observability with Dynatrace
- - -
Subscribe to our YT channel
Stay up-to-date with Dynatrace! Follow us on Facebook, Instagram, LinkedIn, Twitter, Twitch
⭐ When passion meets people magic and innovation happen. ⭐
