OpenPipeline is Dynatrace's platform component to handle ingestion of all data types - whether observability, security or business data - for any source and format. It's built for very large scale and deeply integrated into the Dynatrace platform.
The OpenPipeline gives access to all configuration related to ingestion of data - that's different from "capturing data", so OneAgent configuration is not part of OpenPipeline.
Configuration in the OpenPipeline is split into data types: logs, events, Application Security,... (the coverage will continuously be extended).
Within each data type there are tabs for Ingest sources, Dynamic routing and Pipelines.
- Ingest sources represent how data enters Dynatrace. This could be an HTTP endpoint, OneAgent, OTLP endpoint, extension, or similar, such as OpenTelemetry Collector or Data Acquisition Controller. Endpoints can receive from any data collectors, such as OpenTelemetry, FluentD, custom services…
- Pipelines encapsulate all processing, data extraction and storage assignment - hence also sensitive data handling and security. Here masking, transformation, filtering, adding/removing/renaming fields, extracting metrics or bizevent and much more is configured.
- Routing decides for each record from each source to which pipeline it should go. This can be as fine-grained or coarse as needed, depending on how the customer organization is set up.
We're constantly adding new features to OpenPipeline and making configuration more convenient. So stay tuned via blog posts and release notes.
