https://community.dynatrace.com/t5/Open-Q-A/What-are-the-pipelines-within-OpenPipeline/m-p/261174#M34537 <P><SPAN data-contrast="auto">In the context of data ingest, pipelines are where filtering, processing, enrichment, and extraction of incoming data happen. All of this is defined by “Processors.” </SPAN><SPAN data-ccp-props="{&quot;335559685&quot;:360}">&nbsp;</SPAN></P> <P><SPAN data-contrast="auto">The main benefits of using pipelines are </SPAN></P> <UL> <LI><SPAN data-contrast="auto">clear split of data streams</SPAN></LI> <LI><SPAN data-contrast="auto">distribution of processing configuration and testing to teams that are the data owner&nbsp;</SPAN></LI> </UL> <P><SPAN data-contrast="auto">A pipeline is linked to a specific data type. There can be multiple pipelines per datatype. The dynamic routing defines the pipeline that a record is sent one. Each record goes through only one (!) pipeline.</SPAN></P> <P><SPAN data-contrast="auto">A pipeline contains several steps, called “Stages”: Processing, Metric extraction, Data extraction, Permission, Storage.</SPAN></P> <P>a more detailed explanation can be found in the&nbsp;<A href="https://docs.dynatrace.com/docs/platform/openpipeline/concepts/data-flow" target="_blank">OpenPipeline Documentation</A>.</P> Thu, 31 Oct 2024 09:50:41 GMT thomas_billi 2024-10-31T09:50:41Z https://community.dynatrace.com/t5/Open-Q-A/What-are-the-pipelines-within-OpenPipeline/m-p/261173#M34536 <P><SPAN class="TextRun SCXW85399450 BCX0" data-contrast="auto"><SPAN class="NormalTextRun SCXW85399450 BCX0">What does it consist of? Can a pipeline be linked to multiple data types?</SPAN></SPAN></P> Thu, 31 Oct 2024 08:00:15 GMT https://community.dynatrace.com/t5/Open-Q-A/What-are-the-pipelines-within-OpenPipeline/m-p/261173#M34536 GosiaMurawska 2024-10-31T08:00:15Z https://community.dynatrace.com/t5/Open-Q-A/What-are-the-pipelines-within-OpenPipeline/m-p/261174#M34537 <P><SPAN data-contrast="auto">In the context of data ingest, pipelines are where filtering, processing, enrichment, and extraction of incoming data happen. All of this is defined by “Processors.” </SPAN><SPAN data-ccp-props="{&quot;335559685&quot;:360}">&nbsp;</SPAN></P> <P><SPAN data-contrast="auto">The main benefits of using pipelines are </SPAN></P> <UL> <LI><SPAN data-contrast="auto">clear split of data streams</SPAN></LI> <LI><SPAN data-contrast="auto">distribution of processing configuration and testing to teams that are the data owner&nbsp;</SPAN></LI> </UL> <P><SPAN data-contrast="auto">A pipeline is linked to a specific data type. There can be multiple pipelines per datatype. The dynamic routing defines the pipeline that a record is sent one. Each record goes through only one (!) pipeline.</SPAN></P> <P><SPAN data-contrast="auto">A pipeline contains several steps, called “Stages”: Processing, Metric extraction, Data extraction, Permission, Storage.</SPAN></P> <P>a more detailed explanation can be found in the&nbsp;<A href="https://docs.dynatrace.com/docs/platform/openpipeline/concepts/data-flow" target="_blank">OpenPipeline Documentation</A>.</P> Thu, 31 Oct 2024 09:50:41 GMT https://community.dynatrace.com/t5/Open-Q-A/What-are-the-pipelines-within-OpenPipeline/m-p/261174#M34537 thomas_billi 2024-10-31T09:50:41Z