EXTRACT TOPOLOGY WITH OPENPIPELINE

[FredSelvaGen]

Hi community,

Let's say that a SERVICE named "cart_service" is called by 2 others one (named "webapp_service" and "mobileapp_service")

By using a span openpipeline,  for each collected traces that involved "cart_service", I would like : 

• to Get the caller service id (dt.smartscape.service) 
• to Add this ID as new field in "cart_service" in subtrace root span

I've tried to use "Smartscape edge" feature like below

//DQL processor

fieldsadd dt.smartscape.service_cart = dt.smartscape.service

// Smartscape edge processor

Source type : SERVICE_CART
Source ID field name : dt.smartscape.service_cart

Custom EDGE_TYPE : called_by
Target type : SERVICE_CALLER
Target ID field name : dt.smartscape.service_caller

Unfortunately I get nothing in dt.smartscape.service_caller field

Is this a good usage of "Smartscape edge" feature?

Can you help me?

Thank you 

Freddy

[36Krazyfists]

Tip for you.  The new SmartScape extraction is great, but it doesn't do everything the old Topology Extraction rules did (yet).  

What I do is create something I call "dummy" extensions.  They are Dynatrace Extensions without a data source (typically an extension is used to collect information from a SNMP/WMI/SQL/Prometheus/JMX/Python source.  But I found you don't have to have a data source defined in your extension.).

What I do with these extensions is define my OpenPipeline route and pipelines (this includes my processors, smartscape extractions, metric/event extractions, and bucket assignments), my classic custom topology extractions, detail screens that place the data on the Infrastructure & Operations application (this is one of the things you have to have classic custom topology extractions for.  Extracting just the Smartscape nodes from your telemetry will not allow those entities to show up in the new Infrastructure & Operations app nor in the classic custom device "screens".), overview dashboards (both latest dashboard documents and classic dashboards), and alert templates.

I've basically trained my Cursor (I use Claude Opus as the model as I found that Auto mode; which relies mostly on Cursor's own Composer model; is more prone to mistakes than Claude) by feeding it Dynatrace's "Dynatrace for AI" skill that they share on their public Github repo as well as feeding it all of Dynatrace's Extension docs and a bunch of different official extensions as examples.

It works really well and I can spin up a new working "dummy" extension super quick.  It usually takes several iterations where I have to correct it and make changes, but it's still CONSIDERABLY faster than when I built these myself.  I've gone from weeks to days.

Having said all that, to address your question directly, Edge is synonymous with the old "Relationships" portion of the classic topology model.  Smartscape Nodes are synonymous with the classic "General Types" or entity extraction portion of the old topology model.

So, what I'm saying is that an Edge is for connecting two Smartscape Nodes together.

So, my first question would be: Are you extracting your desired entities that you want to relate together as Nodes?  

If you are already extracting the SERVICE_CART and SERVICE_CALLER entities as Nodes, then you also need to make sure that the data you are using to relate the two together is present in a single span.  OpenPipeline processors only look at one record at a time.  They are unaware of any other span in a distributed trace.  In other words, the way you relate them is by extracting both Node types from a single span.  This will put the entity id of both nodes into that span record.  Then, the edge simply defines a relationship between those two nodes.

You can't define a relationship between two disparate nodes if the ID values for both don't show up in the same record.

Having said that, Dynatrace already maps services together in a distributed trace.  There shouldn't be a need to establish that relationship if you are already seeing the spans for those services in the same trace.

If you aren't seeing that already, then I would say check your service detection rules.  You might not be detecting the various services properly and that's why you are seeing the issue.

[FredSelvaGen · Accepted solution]

Hello  @36Krazyfists 

Thank you for this explanation.

If I understand well, Smartscape features in openpipeline are used for setting up a topology in dynatrace....and not for getting entities topologies or relationships informations.

Thank you also for having share one of openpiline limits : you are right these 2 nodes are not belonging to the same span (not in same record). 

What I would like to create as a metric (without query cost), is the answer to this question : 

"As an SERVICE APPLICATION owner ("cart_service") , can you provide me the performance metrics from each caller/client point of view ("webapp_service" and "mobileapp_service")?" 

It looks like that my need is similar to reproduce in Dynatrace Gen3 a "old" Calculated Service metrics that using Request attribute that  use "matching donwstream services" feature. 

Anyone knows if this kind of features will be handle soon?

Regards.

Freddy