Extension Framework 1.0 (EF 1.0) will be sunset in October 2024

[AntonioSousa]

Just received this, from a client account, and cannot believe it:

As someone that has developed several dozen of extensions, most of them in EF 1, the target is not doable! What is worse is knowing this, this way.

@Mike_L, we need some urgent clarifications on this...

[Mike_L]

Hello,

This message was sent out to all accounts we believe are affected by this discontinuation.

There is an accompanying blog post coming with additional information. We are also working on a how-to guide on how to convert Python extensions to the new framework as soon as customers and partners can do that.

I have forwarded this thread to the work group who are busy with the EF1 removal.

Mike

[AntonioSousa]

@Mike_L,

There are so many doubts about this, but I'm forwarding this internally. We have about 40 extensions, some 10 to 15 scheduled, and I knew this from a client?

[Julius_Loman]

Yes, that's true. Unfortunately, Python Code Extensions 2.0 are not even available yet. We definitely need to know what the sunset in October 2024 means. Extensions will stop working completely? Just new extensions could not be added? Just support will be discontinued and the extensions will keep working?

Even with a successful conversion, there can be quite a significant impact on existing customers in terms of DDUs, as the EF2 is based on a continuous "data-stream" (logs/events/metrics/...) to have the entities created and alive (at least once in 15 minutes). There is currently no option to create the entities without it as it was possible in EF1.

[Mike_L]

Python extensions in EF2 is scheduled to be available somewhere around Perform. We are aiming to release the how-to guides on conversions in December or January so people can prepare for it.

The sunset date means that we will remove the extension framework from OneAgents and ActiveGates.

If you are aware of use cases which is not possible with Extension Framework 2.0 our product management team will be interested to hear about that.

[AntonioSousa]

@Mike_L,

This can't happen like this. Remembers me of Custom Charts 😞
https://community.dynatrace.com/t5/Dashboarding/Custom-charting-deprecation-and-Data-Explorer-gaps/m...

[Mike_L]

We will provide some automation scripts as part of the vscode plugin from my team to simplify the conversions of Python extensions, such as converting a part of the plugin.json file. We are also able to help out through Dynatrace services if the 12 month advance notice is too limited for planning.

For JMX the automatic conversions instructions are already available on https://developer.dynatrace.com/extensions-v2/dynatrace-extensions-vscode/guides/jmx-conversion/

[AntonioSousa]

@Mike_L,

There are so many questions that have arised in the last two hours, since I was notified of this. The Community is certainly not the place to put them, so I have channeled them internally. I hope to get replies really fast.

We all value your contributions here, and especially how fast they are. But I'll repeat what I said initially, given what else I have discovered in the meantime: this sunset timing is not reasonable, and in my opinion, not doable...

[AntonioSousa]

Will staying at a specific version in October 2024, eg. AG v1.299, maintain EF1 support?

[Mike_L]

Probably for a while, but it is uncertain for how long. I recommend against it though as Python 3.8 is out of support in October 2024 so you would be running unsupported software on your OAs/AGs.

[AntonioSousa]

Is this a decision associated with not willing to move beyond 3.8 ?

[Mike_L]

There are many reasons behind this decision. It all boils down to providing our customers with the best possible option for extending the Dynatrace platform so we can invest in a best in class, supported, easy, scalable, secure framework.

[AntonioSousa]

@Mike_L,

Sorry, can't understand how this helps in any of those five points. And mainly, it undermines confidence, going forward.

[Mike_L]

Hopefully the blog post planned by product management will clarify it sufficiently for everyone with questions.

[AntonioSousa]

Now that we are in the hope phase, I hope:

• Dynatrace reconsiders not retiring support for the framework, in the terms announced, given that not even today there is a clear understanding of how EF2 will support many of the existing EF1 possibilities.
• Now that we can't get the cat in the box again, Dynatrace has to publicly reference on how it will support it's frameworks&features in the future, including naturally EF2, but also SDKs, Unified Analysis View, AppEngine, AutomationEngine, etc. We, developers, Partners and especially Clients, need to have a clear understanding of Dynatrace's commitment to each one of them
• I hope this doesn't go down as the Custom Charts saga, for instance. Still today, more than two years after, Data Explorer still does not have features that Custom Charts has.
  https://community.dynatrace.com/t5/Dashboarding/Custom-charting-deprecation-and-Data-Explorer-gaps/m...

[michal_nalezin · Accepted solution]

Hi Antonio,

You're absolutely correct - I acknowledge that our communication with the experts could have been handled better. In September we did have two internal messages shared, followed now by this public message to customers. We are providing a 12-month window prior to the End-of-Life (EOL) of Extension Framework 1.0 (EF1.0). And as we understand that this timeframe may not suffice for some of you, and we want to ensure that Dynatrace will support you in various ways. Starting from automated JMX conversion to dedicated trainings on Python migration.

When we introduced Extension Framework 2.0 (EF2.0) in March 2021 (blog) we already mentioned that EF1.0 would eventually be phased out. Subsequently, our primary focus has been on maximizing the value of the new EF2.0, resulting in limited investments in EF1.0. As a result, maintaining its functionality and security beyond October '24 is not feasible especially that Python 3.8 (base for custom plugins) will reach end of life then.

You've rightly brought up the issue of feature parity between EF2 and EF1. While I agree that we may not have all the same functionalities in EF2 as we did with plugins, we are actively working to bridge most of these gaps or provide suitable workarounds. It's worth noting that a similar process occurred with Custom Charts, where functionalities were added during 12-month period after deprecation was announced.

In the meantime, I suggest we focus on two key areas:

1. How we can assist you and other customers with the migration to EF2.
2. Prioritizing the gaps that need attention.

Looking forward to discuss further
Michal Nalezinski (Senior Product Manager)
Dynatrace Enterprise Applications and Services

[AntonioSousa]

@michal_nalezin,

I have transmitted internally many issues  that I believe are important. You should have access to them, and I'll use the correct forums to deal with them. And that includes the two key areas you mention.

Now, here in the Community, we have to address at least the following:

• Dynatrace has to come forward and say how and till when will functionalities & features be supported? If not, developers, Partners and especially Clients, will not be able to build with confidence in the Dynatrace platform.
• Framing this as a python 3.8 support issue is unbelievable for me!
• Communication was deplorable, especially because the only notice I received was from one of the few tenants that I follow that doesn't have a single EF1 extension.
• I hope we don't go the way Custom Charts went. Still today, two and a half years later, it still has at least one feature very important to me, that is not supported elsewhere by Dynatrace.

Finally, I would say that this is a problem that at the moment few of us have a real perception of the impacts. I have talked to several other EF1 developers, and treatment was the same. Fortunately, clients don't know the real impacts this will have, at least for now.

[Julius_Loman]

@michal_nalezin where can we track the gaps and features, so we can speed up the feature parity? Product Ideas here in the community?

The largest gaps I see are:

• unable to create topology without continuous data stream (in EF1 you don't need any metric/log/event to have the topology). Despite this achievable by sending log/events or metric, it will would significant impact on the customers in terms of DDU as they may not have it in the budget or the extension value may drop due to increased cost in Dynatrace licenses. 
• There are no relations between Dynatrace builtin entities and custom topology entities based on IP addresses / ports, I had a support ticket opened a year ago for that https://one.dynatrace.com/hc/en-us/requests/99626 and it still does not seem to be working as in EF1.
• No possibility to have a link on the EF2.0 entity screen to an external system. EF1 had the button "Show details" which was used to open a source system (source of the data). Customers use that heavily with our Nagios and Zabbix extensions.
  

  

And yes, at least partners with existing extensions in the HUB need to have access to code-level extensions, preferably prior to Perform. We have existing customers, re-developing the extensions in EF2 will take some time and also we need to reach out to our customers to deploy it. 

[lucas_hocker]

Hi Julius, I'm curious about your use case where you want to create topology but not ingest any data. What would be the purpose of the entity, if you don't intend to send in any metrics or events for it?

[AntonioSousa]

@lucas_hocker,

Not about @Julius_Loman use-case, but I have one similar situation, involving CMDB integration.

[Julius_Loman]

Hi @lucas_hocker ,

one of the use cases integrating other 3rd party monitoring tools such as Zabbix and Nagios. Both those extensions are already in the Dynatrace HUB and we have several customers using them. Our two extensions create entities in smartscape and send problem opening events based on problem data in the integrated monitoring system. 

Benefits customers are now using:

• relations between OneAgent monitored entities and entities monitored just by the integrated system, for example for feeding CMDB as @AntonioSousa already mentions.
  As far as I tested, this feature is still not available in EF2 and relations between OneAgent monitored entities and custom topology are still not created as it is in EF1.
• Unified problem view - customers see problems from several monitoring tools in Dynatrace

Typically it's not beneficial just to pump metrics from one tool to another, so we have no continuous data stream. In EF2 we would need to send "dummy" data and the customer will need to pay DDUs for that. Customers really take DDU consumption into account. 

[AlanZ · Accepted solution]

Hi There,

Since we are bringing up the custom chart conversion, may we suggest the same approach for EF2. A big button developed by Dynatrace to handle the upgrade ?

[AurelienGravier]

@AntonioSousa just published :

https://www.dynatrace.com/news/blog/unmatched-scalability-and-security-of-dynatrace-extensions-now-a...

[AntonioSousa]

@AurelienGravier, thanks for the notice 😄

Unfortunately, nothing really new is said, while introducing even more doubt on the process...

The worst part is that this clearly undermines confidence in developers developing in Dynatrace. In a Platform that is trying to move forward convincing developers to invest into it, this is not a good signal for them/us.

[jegron]

Hi @AntonioSousa ! If you will attend to Perform, there will be training on this topic: https://www.dynatrace.com/perform/training/?training=dynatrace-extensions

[AntonioSousa]

@jegron,

Thanks. I know about Perform training, but there are several problems with that:

• We still have almost 3 months till Perform
• As Partners, we are intending to be present at the Partner Summit, so that invalidates Tuesday
• Being there an additional day has a cost
• HoT sessions are $500 for each attendee...

Fortunately, haven't booked flights & hotel yet, but probably not attending the HoT session, given that only one of the five points seems interesting to me.

[jonhaugen]

Hi, 
As a developer of about 50 custom Oneagent and a couple of activegate EF1 extensions I was shocked to get these news. Especially since the Python scripts in EF2 is not available yet. From the planned release of Python scripts I will have to convert two extensions a week until EOL, which is a lot of technical debt. 

Please consider me as an early adopter of EF2.0 python scripts and the documentation since this is of high criticality. 

I have already written a couple of EF2.0 extensions, so I am familiar with most concepts. 

Also, will the HoT session be available virtually? 

[Mike_L]

Hi Jon, I'll catch up with you via email.

The Perform HoT session will be given virtually starting sometime after Perform.

[jegron]

As partner, we are invited to attend a first workshop this Thursday titled "migrating Python-coded extensions from old Extension Framework 1.0 to new version 2.0".

The positive aspect is that the Dynatrace team doesn't rely on Perform to offer insights regarding this End of Life (EoL) situation. I appreciate it.

[Mike_L]

Good to hear that the session is appreciated. There is a second session planned next Thursday due to Thanksgiving this week.

[AlanZ]

Same here,

Another question: Will the JMX/PMI extension editor be deprecated as well ? Anything planned to browse processes JMX and autocreate EF2.0 extensions ?

[Mike_L]

Yes, the JMX/PMI wizard will be deprecated as well. What we are looking to do instead is to expose the JMX class browser over API and integrate it into our VScode addon for extensions. This will enable automatic creation of EF2 JMX extensions just like we do for Prometheus for example.

It is not yet certain that a PMI data source will be created for extension framework 2.0. This is currently being discussed based on PMI being deprecated for quite some time.

[michal_nalezin]

We have reassessed the approach to PMI extensions and revisited our initial decision on deprecating them.
Current plan is to deliver EF2.0 datasource for PMI in CQ4, and allow reasonable time for customers to migrate their configurations.
See also: https://community.dynatrace.com/t5/Extensions/PMI-extensions-migration-to-EF2-0/m-p/234324 

[TorstenHellwig]

We already escalated this on behalf of a large and heavily regulated customer. No chance to get through approvals this fast. 

Why is this so urgent now? We were asking for months for EF2.0 python data source and now we have like 5 months??

[michal_nalezin]

Hi Thorsten,

The information on removing EF1.0 was out in November'23. That was the moment when most of partners and customers notified us about the aggressive timeline of 12 months.

Meanwhile, Dynatrace was releasing Python data source related items since December. We have provided dedicated trainings for partners, early access, VScode addon, etc.

If we can support you more, let's connect directly.
Best regards
Michal Nalezinski

[TorstenHellwig]

Hi Michal,

the notification that Python extensions "will be available" around perform didn't make me start converting extensions as I couldn't participate in the EAP. So the clock startet around March where we could get a pre-prod environment of the customer enabled for Python ext. 2.0.

But it won't help getting test and approval processes sped up - not in our hands and the customer is certainly not risking incompliance with banking authorities for our plugins. Also I can't just quickly onboard any member of the Services team of Dynatrace and try to make them help with development and testing...

My guess is that the customer will move plugin functionality to systems monitoring tools and away from Dynatrace...

kind regards,

torsten 

[pahofmann]

Is there an ETA when the automatic migration for Python extensions will be finished?

[david_lopes]

Automating part of the process is not “automatic migration”, we are working to help as much as possible, including things like:

• converting plugin.json (done)
• migrating dependencies to setup.py
• Setting extension metadata on extension.yaml
• copying your source code files to the extension directory

but migrating the actual lines of code is not something that we will do, this will require manual intervention because of the nature of software development and the change in concepts like MINT metrics with Topology vs Custom Devices

We are looking to add some of these extra features in the upcoming weeks