Support for Azure Databricks Data Factory

[alberto-jesus_1]

Hi,

is there any insight available on Dynatrace brought from Azure Monitor related to Azure Databricks and Data Factory? Do you have any experience getting this info from Azure?

Thanks in advance.

Alberto.

[Karolina_Ruszko]

Hi Alberto,

Not yet, however support for Azure Data Factories should be available very soon. In fact this service is under active development right now.

Best,

Karolina

[luis_mpa]

Hi Karolina R. What is the status of support for Azure Databricks? Does Dynatrace plan to monitor Azure Databricks?

[sia_h · Accepted solution]

Hi Luis,

Azure Data Factory monitoring is available as part of the Azure Monitor integration:

https://www.dynatrace.com/support/help/technology-support/cloud-platforms/microsoft-azure-services/s...

Sia

[luis_mpa]

Hi @Siavash H.., thank you for your reply. But what I need to know is if Dynatrace has planned to monitor Azure Databricks. And in that case, when would it be available and what can we expect from such monitoring (what aspects of Databricks would it monitor?).

Thank you, good day!.

[sia_h]

Hi Luis,

I am unable to find anything on the roadmap for Azure Databricks.

However, I had a look on the Azure documentation and it mentioned the ability to send Azure Databricks application logs using Log4j. In the new Log Monitor, Dynatrace offers generic log ingestion and log4j2 can be easily integrated directly to stream log directly from Databricks to Dynatrace.

I hope this helps. If you're looking for another ability to monitor Databricks I recommend to raise a product idea when the new Community is launched.

Sia

[luis_mpa]

Interesting, @Siavash H. Thanks a lot.

Can you share with me documentation about the new Log Monitor? I'm really interested in this new functionality.

Nice weekend, regards!

[sia_h]

Thanks, and you too Luis!

With regard to the new Log Monitor capabilities I think we'll announce something very soon.

Sia

[apasoquen1]

Any updates as to the new log monitor capabilities for Azure Databricks?

[CMolina]

Hello everyone, i need to bring this topic back to life again, ¿any update?

@sia_h 

[wojciech_grajew]

@apasoquen1 @CMolina as far as I know, Azure Databricks as a supporting service is still not on the roadmap. However, when it comes to log monitoring, our recent solution should support it, please check: https://www.dynatrace.com/support/help/how-to-use-dynatrace/infrastructure-monitoring/cloud-platform...

[jose_damian_rui]

Many Thanks wojciech_grajew

Please any technical document about Dynatrace capacities on Databricks monitoring (currently) will very useful.

On the other hand Any updates about Dynatrace roadmap?

Many Thanks

Damián

Dynatracer

[wojciech_grajew]

Hello Damián,

Unfortunately right now I don't have any more documentation to share except for the link above. Regarding the roadmap, this service is not on it yet.

[jose_damian_rui]

Many Thanks wojciech_grajew for your feedback.

Please one additional question: my understanting is that Apache Spark platforms are supported by Dynatrace... so my question here are

- Dynatrace capacities for monitoring Apache Spark platforms are equivalents and available for Databricks or not ?

- Which are the monitoring capacities to be deployed by Dynatrace in order to consider a plataform (Databrick for instance) as "supporting service" ?  

Kind regards

Damián

[wojciech_grajew]

Hi Damián,

I'm not an expert on Azure Databricks unfortunately. From what I see, Dynatrace supports monitoring Apache Spark through OneAgent:
https://www.dynatrace.com/support/help/technology-support/dynatrace-extensions/supported-out-of-the-...

As Databricks is a cloud service, I'd expect it to be a managed offering where you don't have access to the underlying instances where you could install OneAgent. You'd need to research it though.

Regarding the second question -- I'm not sure I understand. Supporting services is the name we give to the cloud services that Dynatrace can monitor through their public cloud APIs (making requests to Azure Resource Graph and Azure Monitor for example). They are monitored from ActiveGate and don't require OneAgent installation. Right now we plan to implement more supporting services but I don't think Databricks has high priority in this queue:
https://www.dynatrace.com/support/help/how-to-use-dynatrace/infrastructure-monitoring/cloud-platform... 

[joseph_bannert · Accepted solution]

Hi alberto-jesus_1,

You can monitor the Azure Databricks cluster nodes using OneAgent.

Steps:
Navigate to the “Download Dynatrace OneAgent for Linux” screen in your Dynatrace environment and copy the 2 commands highlighted below to create a OneAgent install script.

Paste the commands into a file to save for later, let’s refer to it as installdynatraceagent.sh. Cut out the 2 places where the agent version is specified so the latest version will always be used. Your script should look like this:

wget -O Dynatrace-OneAgent-Linux.sh "https://<domain>/e/xxxxxxx/api/v1/deployment/installer/agent/unix/default/latest?arch=x86&flavor=default" --header="Authorization: Api-Token xxxxxxxx"

 /bin/sh Dynatrace-OneAgent-Linux.sh --set-infra-only=true --set-app-log-content-access=true --set-host-group=<host_group>

Navigate to your Azure Databricks workspace and go to the Admin Console.

Select Global Init Scripts and click add.

Paste the name installdynatraceagent.sh and script we created earlier, select enabled and click add.

Now your clusters will automatically install OneAgent when they are created, for the OneAgent to be installed on clusters already running they will need to be restarted for the script to run.

Done! You now should be able to see your Databricks cluster nodes as shown below in the deployment status.

Advisory notes: 

• Your script should include a host group for the nodes, then in Dynatrace you need to turn off "Detect host or monitoring connection lost problems" on the host group level, this is to avoid an alert storm caused by frequent node termination.
• OneAgent should be used in infra only mode to save license because Databricks nodes typically have high RAM and full stack mode does not give much extra value.

Regards, Joe.

[lawrence_cuneaz]

This is an awesome walkthrough for the OneAgent deployment to Databricks! We can also get deeper metric insight with various splittings via the Ganglia Extension as well for enhanced Datbricks AIOps from alerts and dashboard reports. The extension and instruction can be accessed from the in-product Dynatrace Hub or from the Public Dynatrace Hub. https://www.dynatrace.com/hub/detail/databricks/?query=databricks

[tanmoy]

I have followed the instructions for Databricks extension v1.0.4 and I am not seeing any Ganglia metrics either in dashboard or in Metrics page 

[lawrence_cuneaz]

All steps completed from the Details page, correct?

https://www.dynatrace.com/hub/detail/databricks/?query=databricks#details

[tanmoy]

Yes , the extension is configured in Dynatrace and GlobalInitscripts also updated in Databricks  as per instructions. The cluster is visible in Dynatrace with OneAgent installed but no metrics for ganglia showing up

[lawrence_cuneaz · Accepted solution]

Okay, then please open a support ticket https://www.dynatrace.com/support/contact-support/