Hi, I know I have asked this before but I am really looking to get some useful information on this. So my organisation is currently using splunk across all platforms and they have recently purchased Dynatrace license and trying to move to Dynatrace for end to end applications.
They are not sure if they want to get rid of Splunk and move completely into Dynatrace. I believe both tools compliment each other. BUT i am really struggling to find the use cases where it would be beneficial to have both of them.
If any expert can shed some light on this which I can present to my stakeholders that would be very helpful. They are basically looking to understand what benifits we would get if we use both of them together.
I have 2 applications where both of the tools are installed so how can i present it practically by showing benifits where they compliment each other.
Any help is appreciated.
Solved! Go to Solution.
Well... what are you using Splunk for would be the first question.
DT at this point is more than just an APM. Not gonna lie, APM is the core of DT and in the market, one of the best (well actually the best from my point of view), but also got DEM / AiOps and a bunch of other stuff.
In any case, I would ask what are you doing with Splunk right now.
And if they bought DT, Why? Remember that Splunk bought an APM solution and added it to their SplunkFX (I think it's called) and is using OpenTelemetry for tracing... but what you got? Trace logs and Metrics? Does it have context/Causation? Using Opentelemetry is nice, but is adding work to Developers for things that you can do with an out of the box solution and ready for scale without effort or additional work.
With DT you have a single application that handles that and a lot more (https://www.dynatrace.com/platform/)
@Dante P. Thank you for your response. To answer your question, we use Splunk for creating dashboards, reports, alerts monitoring. We also collect some of the metrics data for some applications.
Now to answer why they bought DT, I would say they want to enable end to end monitoring at single place but they are not sure if they can get rid of Splunk completely because Splunk does a lot of things like creating custom viz in Dynatrace is very difficult while in Splunk it's easy.
I have seen at lot of places it says both tools are different and they complement each other but i don't know how do i use them together to make better monitoring.
What would be the use cases for that? What will i achieve if i integrate them together using a plugin or add on?
So Splunk would be like your main alarm console and for Dashboard/reports then, like a NOC with Dashboards/alarms for Ops and Dash/reports for Business/Managment.
The only use case, as you said, would be forwarding DT to Splunk via Splunk add-ons and keep using what you guys know. (https://answers.dynatrace.com/questions/217422/splunk-integration-with-dynatrace-saas.html for reference) using Splunk as the only console for problems and metrics visualization that also cover things outside DT Platform... things that have been customized to send to Splunk outside of Splunk's normal workflow.
But the same metrics that are currently in Splunk might be able to send it to DT via API metric ingestion. Part of the Metric 2.0 (https://www.dynatrace.com/support/help/dynatrace-api/environment-api/metric-v2/post-ingest-metrics/ Recent Video explaining: https://www.youtube.com/watch?v=5F-7fVbsPdA).
Also at the moment that you would want to troubleshoot, or see why Davis marks a component as Root Cause, you would need to jump to DT Console. So Ops/Devs or whoever is gonna troubleshoot/inform teams might have to learn DT anyway, so why not cut the middle man?
What kind of Viz are you working with Splunk that seems easy to create? DT has heavily invested in Metric these past releases, from my Point of view dashboards are still something that needs heavy investing from DT part. But now with the metric explorer and metric browser is a lot easier to chart those. What kind of viz you guys are using that seems difficult to create via DT?
@Dante P. In Splunk we create lots of visualisations and charts for business like number of orders being placed, number of customers being excluded from seeing offers, customer journey traffic and bunch of other stuffs like behaviour of web services and API's, top products sold, top products viewed, cpu usages, predicting CPU usages etc. etc.
I don't think Dynatrace gives you out of the box functionality to create charts as per your need. The customisation if a bit tricky in DT while in Splunk it is just creation of queries and then you can put them on charts as per your like.
I've also looked into forwarding Dynatrace logs in Splunk but then what would be the use of it? What would i achieve in that case as the costing will be required to ingest the logs in Splunk?
Also the other way round If i send the metrics currently in Splyunk to DT, will that give me anything extra special which is not already available in DT?
That's where I am bit confused. I know both tools are great and have their own advantages but looks like my stakeholders have bought them both and now I have to present them use cases where both will be useful and will be complimenting each other
I see... yeah, right now build some kind of query is not possible, unless we are speaking for USQL. Possible might be here in the future. (https://answers.dynatrace.com/spaces/483/dynatrace-product-ideas/idea/245941/formulas-on-metrics.htm...) but not right now.. so if that is a must, right there is the fundamental for keep using Splunk at least for Dashboards.
Second, I would suggest checking if the metrics that are in Splunk are really the ones in DT. Checking something like (https://www.dynatrace.com/support/help/shortlink/api-metrics2-available-managed#host) host metrics.
If you need to create the use case to show to stakeholders. I would just integrate Splunk and DT and start dashboarding with complex queries all the additional metrics of APM/DEM that DT provides... service calls, database calls.. etc... anything of the value that they wanted to gather from e2e tracing. You can leverage the Smartscape API (https://www.dynatrace.com/support/help/dynatrace-api/environment-api/topology-and-smartscape/) to bring the topology to Splunk and also work with that (Just understand that not all the data from DT is possible to extract. Somethings need to be used from DT console).
Then for Each dashboard, I would also build a markdown that allows the user to Jump to DT console to do the deep troubleshooting, or jump inside Splunk to the DT component and later to DT console for further analysis.
Now if you see that the trend is not building Complex Querys, I would just then try to build them inside DT.
Also having Logs and Metrics outside of the OOTB from DT, if those metrics are attached to an entity then Davis (DT Ai Engine) will use that data to contextualize issues. The Extra is that, if that information is not gathered ootb BY DT, you can ingest it allowing you to perform RCA with those additional metrics. In Splunk.. those metrics are just that... metrics without context.
I will add this... If you own both Dynatrace and Splunk and are talking about only log monitoring, stick with Splunk for that. Dynatrace changed their approach to log monitoring a while back and currently we feel it's a double dip licensing charge compared to Splunk.
The reason being is that Dynatrace forces you to upload/store logs you want to monitor now rather than continuing to have an option to process them locally like they used to. Many have asked to bring back that ability as an option, but to date I have not seen anything that leads me to think they will.
So with Dynatrace you are charged for the storage and charged for the monitoring VS Splunk which only charges for consumption and nothing more. Keep in mind, I am strictly talking about just log monitoring here.
I think that was a move that was not well thought out ahead of time when they did it and hope it changes to be more competitive to Splunk licensing at some point.
Outside of that, I would also really think through your strategy to ensure you are getting the most bang for what you spend on licensing with both Dynatrace and Splunk.
Remember that one vendor package taking in metrics is costing you in terms of licensing. Sending those very same metrics over to yet another vendor package is an additional cost in licensing meaning you start to pay double for the same metric.
Metric tracked in Vendor A = licensing consumption of some kind by Vendor A
Forwarding metric in Vendor A to Vendor B = licensing consumption of some kind by Vendor B
One more thing, Splunk is also a great option to store your Dynatrace audit logs.
Both are top products with each having specific key features that they shine in. Obviously Dynatrace with end-to-end is what stands out as it should. It all really boils down to how you use both products at the end of the day.
Thanks @Dante P. for your detailed explanation. It really helped me visualise and put together few slides to show to my stakeholder.
I think this is enough information to make a fair decision what they would want to use and how.
Thanks again for all the help 🙂
@Dante P. @Larry R. @Chad T. I've looked into the DT Splunk integration and there is an add-on and app using which we can ingest the data in Splunk.
But I am trying to figure out what kind of Data dynatrace logs and what can i send it to Splunk which can be useful and not already available in Splunk. Like user session data or anything else?
Has anyone tried the new BizOps with the Dashboard Powerups? How does it compare to ITSI?
I have! the power ups are cool and really unique. But i did run into issues where the tab would become unresponsive. The other draw back is that if you dont have the powerup plugin installed on your browser, then the title of your tiles are odd and hard to read.
I see some really nice answers. I'll share just a few words, then. My purpose for keeping the two (integrated) is a use case in which I send metrics/problems from Dynatrace to Splunk - that's from my humble work experience. How did you make the connection between the two systems? Did you use any external tool?