Scraping data from Dynatrace documentation

[kadirhan]

Hello ,

Can we pull data from Dynatrace documents up to date (web scraping)?

Is this type of operation allowed? Does it create any problems? I'm thinking of analyzing the information found in the document page in json format. It can be done by writing a simple code along with things like Python web scraping, but I was wondering if this information is available in a json format etc.? Is there an easier way to get this data?

https://www.dynatrace.com/support/help

Kind regards
Kadirhan Cekmez

[AntonioSousa]

@kadirhan,

Have you checked the Dynatrace APIs? They should have the information you need...

[kadirhan]

Hello ,

Thanks for your answer , But I know that we can pull our own environment data using Api . Also, is there an API where we can pull the Dynatrace document page?
I'm talking about the data on the Dynatrace Document page, not our own environment data.

https://www.dynatrace.com/support/help/get-started

Is there a different way to pull the data from the Document page?

Thanks.

[AntonioSousa]

@kadirhan,

OK, I get the idea 🤣

If it's not for something like academia, you don't have to reinvent the wheel. Use Bing (quick example below):

[Taylor-Sanchez]

Hi @kadirhan ,

As @AntonioSousa mentioned, you are able to get information about your environment via the Dynatrace API as linked here:
https://www.dynatrace.com/support/help/dynatrace-api

The information that is returned is in JSON format as you mentioned.

However, in terms of your question, I've noticed that you specify the Dynatrace documents. Do you mean the documentation page or the data collected in your environment? 

[kadirhan]

Hello ,

Thanks for your answer , But I know that we can pull our own environment data using Api . Also, is there an API where we can pull the Dynatrace document page?
I'm talking about the data on the Dynatrace Document page, not our own environment data.

https://www.dynatrace.com/support/help/get-started

Is there a different way to pull the data from the Document page?

Thanks.

[adham_sabry]

Hello Kadirhan, this is not possible. At least, not that I know of.

As far as I understood, please correct me if I am mistaken, you have a chat bot (a GPT-like) and you want to feed it our Dynatrace documentation so that it can answer you.

So, you are looking for an API to hand-deliver you our documentation in an easy format for the chat-bot to understand, unfortunately, we do not have this. So as previously mentioned, please raise a new feature request in Product ideas - Dynatrace Community.

The currently available solution for you is to scrap our documentation, in terms of staying up to date, you have feed.xml as previously provided.

Hope this helps.

[adham_sabry · Accepted solution]

Hello Kadirhan,

Can you please provide more details on what you want to achieve? And example JSON format that you are looking for? Is it that you want our HTML tags to be in JSON format?

In terms of scrapping, you might want to just simply use https://www.dynatrace.com/support/help/feed.xml to only scrap for whats new, considering that this is in XML format, but it shouldn't be difficult to convert it to JSON on your end. If you want us to support JSON Feed, I'd suggest you to please raise a new feature request in Product ideas - Dynatrace Community.

Hope this helps.

Regards,

Adham

[kadirhan]

Hello ,

Thanks for your answer, the link https://www.dynatrace.com/support/help/feed.xml seems to be enough for me. Is it possible to shoot all the contents of the titles in the document via this link?

There is a project I want to do, I want to develop a chatbot with the help of a chatgpt using Dynatrace document data. For this, I need a structure to keep up-to-date document data and to classify all of these data as title - content.

Thanks for support .

[kadirhan]

Hello ,

I see only a few titles and content from the RSS feed, I want to get all the content from the documentr page. is there an easier and different way to do this?

Thanks

[adham_sabry]

No, you will have to scrap the documentation then rely on feed for whats new.