topic How to fully utilize disk space in Dynatrace Managed Q&A

How to fully utilize disk space

gary1024 — Tue, 30 Aug 2022 10:31:09 GMT

Hi, I need help on how to configure my Dynatrace to use more of the disk space instead of leaving them free. Here's what I facing: 1. while my filesystem is only at 92% used /dev/mapper/APPVG-app_lv1 4.8T 4.2T 390G 92% /apps 2. my environment is complaining "Transaction storage has been truncated due to quota limits or there is not enough space available on the disk " 3. here's what I have in /apps 49G agents 797M backup 172G cassandra 28G elasticsearch 784M installer 1.3G log 4.0K nodekeeper 3.9T server Is there a way to make Dynatrace use another 5% of my available space? Thanks for helping in advance... Gary

Re: How to fully utilize disk space

gary1024 — Mon, 22 Jun 2020 02:27:50 GMT

Forgot to mention that I already set 'Unlimited' 'Transaction storage' in CMC. It just won't go beyond 92%.

Re: How to fully utilize disk space

ChadTurner — Mon, 22 Jun 2020 03:58:46 GMT

I don’t think so, the system sets certain amounts of space for certain parts of the application, I would recommend growing them or using Network attached storage via mount points.

Re: How to fully utilize disk space

Radoslaw_Szulgo — Mon, 22 Jun 2020 08:13:28 GMT

Gary,

the reason behind this is the "reserved" space by Dynatrace. It might be observed as free, whereas this is a space for our metrics and configuration storage (Cassandra). The space is utilized by the compaction.

Cassandra uses a Log Structured Merge Tree engine, which allows high write throughput by flushing immutable chunks of data, in the form of SSTables, to disk and deferring consistency on the read phase. Over time, more and more SSTables are written to disk, resulting in a partition having chunks in multiple SSTables, slowing down reads. To limit fragmentation of data, we use a process called compaction to merge sstables together. Several compaction strategies are available in Cassandra that merge SSTables together. These strategies are designed for different workloads and data models. (source: https://thelastpickle.com/blog/2016/12/08/TWCS-part1.html)

Currently Dynatrace Managed requires 100% of metric storage reserved for compaction. E.g for 2 TB metric storage, you need up to 2TB for compaction. We are working now on switching the compaction strategy to TWCS which should drop reserved space significantly.

Here you can find a nice article on compaction nuances: https://thelastpickle.com/blog/2017/03/16/compaction-nuance.html