What is the RAM usage when using -disk-computing flag ?

[dusadvarshit]

Hi team,

I was running a phylogenetic tree construction analysis of about 70K sequences between range of 200-800 amino acids.
My server system has 32 GB RAM and 8 cores. Originally the system keep crashing - so I moved to using -disk-computing flag - but now I notice only 6GB of RAM.

I was wondering if there is a way for me to max my RAM usage and only off-load the extra calculations on disk to speed up the operations.

Thanks!

[cesarpomar]

Hi,

When you enable --disk-computing, the ~6 GB of RAM you’re seeing in use corresponds only to the portions of the workflow that are not handled through disk-based computation. Everything that is being off-loaded to disk will still take advantage of the remaining system memory, potentially up to the full amount of RAM, but in a way that doesn’t show up as "used" RAM in monitoring tools. This cached memory is shown as "available" rather than "used" RAM, since the kernel can reclaim it at any moment for other processes if needed. Some tools, such as htop, represent these categories with different colors: green = used memory and yellow/orange = cache, which helps clarify the difference between actual application memory usage and memory being used as filesystem cache.

So even though your monitoring tools only show around 6 GB of RAM in use, the rest of your memory is being utilized transparently by the OS to support Veryfasttree’s disk-based computation. When Veryfasttree needs more memory than what is immediately available, the kernel uses file-backed pages (via the page cache and mmap) to move some memory pages to disk, making space for new data while still keeping frequently accessed regions cached in RAM.

Let me know if you have any other questions