Is the temporary storage acting like an extra RAM ?

Question

Hello,

I need to use a VM for a software (OrcaWave) which makes expansive calculations in terms of RAM. After setting the different parameters of this software, it gives me a value: the estimated peak memory per thread in GiB.

In OrcaWave, I can set the number of threads used in calculation, which has to satisfy the following relation:

Number of threads used mutiplied by Estimated peak memory per thread < RAM available on the computer (1)

If the relation is not satisfied, all the RAM is used, and OrcaWave starts to use hard disk and temporary storage, but if all disks are too much solicited, I have an Out of memory error and the calculation is canceled: to avoid it I have to reduce the number of threads.

Since I plan to run simulation with an estimated peak memory of at least 64GiB, I need lot of RAM and not that much threads. That is why I think it is better to choose VM with optimized memory like E16dv4 for instance.
The problem is that not all my threads will be used if I want to satisfy (1). Indeed E16dv4 has 128 GiB of RAM so I could use at maximum 2 threads to avoid out of memory or maybe 3 threads if the temporary storage is sufficient.

According to a developper of the software I asked to, when a hard disk (with an OS in it) is used there is a paging effect that decrease the run time, but he does not know if it is also the case for temporary storage.

My main question is: is it better to limit the number of threads to use only the RAM or to increase it a little to use paging and to avoid out of memory error ?

This could help to know whether it is more profitable to use Temporary storage or not (price are less expansive without). And in addition whether temporary storage is equivalent to an extra RAM of not.

Thank you,

Florian

Answer 1

Hi @BOURDREL Florian ,

The Azure Temporary Storage Disk by default should hold the OS Pagefile, if it isn't set to the Temp Drive then we suggest moving it there - The Pagefile is used for offloading low priority/least used memory pages to fetch later. If your application relies too heavily on the Pagefile then this can cause performance issues and isn't advisable unless you're okay with slower object/page fetch times.

I don't think there's a 'best' solution, as it depends on your expected outcome and the application's fetch requirements. You may want to test the environment with a large Pagefile to review if you're okay with the performance and fetch speeds needed for your application.