December 14, 2018 by @BillSchenk
SESYNC’s Windows virtual machines are setup to use dynamic memory. What this means is that your virtual machine will show different memory usage based on its current usage, however, you will still have access to the full amount we allocated to you.
For example, SESYNC may allocate up to 64GB for your virtual machine, however, when you connect to your machine you may only see a fraction of that in the Windows task manager. When your processing starts and requests memory, your allocation will automatically increase up to the limit that we’ve allocated for you.
In a nutshell, it’s to more efficiently use our resources.
We’ve found that most virtual machines we allocate are not used 24 hours a day, 7 days a week, but rather they tend to be used intensly for a few days or weeks and are idle for the following weeks or even months. By using dynamic memory we are able to automatically reallocate unused resources from idle virtual machines to busy ones. If we didn’t use dynamic memory, during this idle time these virtual machines would just be occupying memory that may be needed by other groups. This wastes the memory itself and power needed to keep the memory online.
Theoretically if every virtual machine requested to use its maximum allowed memory at the same time this could happen. In practice this doesn’t happen (and never has) because SESYNC staff closely monitors our virtual cluster resources and overall usage to ensure that there are adqeuite resources available at all times. In addition, VM’s are able to migrate to servers that have available resources.
We have noticed that some applications (R specifically) will look at the computer memory available upon startup and refuse to allocate more than that amount.