Memory Usage FAQ



An Overview of Memory Issues in Hartree-Fock Calculations


There are two different algorithms that may be used by the Ab Initio program, depending on how much memory is available. The two algorithms are: Spartan does a good job choosing which memory module to use. One can override default choices by typing either the DIRECT or INCORE keyword in the Options field in the calculations dialogue. Chances are that if you force INCORE when Spartan doesn't choose this approach, the calculation will end up swapping RAM causing a VERY LARGE AND SIGNIFICANT slowdown! (See our discussion of swap space for more information.)

There is no way to make the DIRECT method use memory more efficiently. It is an all-or-nothing choice. A short table giving the memory requirements for the INCORE method can be found here

One can use the 'TESTMEMORY' keyword as a quick test to see which memory model will be used. No calculation will be done, but some memory information will be printed to the output file.

Setting Spartan's memory limits

Spartan checks the machine to see how much Memory is available. You can set the amount of memory a job can use from within Spartan. The place you do this is different on each platform. The default memory value is also unique for each type machine:

Memory Requirements for Frequency Calculations

Frequency calculations use significantly more RAM than a geometry optimization. The more RAM the better, and quicker the calculation of second derivatives.

Memory Requirements for Semi-Empirical Calculations

While Semi-Empirical calculations take much less memory than Abinitio calculations, however, it can add up for very large systems. As a rough approximation (only good to 1 digit)

memory usage in kB = 950*8*(N2)/1024

where N is the number of atoms.

If using the SM54 Cramer-Truhlar solvation method one can expect another 9*8*(N3)/1024 kB of memory allocated for the semi-empirical modification.



Memory Requirements for DFT Calculations

Memory is usually not a limiting factor in DFT calculations. Since we use a direct method for the exchange calculation. The biggest arrays are linear in the number of atoms, and quadratic with the number of basis functions.



Memory Requirements for post HF Calculations

While more memory is always better with the advanced (post HF) methods, the usual machine limits are the amount of available scratch disk space.

Return to Top

Common Questions

  1. How much memory do I need to run Spartan?

  2. A bare minimum is 256 MB, but 256-512 MB is a good range for modest sized molecules. More is always better, but after about 1 GB of RAM the benefits are harder to come by. The biggest advantage of more memory is that it allows Spartan to choose the faster INCORE methods for Hartree-Fock (HF) calculations. The table below gives some sample memory systems and lists the number of carbon atoms which can be calculated using the INCORE method for the 3-21G and the 6-31G* basis sets.

    Available
    Ram
    Basis
    functions
    3-21G
    Carbons
    6-31G*
    Carbons
    128 MB 107117
    512 MB 1511610
    2024 MB2002213

    Return to Top

  3. My DFT frequency calculation will not run, what can I do?

  4. In our current implementation of DFT analytical frequency we do not use memory wisely. (This is being corrected, changes will appear in future editions.) Currently, as we run out of memory we just crash (sometimes with a very cryptic message). The current workaround is to calculate the Hessian numerically using the FREQ=CD keyword. This will run slower, but will require less memory.
    Return to Top

  5. I'm using the default RAM settings, but Spartan seems to think I don't have much memory at all? (Windows Specific)

  6. If you don't set any of the Memory defaults the Spartan program will examine the computer to see how much memory is available and attempt to use it. (see above). If you only use your machine for running Spartan this is a good idea, but it can be misleading in many 'office' cases. For example if you have been surfing the web, the web browser may be taking memory that it may not really need, but helps it to run faster. (i.e. memory which can be swapped to disk.) It is possible to have a 512 MB machine with almost all the memory being used by other programs. In this case Spartan will attempt to use the minima of 32MB for scratch space. If you always want more memory available for Spartan you should set the default memory in the Monitor to a number larger than 32 MB but less than the amount of physical memory in your machine. (See Setting Spartan's memory limits above.)
    Return to Top

  7. Will increasing swap improve performance?

  8. Probably not. The most likely affect will be to slow down Spartan calculations. But if you know what you are doing, and are careful it might help.

    In the worse case scenario, you will increase your swap, and spartan will think your machine has more RAM than it really does. Spartan will then use this "virtual" ram. Spartan will run, but often the ram required by Spartan will be residing on the disk. Accessing the disk is many order of magnitudes slower than running in Ram, and the speed of Spartan will plummet. A similar problem can occur even if spartan is using less than the total RAM in your system. If a spartan job and another job such as a web browser, a java applet, or even another spartan job are both trying to use the real RAM a thrashing of memory will occur.

    There is a case where increasing swap space may help, but since it is so easy to do something wrong be careful. If there are some bloated applications on your machine which are running, but you are not using them, yet you choose not to exit them, they will take up valuable space in memory; Memory which spartan wants to use. The best thing to do is to quit these applications. But you can also increase swap in the hope that the OS will swap the applications you aren't using out of memory and let Spartan use the real RAM.

    When increasing swap it is very important to set the memory limits of spartan to be less than real Memory in your system. Typical modern operating systems require a bare minimum of 32-64 MB of RAM to run. So the maximum memory should be:

    ("Total Ram" - 32MB)/"Number of jobs running at the same time"


    Return to Top

  9. My job failed with an error about MEM_STATIC. What does this mean?

  10. In DFT calculations the algorithms used partition memory memory into two different regions. For discussion purposes let's call them 'STATIC' and 'HEAP' memory. There are times when the algorithm incorrectly predicts the amount of 'STATIC' memory. When this occurs you may get an error message in the verbose output asking for more memory. This bug can usually be fixed by adding the MEM_STATIC=xx keyword to the output where xx is in megabytes (MB). The default is usually on the order of 32 MB. It is important that this number is less than the total memory allocated in the machine. See the discussion on setting Spartan's memory limits.

    Note: This replaces the deprecated keyword MEMORY=yy where yy was in units of "double-words" (64-bits)
    Return to Top


Wavefunction Support
Author: Phil Klunzinger
Last modified: Tue Sep 12 13:09:05 PDT 2006