Test Suite for VASP

A set of calculations to validate your VASP installation

A collection of VASP calculations that can be used with the Behave testing framework for Python to validate a VASP installation. There are currently five test suites:

basic: a small test suite with simple calculations that completes quickly.
geometry: various kinds of geometry optimization of Si cell.
custom: tests for the presence of custom modifications made at NSC/SNIC sites.
functional: test more advanced features like GW, hybrid calculations, and DOS plots.
production: assorted troublesome production jobs that can run on more than one compute node.

Please note that these are far from complete and do not cover all of VASP’s functionality. For example, there are currently no tests for molecular dynamics simulations.

The actual tests are coded into the *.feature files in the feature/ folder. There are some more descriptions there about what each test is doing. The actual test input may be found in the input/ folder.

Version history

Version 4

Ported the test suite to use Python Behave testing framework.
Added tests of G0W0, DFT-hybrid calculations, and DOS generation.

Version 3

Never released to the public. Added the “custom” tests to check for NSC/SNIC site-local modification.

Version 2

Updated calculations to use the latest PBE POTCARs (released in April 2012)
Included more robust checks of POTCAR files used.
Removed some redundant test cases.

Version 1

Initial release.

Requirements

Python 2.7
Python Behave
Unix-like environment
The new VASP PBE POTCAR library (released in April 2012)

How to install

The source for test suite is available on GitHub:

git clone https://github.com/egplar/vasptest

Download and extract the vasptest folder to a suitable place. (Or use git clone.)
Locate the path to your VASP PBE POTCAR library (the folder named “potpaw_PBE.52”). If you do not have the full POTCAR library, any licensed VASP user can download the file “potpaw_PBE.52.tar.gz” from the VASP community web.
Run the script build-potcar.sh with the path to the POTCAR library as argument. E.g.:

source build-potcar.sh /opt/vasp/potpaw_PBE.52/
The test suite is now ready to use. See below.

How to run

Cd to the directory containing the VASP test suite. Before running, you need to tell where your VASP binary is located that you want to test. This is done by setting the VASPTEXT_EXE environment variable.

For serial VASP:

export VASPTEST_EXE=/path/to/vasp

For parallel VASP, insert your MPI launcher command and give the maximum number of ranks that you want to run on. The test runner will automatically scale down smaller jobs. E.g. for Intel MPI with maximum 64 cores:

export VASPTEST_EXE="mpiexec.hydra -np 64 /path/to/vasp"

Run the test suite via the behave command.

behave -s -c -k

When you run the test, output will given in the following form

...
Scenario: Si-hybrid 
    Given that the POTCAR MD5 is 4e058592231fc4e091ac0c92d87797b0
    When I run VASP with a maximum of 8 ranks
    Then the total energy should be -41.86884258 +/- 1.0e-5 eV
    And self consistency should be reached in 14 iterations
    And the point group symmetry should be T_d
    And the XML output should be valid
...

Unless something goes wrong, this is what you should see. There is one check or test per line written, and if something is found, a Python assertion will be triggered, and usually some indication of what was wrong also gets written out.

Scenario: Si-hybrid 
    Given that the POTCAR MD5 is 4e058592231fc4e091ac0c92d87797b0
    When I run VASP with a maximum of 8 ranks
    Then the total energy should be -41.86884258 +/- 1.0e-5 eV
      Assertion Failed: got -41.82492826 eV instead

All output is also logged to a file called vasptest.log by default (the name can be changed in behave.ini). The input and output from the actual calculations can be found in the tmp/ directory for inspection. It is a good idea to manually empty this folder before each invocation of the test suite.

To select individual test suites, you can give the direct path to a feature definition file:

behave -s -c -k features/basic.feature

The behave program has many features for controlling test execution and reporting. More information can be found either in the web documentation or by invoking the help flag:

behave -h

Test cases

There are currently 5 test suites, totaling 21 test calculations. The tests are intended to be run with the commonly used -DNGZhalf/-DNGXhalf-compiled version of VASP. In theory, the “full” version of VASP with complex-to-complex FFTs should give identical output, but small differences have been observed.

Quick suite

Fe in bcc structure (spin-polarized)
Cu in fcc structure (no spin)
Si in cubic diamond structure
TiO2 in rutile structure

These are quick calculations of simple periodic systems with high symmetry. All these should pass in well-behaved installations. To ensure that these are sane calculations, the results have also been cross-checked against previously published results in the literature, e.g. it is reasonable to expect a magnetic moment of 2.21 with DFT for the included structure of Fe(bcc).

I would like to thank Cecilia Århammar and Biplab Sanyal from the division of materials theory in Uppsala for help with collecting data for this test suite.

Geoopt suite

Performs several kinds of geometry optimizations on Si (cubic diamond).

Optimize direct coordinates only
Only volume relaxation
Only cell shape relaxation
Everything, all degrees of freedom.
Optimize direct coordinates using conjugate gradient algorithm.

Custom suite

Tests for local modifications that we do at NSC and PDC in Sweden.

Check that the path to the vdW kernel file is hard-coded and available.
Check that VASP was compiled -DLONGCHAR to enable long input lines in the POSCAR file, e.g. MAGMOM.
Check that -DnoSTOPCAR was used, this helps a lot when running on parallel file systems.

Functional suite

Some tests of more “advanced” features beyond regular DFT total energy calculations.

Silicon G0W0 calculation
Silicon hybrid-DFT calculation
Regression test to check that generation of PARCHG files work. It can crash with high optimization.
Generate DOS-plot for silicon.

Production suite

A collection of more realistic calculations, which are also longer running. It takes about 3-4 hours to complete in serial mode on a modern CPU. Many of the test cases have displayed problems with different compilations of VASP. Some simple parameter checks are performed to ensure that results are invariant to e.g. changes in NPAR. Hopefully (?), this collection will grow over time.

Mg2Mo6S8 (16 ions / 60 bands / 32 kpts) NPAR = 2
PbO2 (12 ions / 68 bands / 27 kpts)
SO3 molecule surrounded by vacuum (gamma point only)
Li2FeSiO4 (64 ions / 233 bands / gamma point)
CeO2 (108 ions / 528 bands / gamma point)

What is lacking in the test suite

This test suite is ongoing work, the following types of calculations should probably be included in the future for better coverage:

Molecular dynamics
Non-collinear/SO calculations