The Electronic Structure Validation and Verification (ESVV) Project proposes to collect and post results of electronic structure calculations obtained with various codes for a set of benchmark problems with the aim of achieving the following goals:
- Make results of electronic structure calculations widely available
- Establish and discuss consistency of results obtained with various codes
- Analyze differences observed between codes and methods and identify the reasons for the differences observed.
- Provide validation data for specific benchmarks.
The Project is hosted by CECAM Headquarters. Project administrators include the CECAM Director and a Project Leader assisted by Project staff members. Benchmark data is made available on the CECAM web site.
The Project goals include both Verification and Validation activities, which we define as follows for the purpose of the Project:
- Compare results obtained with different codes for a given benchmark. No comparison with experiment is made. The input given for the benchmark includes a set of physical quantities that uniquely specify the benchmark. The output of a benchmark test must contain all physical properties needed to reproduce the calculation as well as the complete set of source files needed to build the relevant version of all codes used in the calculations. The benchmarks are divided into categories according to basis set used, so that comparison of results within one category can reflect numerical errors only.
Examples of Verification activities:
- Compare results obtained with different plane-wave, pseudopotential codes.
- Compare results obtained with different finite-difference codes.
- Compare results obtained with different pseudopotentials.
- Establish the validity of electronic structure approximations such as treatment of exchange and correlation.
- Discuss the current accepted understanding of the differences observed.
Examples of Validation activities:
- Compare reference results for a given XC functional with experiment.
The Project will first focus on the Verification effort, which logically precedes any Validation activity. Benchmarks will be defined by the Project administrators and posted on the Project web site. Some benchmark results will be computed by Project staff in collaboration with the corresponding code development teams contributing the results.
Contributions of benchmark results will be accepted and posted once the Project staff succeeds in reproducing the result. In the first stage, benchmark problems sizes should be compatible with solving the problem on a Linux workstation. Larger problem sizes will by added at a later time.
What the project does not do
The project generally avoids “ranking” of codes by any figure of merit. This evaluation is left to users of the project web site. This policy is intended to encourage contributions from many code developers and avoid any perception of promotion of a code or set of codes by the Project.
The Project does not try to establish a standard for exchanging information about electronic structure data. It is however anticipated that the data gathering process and the comparison process will provide useful information for a future development of common data standards.