Science News
from research organizations

Reliability of material simulations put to test

April 25, 2016
Aalto University
New generations of quantum mechanical simulation codes agree better than earlier generations, scientists have demonstrated.

Researchers show that new generations of quantum mechanical simulation codes agree better than earlier generations'. The study appears in Science.

Several international scientists from over 30 universities and institutes teamed to investigate to what extent quantum simulations of material properties agree when they are performed by different researchers and with different software. Torbjörn Björkman from Åbo Akademi participated from Finland. Björkman has previously worked at COMP Centre of Excellende at Aalto University. "A group of researchers compared the codes, and the results we got were more precise than in any other calculations before," he said

The possibility to produce identical results in independent yet identical researches is a corner stone of science. Only in this way science can identify 'laws', which lead to new insights and new technologies. However, several recent studies have pointed out that such reproducibility does not always come spontaneously. Even predictions by computer codes require caution, since the way in which theoretical models are implemented may affect simulation results.

The research team can now demonstrate that, although a few of the older methods clearly yield deviating results, predictions by recent codes are entirely equivalent. They also define a quality criterion that allows the verification of future software developments against their extensive database. New test data are continuously added to a publicly available website ( The researchers involved hope that their work will contribute to higher standards for materials property simulations, and that it will facilitate the development of improved simulation codes and methods.

The researchers investigated 40 different methods to describe the influence of pressure in 71 different crystals. Due to the highly international composition of the team, discussions and collaboration were mainly conducted via online tools -- similarly to the way people collaborate to write Wikipedia.

The reproducibility is a concern in any field of research that critically depends on computer simulations. For instance, in the study and design of materials there are several independent software packages available, based on quantum physics. They are moreover being used increasingly often in automated procedures with limited human supervision which increases the need for more reliable codes.

Story Source:

Materials provided by Aalto University. Note: Content may be edited for style and length.

Journal Reference:

  1. K. Lejaeghere, G. Bihlmayer, T. Bjorkman, P. Blaha, S. Blugel, V. Blum, D. Caliste, I. E. Castelli, S. J. Clark, A. Dal Corso, S. de Gironcoli, T. Deutsch, J. K. Dewhurst, I. Di Marco, C. Draxl, M. Du ak, O. Eriksson, J. A. Flores-Livas, K. F. Garrity, L. Genovese, P. Giannozzi, M. Giantomassi, S. Goedecker, X. Gonze, O. Granas, E. K. U. Gross, A. Gulans, F. Gygi, D. R. Hamann, P. J. Hasnip, N. A. W. Holzwarth, D. Iu an, D. B. Jochym, F. Jollet, D. Jones, G. Kresse, K. Koepernik, E. Kucukbenli, Y. O. Kvashnin, I. L. M. Locht, S. Lubeck, M. Marsman, N. Marzari, U. Nitzsche, L. Nordstrom, T. Ozaki, L. Paulatto, C. J. Pickard, W. Poelmans, M. I. J. Probert, K. Refson, M. Richter, G.-M. Rignanese, S. Saha, M. Scheffler, M. Schlipf, K. Schwarz, S. Sharma, F. Tavazza, P. Thunstrom, A. Tkatchenko, M. Torrent, D. Vanderbilt, M. J. van Setten, V. Van Speybroeck, J. M. Wills, J. R. Yates, G.-X. Zhang, S. Cottenier. Reproducibility in density functional theory calculations of solids. Science, 2016; 351 (6280): aad3000 DOI: 10.1126/science.aad3000

Cite This Page:

Aalto University. "Reliability of material simulations put to test." ScienceDaily. ScienceDaily, 25 April 2016. <>.
Aalto University. (2016, April 25). Reliability of material simulations put to test. ScienceDaily. Retrieved May 23, 2017 from
Aalto University. "Reliability of material simulations put to test." ScienceDaily. (accessed May 23, 2017).