UniFFBench: evaluating universal machine learning force fields against experimental measurements

Abstract

Universal machine learning force fields (UMLFFs) promise to revolutionize materials science by enabling rapid atomistic simulations across the periodic table. However, their evaluation has been limited to computational benchmarks that may not reflect real-world performance. Here we introduce UniFFBench, a comprehensive evaluation framework featuring the MinX dataset—a diverse collection of 1,500+ mineral systems spanning 85 elements, extreme thermodynamic conditions (0–5,000 K, 0–1,000 GPa) and structural complexity, including partial occupancy and disorder. This diversity, combined with experimental reference values for validation, enables assessment of UMLFF generalization across chemical space and conditions substantially beyond typical training scenarios. Our systematic evaluation of six state-of-the-art UMLFFs reveals a substantial ‘reality gap’: models achieving impressive performance on computational benchmarks often fail when confronted with experimental complexity. Even the best-performing models exhibit higher density prediction error than the threshold required for practical applications. We observe disconnects between simulation stability and mechanical property accuracy, with prediction errors correlating with training data representation rather than the modeling method.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access

Access options

• Purchase on SpringerLink
• Instant access to the full article PDF.

Prices may be subject to local taxes which are calculated during checkout

Data availability

The crystal structure files for all mineral species are publicly accessible and can be downloaded from ref. 48.ary Data 2

Code availability

The UniFFBench framework code is available of the code corresponding to this Re165458 (ref. 49)

References

1. Duval, A. et al. A hitchhiker’s guide to geometric GNNs for 3D atomic systems. Preprint at https://arxiv.org/abs/2312.07511 (2023).

2. Bihani, V. et al. Egraffbench: evaluation of equivariant graph neural network force fields for atomistic simulations. Digit. Discov.3, 759–768 (2024).

3. Batatia, I. et al. A foundation model for atomistic materials chemistry. J. Chem. Phys.163, 184110 (2025).

4. Musaelian, A. et al. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun.14, 579 (2023).

5. Fu, X. et al. Forces are not enough: benchmark and critical evaluation for machine learning force fields with molecular simulations. Preprint at https://arxiv.org/abs/2210.07237 (2023).

6. Miret, S., Lee, K. L. K., Gonzales, C., Nassar, M. & Spellings, M. The Open MatSci ML Toolkit: a flexible framework for machine learning in materials science. Preprint at https://arxiv.org/abs/2210.17484 (2023).

7. Saal, J. E., Kirklin, S., Aykol, M., Meredig, B. & Wolverton, C. Materials design and discovery with high-throughput density functional theory: the Open Quantum Materials Database (OQMD). JOM65, 1501–1509 (2013).

8. Hautier, G., Jain, A. & Ong, S. P. From the computer to the laboratory: materials discovery and design using first-principles calculations. J. Mater. Sci.47, 7317–7340 (2012).

9. Merchant, A. et al. Scaling deep learning for materials discovery. Nature624, 80–85 (2023).

10. Axelrod, S. et al. Learning matter: materials design with machine learning and atomistic simulations. Acc. Mater. Res.3, 343–357 (2022).

11. Yuan, E. C.-Y. et al. Foundation models for atomistic simulation of chemistry and materials. Nat. Rev. Chem.10, 212–230 (2026).

12. Schmidt, J. et al. Improving machine-learning models in materials science through large datasets. Mater. Today Phys.48, 101560 (2024).

13. Lee, K. L. K. et al. MatSciML: a broad, multi-task benchmark for solid-state materials modeling. Preprint at https://arxiv.org/abs/2309.05934 (2023).

14. Deringer, V. L., Caro, M. A. & Csányi, G. Machine learning interatomic potentials as emerging tools for materials science. Adv. Mater.31, 1902765 (2019).

15. Friederich, P., Häse, F., Proppe, J., & Aspuru-Guzik, A. Machine-learned potentials for next-generation matter simulations. Nat. Mater.20, 750–761 (2021).

16. Zeni, C. et al. A generative model for inorganic materials design. Nature639, 624–632 (2025).

17. Deng, B. et al. CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nat. Mach. Intell.5, 1031–1041 (2023).

18. Chen, C. & Ong, S. P. A universal graph deep learning interatomic potential for the periodic table. Nat. Comput. Sci.2, 718–728 (2022).

19. Yang, H. et al. MatterSim: a deep learning atomistic model across elements, temperatures and pressures. Preprint at https://arxiv.org/abs/2405.04967 (2024).

20. Park, Y., Kim, J., Hwang, S. & Han, S. Scalable parallel algorithm for graph neural network interatomic potentials in molecular dynamics simulations. J. Chem. Theory Comput.20, 4857–4868 (2024).

21. Neumann, M. et al. ORB: a fast, scalable neural network potential. Preprint at https://arxiv.org/abs/2410.22570 (2024).

22. Barroso-Luque, L. et al. Open Materials 2024 (OMat24) inorganic materials dataset and models. Preprint at https://arxiv.org/abs/2410.12771 (2024).

23. Levine, D. S. et al. The Open Molecules 2025 (OMol25) dataset, evaluations, and models. Preprint at https://arxiv.org/abs/2505.08762 (2025).

24. Chanussot, L. et al. Open Catalyst 2020 (OC20) dataset and community challenges. ACS Catal.11, 6059–6072 (2021).

25. Riebesell, J. et al. A framework to evaluate machine learning crystal stability predictions. Nat. Mach. Intell.7, 836–847 (2025).

26. Miret, S., Lee, K. L. K., Gonzales, C., Mannan, S. & Krishnan, N. M. Energy & force regression on DFT trajectories is not enough for universal machine learning interatomic potentials. Preprint at https://arxiv.org/abs/2502.03660 (2025).

27. Wood, B. M. et al. UMA: a family of universal models for atoms. Adv. Neural Inf. Process. Syst.38, 129391–129427 (2026).

28. Chiang, Y. et al. MLIP arena: advancing fairness and transparency in machine learning interatomic potentialscess. Syst.38 (2026)

29. Fu, X. et al. Forces are not enough: benchmark and critical evaluation for machine learning force fields with molecular simulations. (2023).

30. Fu, X. et al. Learning smooth and expressive interatomic potentials for physical property prediction. Preprint at https://arxiv.org/abs/2502.12147 (2025).

31. Zaki, M., Mausam, J. & Krishnan, N. M. A. MaScQA: investigating materials science knowledge of large language models. Digit. Discov.3, 313–327 (2024).

32. Miret, S. & Krishnan, N. M. A. Enabling large language models for real-world materials discovery. Nat. Mach. Intell.7, 991–998 (2025).

33. Alampara, N. et al. Probing the limitations of multimodal language models for chemistry and materials research. Nat. Comput. Sci.5, 952–961 (2025).

34. Gonzales, C., Fuemmeler, E., Tadmor, E. B., Martiniani, S. & Miret, S. Benchmarking of universal machine learning interatomic potentials for structural relaxation. OpenReviewhttps://openreview.net/pdf?id=fNyXCCZ0g6 (2024).

35. Tran, R. et al. The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts. ACS Catal.13, 3066–3084 (2023).

36. Warford, T., Thiemann, F. L. & Csányi, G. Better without U: impact of selective Hubbard U correction on foundational MLIPs. Mach. Learn.7, 035033 (2026).

37. Gangan, A. S., Cubuk, E. D., Schoenholz, S. S., Bauchy, M. & Krishnan, N. M. A. Force-field optimization by end-to-end differentiable atomistic simulation. J. Chem. Theory Comput.21, 5867–5879 (2025).

38. Thaler, S. & Zavadlav, J. Learning neural network potentials from experimental data(2021)

39. Downs, R. T. & Hall-Wallace, M. The American Mineralogist Crystal Structure Database. Am. Mineral.88, 247–250 (2003).

40. Hahn, T., Shmueli, U. & Arthur, J. C. W. International Tables for Crystallography Vol. 1 (Reidel Dordrecht, 1983).

41. Burzlaff, H. & Zimmermann, H. Hermann–Mauguin Symbols (Wiley Online Library, 2016).

42. Aroyo, M. I. International Tables for Crystallography: Crystallographic Symmetry (John Wiley & Sons, 2021).

43. Larsen, A. H. et al. The atomic simulation environment—a Python library for working with atoms. J. Phys. Condens. Matter29, 273002 (2017).

44. Materials Properties Open Database (MPOD). MPODhttp://mpod.cimav.edu.mx/ (accessed 1 August 2025).

45. Bitzek, E., Koskinen, P., Gähler, F., Moseler, M. & Gumbsch, P. Structural relaxation made simple. Phys. Rev. Lett.97, 170201 (2006).

46. Berendsen, H. J. C., van Postma, J. P. M., Van Gunsteren, W. F., DiNola, A. R. H. J. & Haak, J. R. Molecular dynamics with coupling to an external bath. J. Chem. Phys.81, 3684–3690 (1984).

47. Matsumoto, Y. An Introduction to Morse Theory Vol. 208 (American Mathematical Society, 2002).

48. Mannan, S. et al. Evaluating universal machine learning force fields against experimental measurements [Dataset]. Zenodohttps://doi.org/10.5281/zenodo.16733258 (2026).

49. Mannan, S., Bihani, V., Gonzales, C. & Maheshwar, A. M3RG-IITD/UniFFBench: UniFFBench-v1.0.1 (v1.0.1). Zenodohttps://doi.org/10.5281/zenodo.20165458 (2026).

Acknowledgements

N.M.A.K. and S. Miret acknowledge financial support for this research from Intel. We thank the High Performance Computing (HPC) facility at IIT Delhi for providing the computational and storage resources used in the postprocessing of the simulations. We also thank A. Maheshwari for providing GPU computing resources and support with running the simulations. N.M.A.K. acknowledges the support from the Alexander von Humboldt Foundation and Anusandhan National Research Foundation through grant number ANRF/ARG/2025/007405/ENS. S. Mannan acknowledges financial support from the Prime Minister’s Research Fellowship (PMRF), Ministry of Education, Government of India.

Authors and Affiliations

Contributions

S. Mannan and N.M.A.K. jointly proposed the idea of benchmarking MLIPs against experimental data; S. Mannan, V.B., C.G. and K.L.K.L. led the implementation of different models and developed the UniFFBench framework. N.N.G., S.R. and S. Miret contributed to the analysis of results and provided scientific insights. Furthermore, N.M.A.K. and S. Miret acquired the funding, administered the project, provided resources and supervised the work. All authors reviewed, edited and approved the final version of the paper.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Katilin McCardle, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Sections 1–15, Figs. 1–35 and Tables 1–10.

Excel sheet for all the minerals.

Raw data for Fig. 2.

Raw data for Fig. 3.

Raw data for Fig. 4.

Raw data for Fig. 5.

Raw data for Fig. 6.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

About this article

Cite this article

Mannan, S., Bihani, V., Gonzales, C. et al. UniFFBench: evaluating universal machine learning force fields against experimental measurements.
Nat Comput Sci (2026). https://doi.org/10.1038/s43588-026-01019-4

• Version of record:14 July 2026

• DOI
  :https://doi.org/10.1038/s43588-026-01019-4