A Web Resource for Standardized Benchmark Datasets, Metrics, and Rosetta Protocols for Macromolecular Modeling and Design

PloS One
Shane Ó ConchúirTanja Kortemme

Abstract

The development and validation of computational macromolecular modeling and design methods depend on suitable benchmark datasets and informative metrics for comparing protocols. In addition, if a method is intended to be adopted broadly in diverse biological applications, there needs to be information on appropriate parameters for each protocol, as well as metrics describing the expected accuracy compared to experimental data. In certain disciplines, there exist established benchmarks and public resources where experts in a particular methodology are encouraged to supply their most efficient implementation of each particular benchmark. We aim to provide such a resource for protocols in macromolecular modeling and design. We present a freely accessible web resource (https://kortemmelab.ucsf.edu/benchmarks) to guide the development of protocols for protein modeling and design. The site provides benchmark datasets and metrics to compare the performance of a variety of modeling protocols using different computational sampling methods and energy functions, providing a "best practice" set of parameters for each method. Each benchmark has an associated downloadable benchmark capture archive containing the input files, analysis scripts...Continue Reading

References

Jan 20, 1995·Science·T Clackson, J A Wells
Jul 7, 1998·Journal of Molecular Biology·A A Bogan, K S Thorn
Dec 11, 1999·Nucleic Acids Research·H M BermanP E Bourne
Sep 14, 2000·Proceedings of the National Academy of Sciences of the United States of America·B Kuhlman, D Baker
Oct 25, 2000·Protein Science : a Publication of the Protein Society·A FiserA Sali
Sep 8, 2001·Journal of Molecular Biology·N V Dokholyan, E I Shakhnovich
Feb 6, 2002·Journal of Molecular Biology·Golan Yona, Michael Levitt
Jun 19, 2002·Accounts of Chemical Research·András FiserAndrej Sali
Jun 25, 2002·Journal of Molecular Biology·Raphael GueroisLuis Serrano
Oct 17, 2002·Proceedings of the National Academy of Sciences of the United States of America·Tanja Kortemme, David Baker
Apr 30, 2003·Protein Science : a Publication of the Protein Society·Adrian A Canutescu, Roland L Dunbrack
Jan 22, 2004·Journal of Computational Chemistry·Evangelos A CoutsiasKen A Dill
Feb 12, 2004·Science's STKE : Signal Transduction Knowledge Environment·Tanja KortemmeDavid Baker
Mar 30, 2004·Proteins·Matthew P JacobsonRichard A Friesner
Jun 3, 2004·Genome Research·Gavin E CrooksSteven E Brenner
Jun 28, 2005·Proteins·Julian MintserisZhiping Weng
Oct 4, 2005·Journal of Computational Chemistry·David A CaseRobert J Woods
Dec 31, 2005·Nucleic Acids Research·M D Shaji KumarAkinori Sarai
Aug 24, 2006·Proteins·Kai ZhuRichard A Friesner
Jul 17, 2007·PLoS Computational Biology·Konstantin B ZeldovichEugene I Shakhnovich
Jul 20, 2007·Structure·Joël Janin, Shoshana Wodak
Sep 11, 2007·Journal of Molecular Biology·Chu WangDavid Baker
Feb 27, 2008·Proteins·Benjamin D SellersMatthew P Jacobson
Jun 10, 2008·Nucleic Acids Research·Steven J DarnellJulie C Mitchell
Oct 3, 2008·PLoS Biology·Raffi TonikianSachdev S Sidhu
Jan 1, 2009·Nature Methods·Alexander BenedixRainer A Böckmann
May 16, 2009·Journal of Computational Chemistry·B R BrooksM Karplus
Jun 30, 2009·Protein Engineering, Design & Selection : PEDS·Vladimir PotapovGideon Schreiber
Sep 10, 2009·Science Signaling·Andreas ErnstSachdev S Sidhu
Jul 27, 2010·Journal of Molecular Biology·Colin A Smith, Tanja Kortemme
Sep 2, 2010·Proteins·Howook HwangZhiping Weng
Dec 29, 2010·Methods in Enzymology·Andrew Leaver-FayPhilip Bradley
Sep 10, 2011·Proteins·Suwen ZhaoRichard A Friesner
Feb 21, 2013·Methods in Enzymology·Noah OllikainenTanja Kortemme
Feb 21, 2013·Methods in Enzymology·Andrew Leaver-FayBrian Kuhlman
May 25, 2013·PloS One·Amelie Stein, Tanja Kortemme
Nov 12, 2013·Nucleic Acids Research·A Patrícia BentoJohn P Overington
Nov 19, 2013·PLoS Computational Biology·Noah Ollikainen, Tanja Kortemme
Nov 30, 2013·Nucleic Acids Research·Robert D FinnMarco Punta

❮ Previous
Next ❯

Citations

Oct 17, 2015·Protein Science : a Publication of the Protein Society·Sarah SirinAmy E Keating
Jul 2, 2016·Journal of Chemical Information and Modeling·Traian SuleaEnrico O Purisima
Apr 22, 2017·Journal of Chemical Theory and Computation·Rebecca F AlfordJeffrey J Gray
Jan 27, 2017·Nature Protocols·Brian D WeitznerJeffrey J Gray
Jul 28, 2017·PloS One·Victor VivcharukEnrico O Purisima
Mar 1, 2018·PLoS Computational Biology·Stefano MottaAlessandro Pandini
Aug 6, 2017·Genes, Chromosomes & Cancer·Caroline KampmeyerRasmus Hartmann-Petersen
Dec 30, 2015·PloS One·Sagar D Khare, Timothy A Whitehead
Jun 22, 2019·Genome Biology·Lukas M WeberMark D Robinson
Jun 3, 2020·Nature Methods·Julia Koehler LemanRichard Bonneau
Nov 3, 2020·Frontiers in Bioengineering and Biotechnology·Brandon FrenzYifan Song
Nov 13, 2020·Scientific Reports·Yoichi KurumidaTomoshi Kameda
Feb 6, 2018·The Journal of Physical Chemistry. B·Kyle A BarlowTanja Kortemme
Aug 24, 2021·Process Biochemistry·Huda Ahmed AlghamdiMuhammad Zohaib Nawaz
Dec 1, 2021·Nature Communications·Julia Koehler LemanRichard Bonneau

❮ Previous
Next ❯

Methods Mentioned

BETA
X-ray
phage display

Software Mentioned

CASP
Rosetta
Robetta
CAPRI
ProTherm
Rosetta Backrub
NGK
Talaris

Related Concepts

Related Feeds

BioHub - Researcher Network

The Chan-Zuckerberg Biohub aims to support the fundamental research and develop the technologies that will enable physicians to cure, prevent, or manage all diseases in our childrens' lifetimes. The CZ Biohub brings together researchers from UC Berkeley, Stanford, and UCSF. Find the latest research from the CZ Biohub researcher network here.