Variant effect prediction tools assessed using independent, functional assay-based datasets: implications for discovery and diagnostics

Human Genomics
Khalid MahmoodDaniel J Park

Abstract

Genetic variant effect prediction algorithms are used extensively in clinical genomics and research to determine the likely consequences of amino acid substitutions on protein function. It is vital that we better understand their accuracies and limitations because published performance metrics are confounded by serious problems of circularity and error propagation. Here, we derive three independent, functionally determined human mutation datasets, UniFun, BRCA1-DMS and TP53-TA, and employ them, alongside previously described datasets, to assess the pre-eminent variant effect prediction tools. Apparent accuracies of variant effect prediction tools were influenced significantly by the benchmarking dataset. Benchmarking with the assay-determined datasets UniFun and BRCA1-DMS yielded areas under the receiver operating characteristic curves in the modest ranges of 0.52 to 0.63 and 0.54 to 0.75, respectively, considerably lower than observed for other, potentially more conflicted datasets. These results raise concerns about how such algorithms should be employed, particularly in a clinical setting. Contemporary variant effect prediction tools are unlikely to be as accurate at the general prediction of functional impacts on proteins a...Continue Reading

References

Jan 11, 2000·Nucleic Acids Research·S T SherryK Sirotkin
May 5, 2001·Genome Research·P C Ng, S Henikoff
Dec 26, 2001·Nucleic Acids Research·Ada HamoshVictor A McKusick
May 20, 2003·Human Mutation·Peter D StensonDavid N Cooper
Jun 27, 2003·Proceedings of the National Academy of Sciences of the United States of America·Shunsuke KatoChikashi Ishioka
Jun 21, 2005·Genome Research·Gregory M CooperArend Sidow
Dec 31, 2005·Nucleic Acids Research·Cathy H WuBaris Suzek
Apr 1, 2010·Nature Methods·Ivan A AdzhubeiShamil R Sunyaev
Aug 3, 2010·Nature Methods·Jana Marie SchwarzDominik Seelow
Dec 15, 2010·PLoS Computational Biology·Eugene V DavydovSerafim Batzoglou
Apr 5, 2011·American Journal of Human Genetics·Abel González-Pérez, Nuria López-Bigas
Jul 6, 2011·Nucleic Acids Research·Boris RevaChris Sander
Jul 11, 2012·Nature Reviews. Genetics·Patrick F SullivanMichael O'Donovan
Jan 15, 2013·Current Protocols in Human Genetics·Ivan AdzhubeiShamil R Sunyaev
Jul 31, 2014·Nature Methods·Douglas M Fowler, Stanley Fields
Mar 6, 2015·Genetics in Medicine : Official Journal of the American College of Medical Genetics·Sue RichardsUNKNOWN ACMG Laboratory Quality Assurance Committee
Apr 1, 2015·Genetics·Lea M StaritaStanley Fields
Jul 25, 2015·BMC Bioinformatics·Jay K KhuranaJuilee Thakar
Aug 14, 2015·Proceedings of the National Academy of Sciences of the United States of America·Lisa A MiosgeT Daniel Andrews
Jan 5, 2016·Nature Genetics·Iuliana Ionita-LazaJoseph D Buxbaum
Jun 9, 2016·Genome Biology·William McLarenFiona Cunningham
Aug 19, 2016·Nature·Monkol LekUNKNOWN Exome Aggregation Consortium
Sep 27, 2016·American Journal of Human Genetics·Nilah M IoannidisWeiva Sieh

❮ Previous
Next ❯

Citations

Nov 30, 2018·PLoS Computational Biology·Brandon M ButlerS Banu Ozkan
Mar 7, 2019·Human Mutation·Peter B McGarveyUNKNOWN UniProt Consortium
Apr 30, 2019·PLoS Computational Biology·Anna-Leigh BrownAnna R Panchenko
Jul 4, 2019·Human Mutation·Maximilian MillerYana Bromberg
Jul 19, 2019·Human Mutation·Panagiotis Katsonis, Olivier Lichtarge
Feb 23, 2020·Journal of Inherited Metabolic Disease·Amy SirrAimée M Dudley
Jun 12, 2020·Experimental Biology and Medicine·Samuel M AdadeyAmbroise Wonkam
Jul 7, 2020·Molecular Systems Biology·Benjamin J Livesey, Joseph A Marsh
Jun 4, 2018·International Journal of Legal Medicine·Frank R WendtBruce Budowle
Jul 11, 2018·Genetics in Medicine : Official Journal of the American College of Medical Genetics·Natasha T StrandeJonathan S Berg
Sep 13, 2018·The Pharmacogenomics Journal·Yitian ZhouVolker M Lauschke
Dec 20, 2018·Frontiers in Pharmacology·Yitian ZhouVolker M Lauschke
Oct 5, 2019·Nucleic Acids Research·Maximilian MillerYana Bromberg
Sep 20, 2020·Journal of Human Genetics·Yukihide Momozawa, Keijiro Mizukami
Jan 23, 2020·Genetics in Medicine : Official Journal of the American College of Medical Genetics·Mark DrostNiels de Wind
Jul 23, 2020·Clinical Pediatric Endocrinology : Case Reports and Clinical Investigations : Official Journal of the Japanese Society for Pediatric Endocrinology·Satoshi Narumi
Nov 25, 2020·International Journal of Molecular Sciences·Iris B A W Te PaskeRicharda M de Voer
May 18, 2021·Clinical Pharmacology and Therapeutics·Yitian Zhou, Volker M Lauschke
Jul 8, 2021·Genetics in Medicine : Official Journal of the American College of Medical Genetics·C Cubuk CanVIG-Uk

❮ Previous
Next ❯

Software Mentioned

GESPA
PolyPhen
HumDiv
fitCons
Condel
MutationAssessor
HumVar
GERP
VEP
SwissVarSelected

Related Concepts

Related Feeds

Breast Cancer: BRCA1 & BRCA2

Mutations involving BRCA1, found on chromosome 17, and BRCA2, found on chromosome 13, increase the risk for specific cancers, such as breast cancer. Discover the last research on breast cancer BRCA1 and BRCA2 here.