Scaling drug indication curation through crowdsourcing

Database : the Journal of Biological Databases and Curation
Ritu KhareZhiyong Lu

Abstract

Motivated by the high cost of human curation of biological databases, there is an increasing interest in using computational approaches to assist human curators and accelerate the manual curation process. Towards the goal of cataloging drug indications from FDA drug labels, we recently developed LabeledIn, a human-curated drug indication resource for 250 clinical drugs. Its development required over 40 h of human effort across 20 weeks, despite using well-defined annotation guidelines. In this study, we aim to investigate the feasibility of scaling drug indication annotation through a crowdsourcing technique where an unknown network of workers can be recruited through the technical environment of Amazon Mechanical Turk (MTurk). To translate the expert-curation task of cataloging indications into human intelligence tasks (HITs) suitable for the average workers on MTurk, we first simplify the complex task such that each HIT only involves a worker making a binary judgment of whether a highlighted disease, in context of a given drug label, is an indication. In addition, this study is novel in the crowdsourcing interface design where the annotation guidelines are encoded into user options. For evaluation, we assess the ability of ou...Continue Reading

References

Aug 11, 2000·BMJ : British Medical Journal·J W ElyP Z Stavri
Jun 18, 2005·BMC Bioinformatics·Christian BlaschkeAlfonso Valencia
Dec 15, 2005·BMC Bioinformatics·Barbara LazzariCarlo Pozzi
Sep 27, 2008·BMC Bioinformatics·Sunghwan SohnW John Wilbur
Feb 17, 2010·Database : the Journal of Biological Databases and Curation·Rezarta Islamaj DoganZhiyong Lu
Nov 26, 2010·Journal of Biomedical Informatics·Aurélie NévéolZhiyong Lu
Jun 17, 2011·BMC Bioinformatics·Antonio Jimeno-YepesAlan R Aronson
Oct 26, 2011·BMC Bioinformatics·Dan HeEran Halperin
Dec 7, 2011·Nucleic Acids Research·Michael Y Galperin, Xosé M Fernández-Suárez
Dec 22, 2011·BMC Bioinformatics·Zhiyong LuW John Wilbur
Dec 22, 2011·BMC Bioinformatics·Cecilia N ArighiCathy H Wu
Mar 17, 2012·Science Translational Medicine·Nicholas P TatonettiRuss B Altman
May 15, 2012·Journal of the American Medical Informatics Association : JAMIA·Allison B McCoyDean F Sittig
Nov 20, 2012·Database : the Journal of Biological Databases and Curation·Kimberly Van AukenUNKNOWN WormBase Consortium
Nov 20, 2012·Database : the Journal of Biological Databases and Curation·Chih-Hsuan WeiZhiyong Lu
Nov 20, 2012·Database : the Journal of Biological Databases and Curation·Sun KimW John Wilbur
Nov 28, 2012·Database : the Journal of Biological Databases and Curation·Thomas C WiegersCarolyn J Mattingly
Nov 30, 2012·IEEE Journal of Biomedical and Health Informatics·A K TriantafyllidisN Maglaveras
Dec 12, 2012·Database : the Journal of Biological Databases and Curation·Cathy H WuW John Wilbur
Jan 19, 2013·Database : the Journal of Biological Databases and Curation·Cecilia N ArighiCathy Wu
Feb 9, 2013·Nature Biotechnology·Karim R LakhaniEva C Guinan
Feb 12, 2013·Database : the Journal of Biological Databases and Curation·Fabio RinaldiTherese Vachon
Mar 12, 2013·Journal of the American Medical Informatics Association : JAMIA·Kin Wah FungDina Demner-Fushman
Apr 12, 2013·Journal of the American Medical Informatics Association : JAMIA·Wei-Qi WeiJoshua C Denny
May 25, 2013·Nucleic Acids Research·Chih-Hsuan WeiZhiyong Lu
Jun 21, 2013·Bioinformatics·Benjamin M Good, Andrew I Su
Aug 24, 2013·Bioinformatics·Robert LeamanZhiyong Lu
Nov 12, 2013·Journal of Biomedical Informatics·Azadeh NikfarjamGraciela Gonzalez
Feb 26, 2014·BMC Bioinformatics·Jiao Li, Zhiyong Lu
Jul 20, 2014·Database : the Journal of Biological Databases and Curation·Rafal RakSophia Ananiadou
Aug 15, 2014·Database : the Journal of Biological Databases and Curation·Manabu ToriiCathy H Wu
Aug 27, 2014·Database : the Journal of Biological Databases and Curation·Yuqing MaoZhiyong Lu
Sep 16, 2014·Journal of Biomedical Informatics·Ritu KhareZhiyong Lu
Sep 24, 2014·Database : the Journal of Biological Databases and Curation·John D BurgerLynette Hirschman
Nov 12, 2014·Journal of Clinical and Diagnostic Research : JCDR·Priyanka ChandPratima Khare

❮ Previous
Next ❯

Citations

Apr 19, 2015·Briefings in Bioinformatics·Ritu KhareZhiyong Lu
May 4, 2015·Briefings in Bioinformatics·Chung-Chi Huang, Zhiyong Lu
Apr 3, 2015·Briefings in Bioinformatics·Jiao LiZhiyong Lu
Dec 29, 2015·Database : the Journal of Biological Databases and Curation·Raul Rodriguez-Esteban
Apr 19, 2016·Database : the Journal of Biological Databases and Curation·Tong Shu LiAndrew I Su
Jul 14, 2016·Methods of Information in Medicine·David S CarrellLynette Hirschman
Aug 10, 2016·Database : the Journal of Biological Databases and Curation·Lynette HirschmanKevin Bretonnel Cohen
Sep 4, 2016·Database : the Journal of Biological Databases and Curation·Qinghua WangCecilia N Arighi
Sep 27, 2016·Nature Communications·Zichen WangAvi Ma'ayan
Apr 9, 2017·Journal of Biomedical Informatics·Anne CocosAaron J Masino
Jan 6, 2018·Journal of General Internal Medicine·Karoline Mortensen, Taylor L Hughes
Sep 25, 2017·ELife·Daniel Scott HimmelsteinSergio E Baranzini
Mar 23, 2018·Journal of Global Health·Kerri Wazny
May 17, 2018·Journal of Medical Internet Research·Perrine CréquitPhilippe Ravaud
Dec 2, 2020·Database : the Journal of Biological Databases and Curation·Diana SousaFrancisco M Couto
Feb 14, 2021·Journal of Biomedical Semantics·Kody MoodleyMichel Dumontier
Dec 24, 2020·Drug Safety : an International Journal of Medical Toxicology and Drug Experience·Alex GartlandGregory Eugene Powell
Aug 20, 2021·Frontiers in Artificial Intelligence·Arjun BhattZhichao Liu
May 6, 2017·Chemical Reviews·Martin KrallingerAlfonso Valencia
Sep 5, 2021·Journal of Biomedical Informatics·Krist ShingjergjiMichel Dumontier

❮ Previous
Next ❯

Software Mentioned

MTurk
RxNorm
DailyMed
MetaMap
LabeledIn
MTurk Developer Sandbox

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.