Deep learning for cellular image analysis

Nature Methods
Erick MoenDavid A Van Valen

Abstract

Recent advances in computer vision and machine learning underpin a collection of algorithms with an impressive ability to decipher the content of images. These deep learning algorithms are being applied to biological images and are transforming the analysis and interpretation of imaging data. These advances are positioned to render difficult analyses routine and to enable researchers to carry out new, previously impossible experiments. Here we review the intersection between deep learning and cellular image analysis and provide an overview of both the mathematical mechanics and the programming frameworks of deep learning that are pertinent to life scientists. We survey the field's progress in four key applications: image classification, image segmentation, object tracking, and augmented microscopy. Last, we relay our labs' experience with three key aspects of implementing deep learning in the laboratory: annotating training data, selecting and training a range of neural network architectures, and deploying solutions. We also highlight existing datasets and implementations for each surveyed application.

References

Jul 1, 1962·Journal of Chromatography·H J MAIER, O C KARPATHY
Jun 9, 2005·IEEE Transactions on Neural Networks·Fok Hing Chi Tivive, Abdesselam Bouzerdoum
Nov 2, 2006·Genome Biology·Anne E CarpenterDavid M Sabatini
Dec 6, 2006·Neural Networks : the Official Journal of the International Neural Network Society·François B VialatteGérard Dreyfus
Nov 21, 2007·IEEE Transactions on Bio-medical Engineering·Nezamoddin N Kachouie, Paul W Fieguth
Jul 22, 2008·Nature Methods·Khuloud JaqamanGaudenz Danuser
Apr 21, 2009·Methods in Molecular Biology·Sean G Megason
Aug 8, 2009·Seminars in Cell & Developmental Biology·Erik MeijeringWiggert A van Cappellen
Sep 26, 2009·Neuron·Eran A MukamelMark J Schnitzer
Jun 12, 2010·Current Biology : CB·Ping WangSuckjoon Jun
Sep 23, 2010·Neural Computation·Dan C CireşanJürgen Schmidhuber
Mar 30, 2011·Journal of the American Dietetic Association·A Catharine RossSue A Shapses
Mar 1, 2012·Nature Methods·Chris AllanJason R Swedlow
Mar 7, 2012·Neural Networks : the Official Journal of the International Neural Network Society·Ricardo de A Araújo
Jun 30, 2012·Nature Methods·Vebjorn LjosaAnne E Carpenter
Jun 30, 2012·Nature Methods·Johannes SchindelinAlbert Cardona
Jan 1, 2013·Nature Reviews. Drug Discovery·Mishal N PatelBissan Al-Lazikani
Mar 5, 2013·Cell·Jeremy E Purvis, Galit Lahav
Feb 15, 2014·Bioinformatics·Martin MaškaCarlos Ortiz-de-Solorzano
Mar 1, 2014·Medical Image Computing and Computer-assisted Intervention : MICCAI·Dan C CireşanJürgen Schmidhuber
Mar 4, 2014·Nature Medicine·Michael AngeloGarry Nolan
Jul 16, 2014·PeerJ·Stéfan van der Waltscikit-image contributors
Nov 22, 2014·IEEE Transactions on Medical Imaging·Klas E G MagnussonHelen M Blau
May 29, 2015·Nature·Yann LeCunGeoffrey Hinton
Jun 2, 2015·Nature Reviews. Drug Discovery·Catherine L JonesThierry T Diagana
Jul 28, 2015·Nature Biotechnology·Babak AlipanahiBrendan J Frey
Nov 6, 2015·Molecular Microbiology·Ahmad PaintdakhiChristine Jacobs-Wagner
Dec 4, 2015·Nature·Davide Castelvecchi
Jan 10, 2016·Journal of Biomolecular Screening·Chetak KandaswamyJorge M Santos
Jun 14, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Shaoqing RenJian Sun
Jun 17, 2016·Bioinformatics·Oren KrausBrendan J Frey
Jul 31, 2016·Molecular Systems Biology·Christof AngermuellerOliver Stegle
Aug 16, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Andrej Karpathy, Li Fei-Fei
Aug 30, 2016·Molecular Microbiology·Stella StylianidouPaul A Wiggins
Oct 8, 2016·Methods : a Companion to Methods in Enzymology·Jean-Yves TinevezKevin W Eliceiri
Nov 1, 2016·Nature Biotechnology·Loïc A RoyerPhilipp J Keller
Nov 5, 2016·PLoS Computational Biology·David A Van ValenMarkus W Covert
Feb 22, 2017·Nature Methods·Felix BuggenthinCarsten Marr
Mar 14, 2017·IEEE Transactions on Medical Imaging·Neeraj KumarAmit Sethi
Apr 20, 2017·Molecular Systems Biology·Oren KrausBrenda J Andrews
May 13, 2017·Science·Peter J ThulEmma Lundberg
Jun 24, 2017·Bioinformatics·Sam CooperChris Bakal
Aug 5, 2017·Medical Image Analysis·Geert LitjensClara I Sánchez
Aug 12, 2017·Scientific Reports·Sajith Kecheril SadanandanCarolina Wählby
Sep 1, 2017·Nature Methods·Juan C CaicedoAnne E Carpenter
Sep 5, 2017·Nature Methods·Jonathan B GrimmLuke D Lavis
Sep 29, 2017·Molecular Biology of the Cell·Christoph SommerDaniel W Gerlich
Oct 27, 2017·Nature·Orit Rozenblatt-RosenSarah A Teichmann
Dec 6, 2017·ELife·Aviv RegevHuman Cell Atlas Meeting Participants
Dec 28, 2017·Proceedings of the National Academy of Sciences of the United States of America·Daniël M Pelt, James A Sethian
Dec 30, 2017·Proceedings of the National Academy of Sciences of the United States of America·Hui LiuZhe Liu
Apr 17, 2018·Cell·Eric M ChristiansenSteven Finkbeiner
Apr 21, 2018·Cell·Devin P Sullivan, Emma Lundberg
Jul 4, 2018·PLoS Biology·Claire McQuinAnne E Carpenter
Jul 11, 2018·IEEE Transactions on Neural Networks and Learning Systems· Fuyong Xing Lin Yang
Jul 18, 2018·Nature Methods·Michał JanuszewskiViren Jain
Aug 2, 2018·Nature Methods·Alex J HughesZev J Gartner
Aug 22, 2018·Nature Neuroscience·Alexander MathisMatthias Bethge
Aug 24, 2018·Proceedings of the National Academy of Sciences of the United States of America·Jay M NewbySamuel K Lai
Sep 1, 2018·Cell·Nao NittaKeisuke Goda
Sep 2, 2018·Nature Methods·Matthias G HaberlMark H Ellisman
Sep 25, 2018·Nature Biotechnology·Ryan PoplinMark A DePristo
Nov 28, 2018·Nature Methods·Martin WeigertEugene W Myers
Dec 19, 2018·Nature Methods·Thorsten FalkOlaf Ronneberger
Dec 23, 2018·Nature Methods·Talmo D PereiraJoshua W Shaevitz
Jan 15, 2019·Current Biology : CB·Giuseppe FacchettiMartin Howard
Jan 16, 2019·Nature Methods·Francisco Romero-FerreroGonzalo G de Polavieja
Jul 10, 2019·IEEE Transactions on Medical Imaging·Faisal MahmoodNicholas J Durr
Jul 18, 2019·Cytometry. Part a : the Journal of the International Society for Analytical Cytology·Juan C CaicedoAnne E Carpenter

Citations

Jul 10, 2019·Nature Methods·Chinmay Belthangady, Loïc A Royer
Sep 20, 2019·The Plant Journal : for Cell and Molecular Biology·Eun-Cheon LimKyuha Choi
Nov 30, 2019·Nature Methods·Wei OuyangEmma Lundberg
Oct 23, 2019·Nature Methods·Juan C CaicedoAnne E Carpenter
Apr 14, 2020·PLoS Computational Biology·Jean-Baptiste LugagneMary J Dunlop
May 23, 2020·FEBS Letters·Anna E Eastman, Shangqin Guo
Jun 4, 2020·Circulation Research·Shah R AliHesham A Sadek
Jul 10, 2020·Lab on a Chip·Akihiro IsozakiKeisuke Goda
Jun 20, 2020·FEMS Microbiology Reviews·Valentine Lagage, Stephan Uphoff
Aug 25, 2020·Journal of Materials Chemistry. B, Materials for Biology and Medicine·John W Smith, Qian Chen
Sep 15, 2020·PLoS Computational Biology·Linfeng YangJan T Liphardt
Aug 11, 2020·Biology of Reproduction·Manon LesageViolette Thermes
Dec 15, 2019·Journal of Biomedical Optics·Lopamudra MukherjeeKevin W Eliceiri
Mar 25, 2020·Biomedical Optics Express·Leonhard MöcklW E Moerner
Jan 2, 2020·Nature Reviews. Rheumatology·Felix J Hartmann, Sean C Bendall
Nov 30, 2019·Nature Methods·Wei OuyangChristophe Zimmer
Feb 9, 2020·Scientific Reports·Zhao B KangChristophe Antczak
Jun 13, 2020·Journal of Cell Science·Guillaume JacquemetChristophe Leterrier
Mar 3, 2020·Microscopy·Sadao OtaRyoichi Horisaki
Dec 6, 2019·Scientific Reports·Dongyoung KimYoon-Kyoung Cho
Sep 11, 2020·Digestive Endoscopy : Official Journal of the Japan Gastroenterological Endoscopy Society·Hiroshi NakaseHiro-O Yamano
Oct 27, 2020·Journal of Forensic Sciences·Jack GarlandRexson Tse
Nov 4, 2020·Journal of Applied Toxicology : JAT·Anshul Srivastava, Joseph P Hanig
Nov 4, 2020·Cytometry. Part a : the Journal of the International Society for Analytical Cytology·George ZakiGianluca Pegoraro
Nov 28, 2020·Nature Communications·Qingyu ZhaoKilian M Pohl
Dec 15, 2020·Briefings in Bioinformatics·Ninon BurgosOlivier Colliot
Dec 11, 2020·Micromachines·Shaobo LuoAi Qun Liu
Oct 22, 2020·The Journal of Biological Chemistry·Jacob Aaron KlicksteinMalavika Raman
Dec 29, 2020·PLoS Computational Biology·Julienne LaChance, Daniel J Cohen
Oct 1, 2020·Computational and Structural Biotechnology Journal·Erik Meijering
Jan 14, 2021·Scientific Reports·Luca MasinLies De Groef
Jan 14, 2021·Trends in Cancer·Christian GnannEmma Lundberg
Jul 7, 2020·Journal of Pharmacological and Toxicological Methods·Mahnaz MaddahAlexandre J S Ribeiro
Jan 1, 2021·The Journal of Biological Chemistry·Katharina M SchererClemens F Kaminski
Dec 8, 2020·Computers in Biology and Medicine·Bruno C Gregório da SilvaRicardo J Ferrari
Apr 13, 2020·Methods : a Companion to Methods in Enzymology·Steven Vermeulen, Jan de Boer
Nov 27, 2020·FEMS Microbiology Reviews·Hannah Jeckel, Knut Drescher
Feb 4, 2021·Accounts of Chemical Research·Bhupalee KalitaKieron Burke
Oct 24, 2020·European Journal of Nuclear Medicine and Molecular Imaging·Hongmei WangKristen W Yeom
Nov 12, 2020·Sensors·Pasquale MemmoloCosimo Distante
Jan 13, 2021·Development·Paul Villoutreix
Dec 24, 2020·Nature Reviews. Drug Discovery·Srinivas Niranj ChandrasekaranAnne E Carpenter
Jan 6, 2021·Nature Microbiology·Raimo HartmannKnut Drescher
Nov 15, 2020·Communications Biology·Samuel BerrymanHongshen Ma
Feb 12, 2021·Nature Communications·Krisztian KoosPeter Horvath
Feb 17, 2021·Nature Methods·Michael Lawson, Johan Elf
Jan 30, 2021·Patterns·Aleix Boquet-PujadasNancy Guillén
Feb 16, 2020·Cell·Shan ZhaoAli Ertürk
Feb 23, 2021·PLoS Computational Biology·Philipp MergenthalerDavid W Andrews
Jan 20, 2021·ACS Nano·Xingchen DongAlexander W Koch
Mar 5, 2021·Journal of Chemical Information and Modeling·Maria-Anna TrapotsiAndreas Bender
Mar 3, 2021·Light, Science & Applications·Xinyang LiQionghai Dai
Jan 18, 2021·The Journal of Biological Chemistry·Jacob Aaron KlicksteinMalavika Raman
Mar 10, 2021·Nature Methods·Linjing FangUri Manor
Mar 11, 2021·Applied Optics·Ryoichi HorisakiJun Tanida
Mar 12, 2021·Cell Communication and Signaling : CCS·Susan E LeggettIan Y Wong
Apr 16, 2020·Structure·Yilai LiMichael A Cianfrocco
Apr 3, 2021·Journal of Cell Science·Meghan K Driscoll, Assaf Zaritsky
Apr 16, 2021·Nature Communications·Lucas von ChamierRicardo Henriques
Apr 17, 2021·PloS One·Fabian EnglbrechtAndreas R Bausch
Apr 19, 2021·Expert Opinion on Drug Discovery·W Patrick Walters, Regina Barzilay
Apr 12, 2021·Cytometry. Part a : the Journal of the International Society for Analytical Cytology·Shanshan LiuXuantao Su
May 1, 2021·Sensors·Soumaya DghimRadim Burget
Apr 20, 2021·Molecular Biology of the Cell·Alice M LucasAnne E Carpenter
Apr 30, 2021·Frontiers in Neuroscience·Xinglong WuJin Huang
May 1, 2021·International Journal of Molecular Sciences·Ewa BączyńskaBłażej Ruszczycki
May 7, 2021·Nature Communications·Abel SzkalisityPeter Horvath
May 12, 2021·Nature Communications·Yanjing ZhuRongrong Zhu
May 12, 2021·Annals of Biomedical Engineering·Jawaad SheriffDanny Bluestein
May 24, 2021·Current Opinion in Chemical Biology·Aditya PratapaJuan C Caicedo
Jun 7, 2021·Computers in Biology and Medicine·Zhichao LiuYingke Xu
Jun 11, 2021·Spectrochimica Acta. Part A, Molecular and Biomolecular Spectroscopy·JiaQi YueYuanZhong Wang

Methods Mentioned

BETA
biosensors
differential interference contrast microscopy
fluorescence microscopy

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Hereditary Sensory Autonomic Neuropathy

Hereditary Sensory Autonomic Neuropathies are a group of inherited neurodegenerative disorders characterized clinically by loss of sensation and autonomic dysfunction. Here is the latest research on these neuropathies.

Spatio-Temporal Regulation of DNA Repair

DNA repair is a complex process regulated by several different classes of enzymes, including ligases, endonucleases, and polymerases. This feed focuses on the spatial and temporal regulation that accompanies DNA damage signaling and repair enzymes and processes.

Glut1 Deficiency

Glut1 deficiency, an autosomal dominant, genetic metabolic disorder associated with a deficiency of GLUT1, the protein that transports glucose across the blood brain barrier, is characterized by mental and motor developmental delays and infantile seizures. Follow the latest research on Glut1 deficiency with this feed.

Separation Anxiety

Separation anxiety is a type of anxiety disorder that involves excessive distress and anxiety with separation. This may include separation from places or people to which they have a strong emotional connection with. It often affects children more than adults. Here is the latest research on separation anxiety.

KIF1A Associated Neurological Disorder

KIF1A associated neurological disorder (KAND) is a rare neurodegenerative condition caused by mutations in the KIF1A gene. KAND may present with a wide range and severity of symptoms including stiff or weak leg muscles, low muscle tone, a lack of muscle coordination and balance, and intellectual disability. Find the latest research on KAND here.

Regulation of Vocal-Motor Plasticity

Dopaminergic projections to the basal ganglia and nucleus accumbens shape the learning and plasticity of motivated behaviors across species including the regulation of vocal-motor plasticity and performance in songbirds. Discover the latest research on the regulation of vocal-motor plasticity here.

Related Papers

BioRxiv : the Preprint Server for Biology
Aritra ChowdhurySimone Bianco
IEEE Transactions on Neural Networks and Learning Systems
Fuyong Xing Lin Yang
© 2021 Meta ULC. All rights reserved