Capsule networks as recurrent models of grouping and segmentation

PLoS Computational Biology
A DoerigMichael H Herzog

Abstract

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that CapsNets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

References

Oct 1, 1975·Vision Research·G Westheimer, G Hauske
Jan 1, 1986·Psychological Research·E A Styles, D A Allport
Jan 1, 1985·Vision Research·D M LeviA P Aitsebaomo
Jun 6, 1996·Nature·S ThorpeC Marlot
Nov 14, 2000·Trends in Neurosciences·V A Lamme, P R Roelfsema
Jun 27, 2001·Nature Neuroscience·L ParkesM Morgan
Jan 25, 2002·Nature·Michael H Herzog, Manfred Fahle
Jan 27, 2005·Journal of Vision·Denis G PelliNajib J Majaj
Oct 7, 2008·Current Opinion in Neurobiology·Denis G Pelli
Mar 11, 2009·Journal of Vision·Toni P SaarelaMichael H Herzog
May 21, 2010·Psychological Science·B SayimM H Herzog
Oct 5, 2010·Journal of Vision·T P SaarelaM H Herzog
Mar 23, 2011·Trends in Cognitive Sciences·David Whitney, Dennis M Levi
Feb 14, 2012·Neuron·James J DiCarloNicole C Rust
Sep 29, 2012·Journal of Vision·Mauro ManassiMichael H Herzog
Apr 9, 2013·Vision Research·Julian M WallaceBosco S Tjan
Nov 12, 2013·Journal of Vision·Mauro ManassiMichael H Herzog
May 9, 2014·Proceedings of the National Academy of Sciences of the United States of America·Daniel L K YaminsJames J DiCarlo
Nov 7, 2014·PLoS Computational Biology·Seyed-Mahdi Khaligh-Razavi, Nikolaus Kriegeskorte
Jan 1, 2015·Journal of Vision·Srimant Prasad TripathyHarold E Bedell
May 20, 2015·Vision Research·K E Overvliet, B Sayim
Feb 26, 2016·Journal of Vision·Mauro ManassiMichael H Herzog
May 12, 2016·Current Biology : CB·Matthew V PachaiMichael H Herzog
Apr 25, 2017·Psychological Review·Gregory FrancisMichael H Herzog
Sep 28, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Weiming HuStephen Maybank
Sep 29, 2017·Frontiers in Psychology·Courtney J SpoererNikolaus Kriegeskorte
Jan 3, 2018·NeuroImage·Brandon M TurnerBirte U Forstmann
Feb 7, 2018·Current Biology : CB·Mauro Manassi, David Whitney
Jun 16, 2018·Science·S M Ali EslamiDemis Hassabis
Aug 15, 2018·Proceedings of the National Academy of Sciences of the United States of America·Hanlin TangGabriel Kreiman
Sep 18, 2018·IEEE Transactions on Pattern Analysis and Machine Intelligence·Avishek DuttaMichael Hahn
Dec 12, 2018·PLoS Computational Biology·Nicholas BakerPhilip J Kellman
Apr 6, 2019·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Taekjun KimAnitha Pasupathy
May 11, 2019·PLoS Computational Biology·Adrien DoerigMichael H Herzog
Jul 28, 2019·Journal of Vision·Ruth RosenholtzShaiyan Keshvari
Aug 9, 2019·Annual Review of Vision Science·Thomas Serre
Oct 9, 2019·Proceedings of the National Academy of Sciences of the United States of America·Tim C KietzmannNikolaus Kriegeskorte

❮ Previous
Next ❯

Citations

Jul 17, 2021·NeuroImage·Maya A JastrzębowskaMichael H Herzog
Jul 7, 2021·PLoS Computational Biology·Alban BornetErik Van der Burg
Sep 14, 2021·Journal of Vision·Oh-Hyeon ChoungMichael H Herzog
Oct 29, 2021·Journal of Vision·Robert J LeeJasna Martinovic

❮ Previous
Next ❯

Methods Mentioned

BETA
feature extraction

Software Mentioned

Adam
ffCNNs
Tensorflow
Python
CapsNets

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.