Learning hierarchical features for scene labeling

IEEE Transactions on Pattern Analysis and Machine Intelligence
Clément FarabetYann LeCun

Abstract

Scene labeling consists of labeling each pixel in an image with the category of the object it belongs to. We propose a method that uses a multiscale convolutional network trained from raw pixels to extract dense feature vectors that encode regions of multiple sizes centered on each pixel. The method alleviates the need for engineered features, and produces a powerful representation that captures texture, shape, and contextual information. We report results using multiple postprocessing methods to produce the final labeling. Among those, we propose a technique to automatically retrieve, from a pool of segmentation components, an optimal set of components that best explain the scene; these components are arbitrary, for example, they can be taken from a segmentation tree or from any family of oversegmentations. The system yields record accuracies on the SIFT Flow dataset (33 classes) and the Barcelona dataset (170 classes) and near-record accuracy on Stanford background dataset (eight classes), while being an order of magnitude faster than competing approaches, producing a $(320\times 240)$ image labeling in less than a second, including feature extraction.

References

Nov 4, 2004·IEEE Transactions on Pattern Analysis and Machine Intelligence·Cheng-Lin LiuHiromichi Fujisawa
Mar 4, 2005·IEEE Transactions on Pattern Analysis and Machine Intelligence·Yuri Boykov, Vladimir Kolmogorov
Sep 30, 2005·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Feng NingPaolo Emilio Barbano
Apr 6, 2007·Journal of Neural Engineering·Russell S WitteDaryl R Kipke
Aug 25, 2010·IEEE Transactions on Pattern Analysis and Machine Intelligence·Pablo ArbelaezJitendra Malik
Dec 7, 2011·IEEE Transactions on Pattern Analysis and Machine Intelligence·Joao Carreira, Cristian Sminchisescu

Citations

Jan 15, 2014·PloS One·Thomas HannaganJonathan Grainger
Apr 16, 2014·Computational and Mathematical Methods in Medicine·Guo-Ping LiuPeng Qian
Dec 10, 2015·Computers in Biology and Medicine·Lu LiDayong Gao
Aug 25, 2015·Current Opinion in Neurobiology·Yasser Roudi, Graham Taylor
Dec 11, 2014·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Yazhou LiuQuansen Sun
Jan 27, 2015·IEEE Transactions on Neural Networks and Learning Systems·Yuan YuanXiaoqiang Lu
May 13, 2015·IEEE Transactions on Bio-medical Engineering·Youyi SongTianfu Wang
Sep 5, 2015·IEEE Transactions on Visualization and Computer Graphics·Clemens ArthVincent Lepetit
Aug 16, 2015·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Anran WangTat-Jen Cham
Nov 6, 2015·IEEE Transactions on Pattern Analysis and Machine Intelligence·Xiaodan LiangShuicheng Yan
Dec 15, 2015·IEEE Transactions on Pattern Analysis and Machine Intelligence·Ross GirshickJitendra Malik
Jun 13, 2015·IEEE Transactions on Neural Networks and Learning Systems·Maoguo GongLicheng Jiao
Feb 6, 2016·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Xutao LiMichael K Ng
Sep 29, 2015·IEEE Transactions on Medical Imaging·Fuyong XingLin Yang
Sep 4, 2015·IEEE Transactions on Pattern Analysis and Machine Intelligence·Mojtaba Seyedhosseini, Tolga Tasdizen
Mar 2, 2016·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Bing ShuaiBing Wang
Dec 20, 2015·IEEE Transactions on Medical Imaging·Yanrong GuoDinggang Shen
Dec 3, 2014·Neural Networks : the Official Journal of the International Neural Network Society·Jürgen Schmidhuber
Jul 11, 2015·Nature Reviews. Neuroscience·Rafael Yuste
May 29, 2015·Nature·Yann LeCunGeoffrey Hinton
Jan 18, 2015·Neural Networks : the Official Journal of the International Neural Network Society·Yoshua Bengio, Honglak Lee
Feb 21, 2014·Current Opinion in Neurobiology·David Daniel Cox
Feb 26, 2016·IEEE Transactions on Medical Imaging·Michiel KallenbergMartin Lillholm
Feb 18, 2016·IEEE Transactions on Medical Imaging·Mark J J P van GrinsvenClara I Sanchez
Jan 8, 2016·IEEE Transactions on Neural Networks and Learning Systems·Tianshui ChenXuelong Li
Jun 17, 2016·Medical Image Analysis·Mohammad HavaeiHugo Larochelle
Dec 30, 2014·PloS One·Bruno J T FernandesTsang I Ren
Aug 5, 2015·IEEE Transactions on Cybernetics·Yicong Zhou, Yantao Wei
Jul 18, 2016·Journal of Structural Biology·Feng WangJianyang Zeng
Jul 31, 2016·Briefings in Bioinformatics·Seonwoo MinSungroh Yoon
Aug 25, 2015·IEEE Transactions on Cybernetics·Liang Du, Haibin Ling
Dec 15, 2015·IEEE Transactions on Pattern Analysis and Machine Intelligence·Fayao LiuIan Reid
Dec 15, 2015·IEEE Transactions on Pattern Analysis and Machine Intelligence·Yi SunXiaoou Tang
Mar 10, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence· Xiaodan Liang Shuicheng Yan
Sep 14, 2016·IEEE Transactions on Medical Imaging·Youyi SongTianfu Wang
Jan 24, 2017·IEEE Transactions on Medical Imaging·Ahmed FakhryShuiwang Ji
Nov 11, 2016·IEEE Transactions on Medical Imaging·Ryan J CunninghamIan D Loram
Jun 14, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Bharath HariharanJitendra Malik
Jun 1, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Evan ShelhamerTrevor Darrell
Mar 16, 2017·Critical Care Medicine·Andy J MaSuchi Saria
Feb 17, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Jianping FanJinye Peng
Feb 17, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Feiyang ChengHong Zhang
Feb 12, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Mingyuan Jiu, Hichem Sahbi
Mar 10, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Zhongwen HuZhaocong Wu
Jan 24, 2017·IEEE Transactions on Medical Imaging·Juan WangPierre Baldi
Jul 23, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Marius CordtsStefan Roth
Mar 10, 2017·IEEE Transactions on Medical Imaging·Gustavo CarneiroNassir Navab
Jun 24, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Csaba Benedek
May 4, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Liang-Chieh ChenAlan L Yuille
Mar 27, 2018·Public Health Nutrition·Wenyan JiaMingui Sun
Jun 3, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Guosheng LinIan Reid
Jun 11, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Bing ShuaiGang Wang
Jun 15, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Fatemeh Sadat SalehStephen Gould
Oct 19, 2017·Computer Assisted Surgery·Xiaoqing Zhang
Mar 3, 2017·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Fayao LiuChunhua Shen
Mar 29, 2016·IEEE Transactions on Pattern Analysis and Machine Intelligence·Zifeng WuTieniu Tan
May 18, 2018·IEEE Transactions on Neural Networks and Learning Systems·Zhipeng RenZhipeng Ren
Jan 7, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Vijay BadrinarayananRoberto Cipolla
May 29, 2018·Nanoscale·Sungwoo ChunWanjun Park
Aug 11, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Ziwei LiuXiaoou Tang
Dec 19, 2017·Bioinformatics·Kristina PreuerGünter Klambauer
Jul 12, 2018·IEEE Transactions on Neural Networks and Learning Systems·Shuyuan YangRundong Li
Sep 1, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Xingyu ZengXiaogang Wang
Sep 19, 2017·IEEE Transactions on Medical Imaging·Gustavo CarneiroAndrew P Bradley
Jul 13, 2016·Journal of Biomedical Optics·Pavle PrentašicSven Loncaric
Sep 15, 2017·IEEE Transactions on Pattern Analysis and Machine Intelligence·Jiangye Yuan
Nov 22, 2017·Medical Physics·Aly A MohamedShandong Wu
Jun 9, 2019·Sensors·Suvash SharmaMuhammad Aminul Islam
Oct 3, 2017·Chemical & Pharmaceutical Bulletin·Kozo TakayamaKenji Sugibayashi
Jan 3, 2020·Physical Chemistry Chemical Physics : PCCP·Feifei HuangWenjing Hong
Mar 20, 2018·Alzheimer's & Dementia : the Journal of the Alzheimer's Association·Stuart MaudsleyBrain Health Modeling Initiative (BHMI)
Oct 5, 2019·Journal of Neural Engineering·Yimin HouXiangmin Lun
Aug 23, 2018·Optics Express·Ang LiuXiaoping Liu
Jul 16, 2020·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Georgia KoppeDaniel Durstewitz
Jun 3, 2020·Current Medical Imaging·Mazhar Basyouni Tayel, Azza Mahmoud Elbagoury
Aug 11, 2020·Computer Methods in Biomechanics and Biomedical Engineering·Ying WangBowei Fu
Jun 19, 2018·Frontiers in Neuroinformatics·Luis G HernándezJavier M Antelis
Mar 7, 2019·Journal of Healthcare Engineering·Zhenglun KongShengpu Xu
Feb 17, 2019·Molecular Psychiatry·Daniel DurstewitzAndreas Meyer-Lindenberg
Jul 21, 2019·Sensors·Itsaso Rodríguez-MorenoEkaitz Jauregi
Sep 17, 2020·Sensors·Valentina FranzoniOsvaldo Gervasi
Aug 25, 2020·Translational Vision Science & Technology·Qitong GaoMajda Hadziahmetovic
Jun 2, 2018·Journal of Healthcare Engineering·Zenghai ChenZheru Chi
Jun 16, 2018·Scientific Reports·Adedotun AkintayoSoumik Sarkar
Feb 17, 2019·Journal of Neuroengineering and Rehabilitation·Francesca StivalManfredo Atzori
Jul 31, 2019·Scientific Reports·Christian BerglerAndreas Maier
Feb 11, 2020·Frontiers in Computational Neuroscience·Subhashis Banerjee, Sushmita Mitra
Aug 25, 2020·Multimedia Tools and Applications·Valentina FranzoniAlfredo Milani
Jan 18, 2017·Neural Computation·Arash SamadiDouglas B Tweed
Feb 15, 2020·Plant Methods·Abraham George SmithCamilla Ruø Rasmussen
Jul 28, 2020·Environmental Science and Pollution Research International·Kyriaki KyliliAlessandro Artusi
Sep 19, 2018·Computational Intelligence and Neuroscience·Md Zahangir AlomVijayan K Asari
Jun 2, 2020·Biomedical Engineering Letters·Afzal Hussain Shahid, Maheshwari Prasad Singh
Feb 19, 2019·Journal of Healthcare Engineering·Jinjin HaiBin Yan
Jan 6, 2019·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Kevin R CoffeyJohn F Neumaier
May 15, 2020·Sensors·Rizwan Ahmed AnsariKrishna Mohan Buddhiraju
Jan 9, 2020·Clinical Cancer Research : an Official Journal of the American Association for Cancer Research·Feng XieTian Xia
Mar 27, 2019·Entropy·Yunqi TangBingxian Lin

Related Concepts

Genetic Vectors
Trees (plant)
Extraction
Shapes
Size

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Systemic Juvenile Idiopathic Arthritis

Systemic juvenile idiopathic arthritis is a rare rheumatic disease that affects children. Symptoms include joint pain, but also fevers and skin rashes. Here is the latest on this disease.

Chromatin Regulation and Circadian Clocks

The circadian clock plays an important role in regulating transcriptional dynamics through changes in chromatin folding and remodelling. Discover the latest research on Chromatin Regulation and Circadian Clocks here.

Central Pontine Myelinolysis

Central Pontine Myelinolysis is a neurologic disorder caused most frequently by rapid correction of hyponatremia and is characterized by demyelination that affects the central portion of the base of the pons. Here is the latest research on this disease.

Myocardial Stunning

Myocardial stunning is a mechanical dysfunction that persists after reperfusion of previously ischemic tissue in the absence of irreversible damage including myocardial necrosis. Here is the latest research.

Pontocerebellar Hypoplasia

Pontocerebellar hypoplasias are a group of neurodegenerative autosomal recessive disorders with prenatal onset, atrophy or hypoplasia of the cerebellum, hypoplasia of the ventral pons, microcephaly, variable neocortical atrophy and severe mental and motor impairments. Here is the latest research on pontocerebellar hypoplasia.

Cell Atlas Along the Gut-Brain Axis

Profiling cells along the gut-brain axis at the single cell level will provide unique information for each cell type, a three-dimensional map of how cell types work together to form tissues, and insights into how changes in the map underlie health and disease of the GI system and its crosstalk with the brain. Disocver the latest research on single cell analysis of the gut-brain axis here.

Chronic Traumatic Encephalopathy

Chronic Traumatic Encephalopathy (CTE) is a progressive degenerative disease that occurs in individuals that suffer repetitive brain trauma. Discover the latest research on traumatic encephalopathy here.

Related Papers

Conference Proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society
Nikolaos Giannakeas, Dimitrios I Fotiadis
Neural Networks : the Official Journal of the International Neural Network Society
Kevin Bailly, Maurice Milgram
© 2021 Meta ULC. All rights reserved