Learning semantic and visual similarity for endomicroscopy video retrieval

IEEE Transactions on Medical Imaging
Barbara AndréNicholas Ayache

Abstract

Content-based image retrieval (CBIR) is a valuable computer vision technique which is increasingly being applied in the medical community for diagnosis support. However, traditional CBIR systems only deliver visual outputs, i.e., images having a similar appearance to the query, which is not directly interpretable by the physicians. Our objective is to provide a system for endomicroscopy video retrieval which delivers both visual and semantic outputs that are consistent with each other. In a previous study, we developed an adapted bag-of-visual-words method for endomicroscopy retrieval, called "Dense-Sift," that computes a visual signature for each video. In this paper, we present a novel approach to complement visual similarity learning with semantic knowledge extraction, in the field of in vivo endomicroscopy. We first leverage a semantic ground truth based on eight binary concepts, in order to transform these visual signatures into semantic signatures that reflect how much the presence of each semantic concept is expressed by the visual words describing the videos. Using cross-validation, we demonstrate that, in terms of semantic detection, our intuitive Fisher-based method transforming visual-word histograms into semantic es...Continue Reading

References

Aug 1, 1991·Computer Methods and Programs in Biomedicine·M J Feldman, G O Barnett
Aug 13, 2009·World Journal of Gastroenterology : WJG·Omer KhalidDouglas K Rex
Nov 21, 2009·IEEE Transactions on Pattern Analysis and Machine Intelligence·Liu YangMahadev Satyanarayanan
Apr 9, 2010·Journal of Digital Imaging·Ceyhun Burak AkgülBurak Acar
Dec 2, 2010·IEEE Transactions on Medical Imaging·Uri AvniJacob Goldberger

❮ Previous
Next ❯

Citations

Jul 24, 2013·Medical & Biological Engineering & Computing·Giulio IannelloPaolo Soda
Mar 8, 2016·Medical Image Analysis·Georg WimmerAndreas Uhl
Oct 23, 2012·Medical Image Analysis·Toru TamakiShinji Tanaka
Jun 22, 2012·Medical Image Analysis·R KwittF Wrba
Sep 20, 2015·Medical Image Analysis·Michael HäfnerShigeto Yoshida
Jan 24, 2015·IEEE Transactions on Medical Imaging·Yang SongMei Chen
Nov 2, 2014·IEEE Transactions on Bio-medical Engineering·Menglin JiangDimitris Metaxas
May 9, 2014·IEEE Transactions on Medical Imaging·Adrien DepeursingeDaniel Rubin
Nov 16, 2013·IEEE Journal of Biomedical and Health Informatics·Subrahmanyam Murala, Q M Jonathan Wu
Nov 30, 2012·IEEE Journal of Biomedical and Health Informatics·Qianni Zhang, Ebroul Izquierdo
Dec 2, 2014·IEEE Journal of Biomedical and Health Informatics·José RamosAurélio Campilho
Oct 30, 2015·IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society·Shiv Ram DubeyRajat Kumar Singh
Mar 19, 2014·Journal of Biomedical Informatics·Camille KurtzDaniel L Rubin
Sep 16, 2015·IEEE Transactions on Bio-medical Engineering·Fan Zhang Alzheimers Disease Neuroimaging Initiative
Aug 16, 2016·Medical Image Analysis·Menglin JiangDimitris N Metaxas
May 14, 2020·Journal of Digital Imaging·Muhammad KashifFurqan Shaukat
Jun 28, 2018·International Journal of Computer Assisted Radiology and Surgery·Yachun LiStamatia Giannarou
Jul 6, 2020·Frontiers of Medicine·Allison J Navarrete-Welton, Daniel A Hashimoto
Oct 17, 2017·Medical Image Analysis·Zhongyu LiShaoting Zhang

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.