Towards a generalization of information theory for hierarchical partitions

Physical Review. E
Juan Ignacio PerottiFabio Saracco

Abstract

Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrativ...Continue Reading

References

Mar 15, 2003·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·Erzsébet Ravasz, Albert-László Barabási
Feb 3, 2004·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·R GuimeràA Arenas
Jan 28, 2005·Nature·Chaoming SongHernán A Makse
Feb 9, 2005·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·Juyong Park, M E J Newman
Feb 7, 2007·Physical Review Letters·Changsong ZhouJürgen Kurths
Aug 7, 2007·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·Lev MuchnikYoram Louzoun
Sep 21, 2007·Proceedings of the National Academy of Sciences of the United States of America·Marta Sales-PardoLuís A Nunes Amaral
Jan 7, 2014·PloS One·Gergely TibélyGergely Palla
Feb 11, 2014·Molecular Biology and Evolution·Leonidas SalichosAntonis Rokas
Aug 12, 2014·Nature Communications·Martin RosvallRenaud Lambiotte
Jun 16, 2015·Journal of Statistical Physics·Dirk HelbingMatjaž Perc
Jan 15, 2016·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·Juan Ignacio PerottiGuido Caldarelli
Apr 15, 2016·Physical Review. E·Masoumeh KheirkhahzadehMartin Rosvall
Nov 2, 2016·PloS One·Gergely TibélyGergely Palla
Apr 27, 2017·Scientific Reports·Sebastian GrauwinCarlo Ratti
Jan 20, 2018·Physical Review. E·Zhao YangClaudio J Tessone
Sep 27, 2018·Physical Review. E·Byoung-Hwa LeeHang-Hyun Jo
Jun 14, 2019·Scientific Reports·Alexander J GatesYong-Yeol Ahn
Oct 28, 2019·Nature Communications·Aleix BassolasJosé J Ramasco
May 20, 2020·Physical Review. E·M E J NewmanJean-Gabriel Young

❮ Previous
Next ❯

Citations

May 20, 2021·Physical Review. E·Giulia Bertagnolli, Manlio De Domenico

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.