PMID: 15244698Jul 13, 2004Paper

Estimating mutual information

Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics
Alexander KraskovPeter Grassberger

Abstract

We present two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y). In contrast to conventional estimators based on binnings, they are based on entropy estimates from k -nearest neighbor distances. This means that they are data efficient (with k=1 we resolve structures down to the smallest possible scales), adaptive (the resolution is higher where data are more numerous), and have minimal bias. Indeed, the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale, giving typically systematic errors which scale as functions of k/N for N points. Numerically, we find that both families become exact for independent distributions, i.e. the estimator M(X,Y) vanishes (up to statistical fluctuations) if mu(x,y)=mu(x)mu(y). This holds for all tested marginal distributions and for all dimensions of x and y. In addition, we give estimators for redundancies between more than two random variables. We compare our algorithms in detail with existing algorithms. Finally, we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from ...Continue Reading

References

Jul 1, 1996·Journal of Clinical Gastroenterology·A González RamírezS Avila
Nov 24, 1998·IEEE Engineering in Medicine and Biology Magazine : the Quarterly Magazine of the Engineering in Medicine & Biology Society·B PompeM Eiselt
Feb 1, 1986·Physical Review A: General Physics·A M Fraser, H L Swinney
Sep 1, 1995·Physical Review. E, Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics·Y I MoonU Lall
Aug 10, 2000·Cell·T R HughesS H Friend
Nov 23, 2000·Physical Review. E, Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics·H Matsuda
Jan 7, 2003·Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics·Jonathan D Victor
Jan 29, 2003·IEEE Transactions on Bio-medical Engineering·Frank MeineckeKlaus-Robert Müller
Oct 1, 1998·Trends in Cognitive Sciences·K J Friston

❮ Previous
Next ❯

Citations

Mar 25, 2010·Journal of Computational Neuroscience·Pierre-Olivier Amblard, Olivier J J Michel
Aug 14, 2010·Journal of Computational Neuroscience·Raul VicenteGordon Pipa
Aug 28, 2010·Journal of Computational Neuroscience·Joseph T LizierMikhail Prokopenko
Jul 31, 2013·Neural Networks : the Official Journal of the International Neural Network Society·Benoît FrénayMichel Verleysen
Mar 21, 2008·Computational Intelligence and Neuroscience·Cota Navin Gupta, Ramaswamy Palaniappan
May 20, 2005·Neural Computation·Matthew B KennelE J Chichilnisky
Aug 19, 2005·Neural Computation·Marc M Van Hulle
Dec 19, 2007·Neural Computation·Marc M Van Hulle
Jan 1, 2013·Neural Computation·Taiji Suzuki, Masashi Sugiyama
Jun 24, 2011·Biology Direct·Gökmen Altay, Frank Emmert-Streib
Feb 2, 2011·PLoS Computational Biology·Adam B Barrett, Anil K Seth
Dec 5, 2012·PLoS Computational Biology·Elisa CiliaTom Lenaerts
Apr 16, 2011·PloS One·Jouni TakaloMikko Vähäsöyrinki
Nov 1, 2012·PloS One·Murilo S BaptistaCelso Grebogi
Mar 8, 2013·PloS One·Michael WibralRaul Vicente
Mar 7, 2014·PloS One·X Rosalind WangStephen C Trowell
Jul 30, 2014·PloS One·Patricia WollstadtMichael Wibral
Jul 21, 2007·The Journal of Chemical Physics·Benjamin J KillianMichael K Gilson
Mar 8, 2013·The Journal of the Acoustical Society of America·Theodoros PetsatodisRamjee Prasad
Jun 13, 2014·Current Opinion in Biotechnology·Andre Levchenko, Ilya Nemenman
Apr 9, 2014·Bioinformatics·Alexandra JauhiainenSimon Tavaré
Dec 9, 2014·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Stephen E Robinson, Arnold J Mandell
Mar 26, 2010·Analytical and Bioanalytical Chemistry·Yu B MonakhovaS A Astakhov
Jan 6, 2016·Current Opinion in Microbiology·Gabriele Micali, Robert G Endres
Jan 19, 2016·Frontiers in Systems Neuroscience·André M Bastos, Jan-Mathijs Schoffelen
Feb 3, 2016·Frontiers in Human Neuroscience·Tetsuo KidaRyusuke Kakigi
Nov 28, 2012·Fluid Phase Equilibria·Krishna M YerramsettyKhaled A M Gasem
Apr 27, 2012·Chaos, Solitons, and Fractals·D J Albers, George Hripcsak
Dec 14, 2011·Medical Image Analysis·Vanessa Gómez-VerdejoAntonio Oliviero
Jan 30, 2016·Decision Support Systems·Saike HeDaniel Zeng

❮ Previous
Next ❯

Related Feeds

Anxiety Disorders

Discover the latest research on anxiety disorders including agoraphobia, panic disorder, obsessive-compulsive disorder, and post-traumatic stress disorder here.

Related Papers

Physical Review Letters
T Schreiber
Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics
Harald StögbauerPeter Grassberger
© 2022 Meta ULC. All rights reserved