Abstract
While promising for many applications, electroencephalography (EEG)-based brain-computer interfaces (BCIs) are still scarcely used outside laboratories, due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires the use of appropriate reliability metrics to quantify both the classification algorithm and the BCI user's performances. So far, classification accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study BCI users' skills. Here, we propose a definition and new metrics to quantify such BCI skills for mental imagery (MI) BCIs, independently of any classification algorithm. We first show in this paper that CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful self-modulation of EEG patterns by the user. We then propose a definition of MI-BCI skills that reflects how well the user can self-modulate EEG patterns, and thus how well he could control an MI-BCI. Finally, we propose new performance metrics, classDis, restDist and classStab that specifically measure how distinct and stable the EEG patterns produced by the user are, independently of any...Continue Reading
References
Oct 1, 1978·Perceptual and Motor Skills·S G Vandenberg, A R Kuse
Sep 28, 1998·IEEE Transactions on Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·J R WolpawG Pfurtscheller
Feb 24, 2001·IEEE Transactions on Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·H RamoserG Pfurtscheller
Apr 6, 2007·Journal of Neural Engineering·F LotteB Arnaldi
Jul 3, 2007·IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·Luigi BianchiMaria Grazia Marciani
Aug 4, 2009·NeuroImage·Ryota Tomioka, Klaus-Robert Müller
Aug 25, 2009·Journal of Neural Engineering·E A FeltonJ C Williams
Jan 7, 2010·Medical & Biological Engineering & Computing·Gernot R Müller-PutzGert Pfurtscheller
Jan 13, 2010·IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·Bernardo Dal SenoLuca T Mainardi
Dec 18, 2010·Neural Computation·Carmen VidaurreBenjamin Blankertz
Mar 26, 2011·Journal of Neural Engineering·Carmen VidaurreBenjamin Blankertz
May 17, 2011·Journal of Neuroscience Methods·Dennis J McFarlandJonathan R Wolpaw
May 4, 2013·Journal of Neural Engineering·Eoin ThomasMaureen Clerc
Sep 26, 2013·Frontiers in Human Neuroscience·Fabien LotteChristian Mühl
Oct 3, 2013·PloS One·Elisabeth V C FriedrichReinhold Scherer
Oct 16, 2013·Neuroscience and Biobehavioral Reviews·John H Gruzelier
Mar 25, 2014·Journal of Neural Engineering·N Jeremy HillGerwin Schalk
May 20, 2014·Journal of Neural Engineering·David E ThompsonJane E Huggins
Jun 6, 2014·PloS One·Muhammed Jamsheer K, Ashverya Laxmi
Jul 12, 2014·PloS One·Josef FallerGernot R Müller-Putz
Sep 10, 2014·IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·Kenneth ColwellKenneth Morton
Mar 13, 2015·Frontiers in Behavioral Neuroscience·Robert Bauer, Alireza Gharabaghi
Apr 19, 2015·NeuroImage·Catharina ZichMaarten De Vos
Sep 16, 2015·IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·Linsey RoijendijkJason Farquhar
Dec 2, 2015·PloS One·Camille JeunetFabien Lotte
May 14, 2016·Journal of Neural Engineering·Camille JeunetFabien Lotte
Sep 4, 2016·Progress in Brain Research·C JeunetF Lotte
Nov 16, 2016·IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society·Florian YgerFabien Lotte
Feb 23, 2017·PloS One·Martin Spüler
Jan 1, 2017·Brain Computer Interfaces·Ricardo ChavarriagaReinhold Scherer
Citations
Nov 16, 2019·Journal of Neural Engineering·Amanda Medeiros de FreitasJérémie Mattout
Nov 15, 2019·Journal of Neural Engineering·Daniel Freer, Guang-Zhong Yang
Dec 12, 2019·Journal of Neural Engineering·Rakib Al-FahadGavin M Bidelman
Jan 22, 2020·Journal of Neural Engineering·Marko AngjelichinoskiVahid Tarokh
Jan 21, 2020·Journal of Neural Engineering·Takashi NishimotoShin Ishii
Mar 19, 2020·Sensors·Mareike DaeglauCatharina Zich
Sep 15, 2018·Journal of Neural Engineering·Zied TayebJörg Conradt
Mar 28, 2020·Journal of Neural Engineering·Carolina MigliorelliVictoria San Antonio-Arce
Apr 10, 2020·Journal of Neural Engineering·Andreas SchwarzGernot R Müller-Putz
Aug 12, 2020·Journal of Neural Engineering·Yaqi ChuYiwen Zhao
May 7, 2019·Frontiers in Human Neuroscience·Jianjun Meng, Bin He
Sep 10, 2020·Frontiers in Human Neuroscience·Minji LeeSeong-Whan Lee
Sep 12, 2020·Journal of Neural Engineering·Demetres Kostas, Frank Rudzicz
May 6, 2020·Journal of Neural Engineering·Jennifer StisoDanielle S Bassett
Nov 13, 2020·Journal of Neural Engineering·Aline RocFabien Lotte
Apr 6, 2021·Frontiers in Human Neuroscience·Camille BenarochFabien Lotte
Mar 20, 2019·L'Encéphale·J-M BatailUNKNOWN NExT group
Jun 25, 2021·Frontiers in Human Neuroscience·Xu DuanZhenzhen Wang