pubmed.ncbi.nlm.nih.gov

The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry - PubMed

  • ️Fri Jan 01 2016

The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry

Sebastiaan Mathôt et al. PLoS One. 2016.

Abstract

We present a new human-computer interface that is based on decoding of attention through pupillometry. Our method builds on the recent finding that covert visual attention affects the pupillary light response: Your pupil constricts when you covertly (without looking at it) attend to a bright, compared to a dark, stimulus. In our method, participants covertly attend to one of several letters with oscillating brightness. Pupil size reflects the brightness of the selected letter, which allows us-with high accuracy and in real time-to determine which letter the participant intends to select. The performance of our method is comparable to the best covert-attention brain-computer interfaces to date, and has several advantages: no movement other than pupil-size change is required; no physical contact is required (i.e. no electrodes); it is easy to use; and it is reliable. Potential applications include: communication with totally locked-in patients, training of sustained attention, and ultra-secure password input.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. The selection procedure.

a) Participants selected one of two (Phase 1), four (Phase 2), or eight (Phase 3) simultaneously presented stimuli. b) During each cycle, the brightness of the stimulus gradually changed in 0.5 s, and then remained constant for 0.75 s. Pupil size was measured during the last 0.25 s. c) The target stimulus was indicated by a cue. This example shows a correct selection, because the selected stimulus (‘a’) matches the cue. The size of the letters indicated how close they were to being selected. When a letter was selected, it smoothly moved toward the center. d) If there were more than two letters, letters were grouped by the brightness of their background. One group was eliminated on each selection, after which the remaining group was subdivided anew. This step-wise selection procedure repeated until a single winning stimulus remained.

Fig 2
Fig 2. Pupillary responses during one brightness-transition cycle.

a) Example data from one participant. Pupil size as a function of whether the target changes from bright to dark (blue line) or from dark to bright (orange line). Shadings indicate standard deviation. b) The pupil size difference (i.e. orange—blue) for all participants. The participant indicated in red did not reach our criteria for successful training. The participant indicated by the arrow corresponds to the example shown in (a). All data is from Phase 1, in which participants selected one out of two stimuli.

Fig 3
Fig 3. Selection accuracy (top row) and speed (bottom row) for individual participants (gray bars) and across participants (blue bars).

Horizontal dashed lines indicate chance level. a) Results for Phase 1. b) Results for Phase 2. c) Results for Phase 3. Error bars indicate 95% confidence intervals, within-subject where applicable [28].

Fig 4
Fig 4. The information-transfer rate (ITR) in bits per minute.

Bars indicate the mean ITR. Dots indicate individual participants.

Fig 5
Fig 5. Selection accuracy (top row) and speed (bottom row) as a function of block number.

Blue lines indicate across-participant means during the first six blocks, which were completed by all participants. The size of the gray circles indicates how often a score occurred. Performance during gaze-stabilization blocks is indicated by Stb. Horizontal dotted lines indicate chance level. a) Results for Phase 1. b) Results for Phase 2. c) Results for Phase 3. Error bars indicate 95% within-subject confidence intervals [28].

Fig 6
Fig 6. The symbol-selection procedure used for free writing.

Initially, there are eight groups of characters and control symbols (‘backspace’, ‘space’, and ‘accept’). When one group has been selected (here ‘abcd’), it unfolds into four individual symbols (here ‘a’, ‘b’, ‘c’, and ‘d’), after which a final selection is made (here ‘a’).

Similar articles

Cited by

References

    1. Birbaumer N. Breaking the silence: Brain–computer interfaces (BCI) for communication and motor control. Psychophysiol. 2006;43: 517–532. 10.1111/j.1469-8986.2006.00456.x - DOI - PubMed
    1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophysiol. 2002;113: 767–791. 10.1016/s1388-2457(02)00057-3 - DOI - PubMed
    1. Donoghue JP. Bridging the brain to the world: a perspective on neural interface systems. Neuron. 2008;60: 511–521. 10.1016/j.neuron.2008.10.037 - DOI - PubMed
    1. Nicolas-Alonso LF, Gomez-Gil J. Brain computer interfaces, a review. Sensors. 2012;12: 1211–1279. 10.3390/s120201211 - DOI - PMC - PubMed
    1. Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroen Clin Neuro. 1988;70: 510–523. 10.1016/0013-4694(88)90149-6 - DOI - PubMed

Publication types

MeSH terms

LinkOut - more resources