Semantic brain-computer interfacing
Brain-computer interfaces (BCIs) based on identifying activity in the brain related to semantic concepts have the potential to be highly intuitive and allow greater levels of accuracy and communication speed than possible with current BCIs. I will investigate novel machine learning techniques to develop a new type of semantic BCI based on simultaneous recording of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS).
We published our first results to decode semantic categories of imagined concepts of animals and tools in the recorded brain activity by functional near-infrared spectroscopy (fNIRS).
Decoding of semantic categories of imagined concepts of animals and tools in fNIRS. Journal of Neural Engineering 2021. Milan Rybář, Riccardo Poli, and Ian Daly. Objective: Semantic decoding refers to the identification of semantic concepts from recordings of an individual's brain activity. It has been previously reported in fMRI and EEG. We investigate whether semantic decoding is possible with functional near-infrared spectroscopy (fNIRS). Specifically, we attempt to differentiate between the semantic categories of animals and tools. We also identify suitable mental tasks for potential brain-computer interface (BCI) applications. Approach: We explore the feasibility of a silent naming task, for the first time in fNIRS, and propose three novel intuitive mental tasks based on imagining concepts using three sensory modalities: visual, auditory, and tactile. Participants are asked to visualize an object in their minds, imagine the sounds made by the object, and imagine the feeling of touching the object. A general linear model is used to extract hemodynamic responses that are then classified via logistic regression in a univariate and multivariate manner. Main results: We successfully classify all tasks with mean accuracies of 76.2% for the silent naming task, 80.9% for the visual imagery task, 72.8% for the auditory imagery task, and 70.4% for the tactile imagery task. Furthermore, we show that consistent neural representations of semantic categories exist by applying classifiers across tasks. Significance: These findings show that semantic decoding is possible in fNIRS. The study is the first step toward the use of semantic decoding for intuitive BCI applications for communication.