Dissertations / Theses on the topic 'Brain decoding'

A brain-computer interface (BCI) allows human to communicate with a computer by thoughts. Recent advances in brain decoding have shown the capability of BCIs in monitoring physiological and cognitive state of the brain, including drowsiness. Since drowsy driving has been an urgent issue in vehicle safety that causes numerous deaths and injuries, BCIs based on non-invasive electroencephalogram (EEG) are developed to monitor drivers’ drowsiness continuously and instantaneously. Nonetheless, on the pathway of transitioning laboratory-oriented BCI into real-world applications, there are major challenges that limit the usability and convenience for drowsiness detection (DD). To completely understand the association between human EEG and drowsiness, this study employed a large-scale dataset collected from simulated driving experiments with a lane-keeping task and EEG recordings. A DD-BCI that acquires EEG from only non-hair-bearing (NHB) areas was proposed to maximize the comfort and convenience. The performance of the NHB DD-BCI was validated and compared with that using whole-scalp EEG, showing no significant difference in the accuracy of alert/drowsy classification. In addition, a subject-transfer framework that leverages large-scale existing data from other subjects was proposed to reduce the calibration time of a DD-BCI. Alert baseline data were involved to enhance the efficiency of subject-to-subject model transfer. The subject-transfer approach significantly reduced the calibration time of the DD-BCI, exhibiting the potential in facilitating plug-and-play brain decoding for real-world BCI applications. Overall, this thesis presents the contributions to developing a DD-BCI for real-world use with maximal usability and convenience. The methodologies and findings could further catalyze the exploration of real-world BCIs in more applications.

Functional Magnetic Resonance Imaging (fMRI) has established the field of brain decoding, meaning the prediction of the task that a subject is performing in the MRI scanner, given the corresponding images. This has been quite successful, especially when attempting discrimination between the representations of two or four distinct stimuli across the brains of multiple subjects. However, there are currently only a few studies that deal with ways to improve the scalability of existing brain decoding methodologies, in order for the resulting classifiers to be able to discriminate among tens or hundreds of possible stimuli. Such advances have potential for the creation of rigorous brain-computer interfaces, which could establish a solid communication channel with people in a vegetative state. In this work, I propose and evaluate a series of methods leading to the development of a new data-driven, scalable brain decoding framework that will enable better stimulus discrimination. The methods include: (1) A novel inter-subject spatial feature selection method that can be run using the native brain images of each subject directly, and which is not sensitive to differences in the morphology of the brain of each subject. (2) Three novel data-driven feature selection methods that use statistical association metrics in order to select regions that exhibit similar behaviour across-subjects over the course of a given experiment. The methods aim to promote enhanced exploratory power and are not susceptible to region-specific variations of the haemodynamic response function. (3) Two novel data-driven temporal denoising algorithms that can be used to improve the signal-to-noise ratio of any given task-related fMRI image and which do not impose constraints in either the experimental design nor the nature of the involved stimuli. (4) A thorough evaluation of four intensity normalisation techniques that are commonly used for across-subjects and across-sessions decoding, in order to determine their applicability for across-datasets decoding. (5) A novel feature compression and information recovery method that aims at lowering the system memory requirements for training and testing a large-scale brain decoding model using multiple datasets simultaneously.

The studies in this thesis explored how pain and its relief are represented in the human brain. Pain and relief are important survival signals that motivate escape from danger and search for safety, however, they are often evaluated by subjective descriptions only. Studying how humans learn and adapt to pain and relief allows objective investigation of the information processing and neural circuitry underlying these internal experiences. My research set out to use computational learning models to provide mechanistic explanations for the behavioural and functional neuroimaging data collected in pain/relief learning experiments with independent groups of healthy human participants. With a Pavlovian acute pain conditioning task in Experiment 1, I found that 'associability' (a form of uncertainty signal) had a crucial role in controlling the learning rates of different conditioned responses, and can be used to anatomically dissociate underlying neural systems. Experiment 2 focused on relief learning of terminating a tonic pain stimulus, in which the priority for relief-seeking is in conflict with the general suppression of cognition and attention. I showed that associability during active learning not only controls the relief learning rate, but also correlates with endogenously modulated (reduced) ongoing pain. This finding was confirmed in Experiment 3 using an independent active relief learning paradigm in a complex dynamic environment. Critically, both experiments showed that associability was correlated with responses in the pregenual anterior cingulate cortex (pgACC), a brain region previously implicated in aspects of endogenous pain control related to attention and controllability. This provided a potential computational account of an information-sensitive endogenous analgesic mechanism. In Experiment 4, I explored the implications of endogenous controllability for technology-based pain therapeutics. I designed an adaptive closed-loop system that learned to control pain stimulation using decoded real-time pain representations from the brain. Subjects were shown to actively enhance the discriminability of pain only in the pgACC, and uncertainty during learning again correlated with endogenously modulated pain and were associated with pgACC responses. Together, these studies (i) show the importance of uncertainty in controlling learning during both acute and tonic pain, (ii) describe how uncertainty also flexibly modulates pain to maximise the impact of learning, (iii) illustrate a central role for the pgACC in this process, and (iv) reveal the implications for future technology-based therapeutic systems.

Brain-Computer Interface (BCI) offers the opportunity to paralyzed patients to control their movements without any neuromuscular activity. Signal processing of neuronal activity enables to decode movement intentions. Ability for patient to control an effector is closely linked to this decoding performance. In this study, I tackle a recent way to decode neuronal activity: Deep learning. The study is based on public data extracted by Schalk et al. for BCI Competition IV. Electrocorticogram (ECoG) data from three epileptic patients were recorded. During the experiment setup, the team asked subjects to move their fingers and recorded finger movements thanks to a data glove. An artificial neural network (ANN) was built based on a common BCI feature extraction pipeline made of successive convolutional layers. This network firstly mimics a spatial filtering with a spatial reduction of sources. Then, it realizes a time-frequency analysis and performs a log power extraction of the band-pass filtered signals. The first investigation was on the optimization of the network. Then, the same architecture was used on each subject and the decoding performances were computed for a 6-class classification. I especially investigated the spatial and temporal filtering. Finally, a preliminary study was conducted on prediction of finger movement. This study demonstrated that deep learning could be an effective way to decode brain signal. For 6-class classification, results stressed similar performances as traditional decoding algorithm. As spatial or temporal weights after training are slightly described in the literature, we especially worked on interpretation of weights after training. The spatial weight study demonstrated that the network is able to select specific ECoG channels notified in the literature as the most informative. Moreover, the network is able to converge to the same spatial solution, independently to the initialization. Finally, a preliminary study was conducted on prediction of movement position and gives encouraging results.

The analysis and understanding of physiological and brain signals is critical in order to decode user’s behavioral/neural outcome measures in different domain scenarios. Personal Health-Care agents have been proposed recently in order to monitor and acquire reliable data from daily activities to enhance control participants’ wellbeing, and the quality of life of multiple non-neurotypical participants in clinical lab-controlled studies. The inclusion of new wearable devices with increased and more compact memory requirements,and the possibility to include long-size datasets on the cloud and network-based applications agile the implementation of new improved computational health-care agents. These new enhanced agents are able to provide services including real time health-care,medical monitoring, and multiple biological outcome measures-based alarms for medicaldoctor diagnosis. In this dissertation we will focus on multiple Signal Processing (SP), Machine Learning (ML), Saliency Relevance Maps (SRM) techniques and classifiers with the purpose to enhance the Personal Health-care agents in a multimodal clinical environment. Therefore, we propose the evaluation of current state-of-the-art methods to evaluate the incidence of successful hypertension detection, categorical and emotion stimuli decoding using biosignals. To evaluate the performance of ML, SP, and SRM techniques proposed in this study, wedivide this thesis document in two main implementations: 1) Four different initial pipelines where we evaluate the SP, and ML methodologies included here for an enhanced a) Hypertension detection based on Blood-Volume-Pulse signal (BVP) and Photoplethysmography (PPG) wearable sensors, b) Heart-Rate (HR) and Inter-beat-interval (IBI) prediction using light adaptive filtering for physical exercise/real environments, c) Object Category stimuli decoding using EEG features and features subspace transformations, and d) Emotion recognition using EEG features from recognized datasets. And 2) A complete performance and robust SRM evaluation of a neural-based Emotion Decoding/Recognition pipeline using EEG features from Autism Spectrum Disorder (ASD) groups. This pipeline is presented as a novel assistive system for lab-controlled Face Emotion Recognition (FER) intervention ASD subjects. In this pipeline we include a Deep ConvNet asthe Deep classifier to extract the correct neural information and decode emotions successfully.

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2011.
Cataloged from PDF version of thesis.
Includes bibliographical references.
The field of neuroscience has the potential to address profound questions including explaining how neural activity enables complex behaviors and conscious experience. However, currently the field is a long way from understanding these issues, and progress has been slow. One of the main problems holding back the pace of discovery is that it is still unclear how to interpret neural activity once it has been recorded. This lack of understanding has led to many different data analysis methods, which makes it difficult to evaluate the validity and importance of many reported results. If a clearer understanding of how to interpret neural data existed, it should be much easier to answer other questions about how the brain functions. In this thesis I describe how to use a data analysis method called 'neural population decoding' to analyze data in a way that is potentially more relevant for understanding neural information processing. By applying this method in novel ways to data from several vision experiments, I am able to make several new discoveries, including the fact that abstract category information is coded in the inferior temporal cortex (ITC) and prefrontal cortex (PFC) by dynamic patterns of neural activity, and that when a monkey attends to an object in a cluttered display, the pattern of ITC activity returns to a state that is similar to when the attended object is presented alone. These findings are not only interesting for insights that they give into the content and coding of information in high level visual areas, but they also demonstrate the benefits of using neural population decoding to analyze data. Thus, the methods developed in this thesis should enable more rapid progress toward an algorithmic level understanding of vision and information processing in other neural systems.
by Ethan M. Meyers.
Ph.D.

Thesis: Ph. D. in Neuroscience, Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2016.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 126-170).
The ability to engage in appropriate social interaction is a critical component of daily life that requires integration of multiple neural processes and can be perturbed in numerous psychiatric diseases (Adolphs et al. 2003; Frith et al. 2008). One approach to begin understanding how the brain supports a complex array of social behaviors is to study innate, evolutionarily conserved social behaviors. Observational fear learning is one such social behavior that offers a distinct advantage for survival and is thus highly conserved across various species including rodents (Heyes et al. 1990; Kavaliers et al. 2001), monkeys (Mineka et al. 1984), and humans (Olsson et al. 2007). The data presented in this thesis combines in vivo electrophysiology, optogenetics, and rodent behavior in order to answer a number of questions about the role of the anterior cingulate cortex (ACC) and the basolateral amygdala (BLA) in observational fear learning. We show that both the ACC and the BLA contain neurons that show conditioned responses to the cue and are therefore neural correlates of observational fear learning. We photo-identify neurons within the ACC-BLA network and show that the ACC-BLA network has an enhanced representation of cue information when compared to out of network neurons. In addition, we show that ACC neurons that project to the BLA encode cue information. Next, we inhibit ACC input to the BLA during the cue and show that this impairs observational learning but not classical fear conditioning. Further, inhibition of ACC input to the BLA changes the cue response of a subset of BLA neurons. Lastly, we show that ACC input to the BLA is necessary for normal social interaction. Together, this data provides the first circuit level analysis of observational fear learning. It establishes that the transfer of cue information from the ACC to the BLA plays a causal role in enabling observational learning and that this same input is needed for general social behavior.
by Stephen Azariah Alisop.
Ph. D. in Neuroscience

Thesis (Sc. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2005.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Includes bibliographical references (leaves 87-97).
Sequential processing --- using multiple sensory stimuli to plan and control a set of ordered movements --- is a central aspect of human behavior. Because previous and future movements must be stored during the execution of any movement in a sequence, memory is an indispensable aspect of sequential behavior. To study how memory is used to link sensory inputs to sequential motor outputs, we have used the oculomotor system as a model. We trained monkeys to remember the location of two spatial cues over a brief delay, and then make two eye movements to the remembered locations in the order that they appeared. We explored the role of two different frontal eye movement areas, the frontal and supplementary eye fields (FEF and SEF) during this memory delay. While both the FEF and SEF have shown to be important for sequential behavior, their individual roles are unknown. Here, using physiology, we show that the FEF is important for storing the location of multiple cues and their order in memory. In the SEF, we show that memory period stimulation can affect the order of a sequence, changing the goal of the entire sequence but not the individual movement components.
(cont.) Thus, both areas appear to play complementary roles in sequential planning: the FEF stores target locations, while the SEF appears to control the order of a response sequence, coding entire sequences without affecting the locations of the intermediate targets. This work bears on several outstanding questions in the field. It clarifies the individual roles of the FEF and SEF during sequencing: the FEF may serve as a buffer for multiple memories while the SEF plays a role in organizing movement sequences. It relates several prior SEF results, suggesting that a primary role of SEF may be to specify movements by their goal. Finally, we suggest that this goal-centered scheme may be a fundamental way that many different types of movements are encoded.
by Hark H. Histed.
Sc.D.

Functional brain images are extraordinarily rich data sets that reveal distributed brain networks engaged in a wide variety of cognitive operations. It is a substantial challenge both to create models of cognition that mimic behavior and underlying cognitive processes and to choose a suitable analytic method to identify underlying brain networks. Most of the contemporary techniques used in analyses of functional neuroimaging data are based on univariate approaches in which single image elements (i.e. voxels) are considered to be computationally independent measures. Beyond univariate methods (e.g. statistical parametric mapping), multivariate approaches, which identify a network across all regions of the brain rather than a tessellation of regions, are potentially well suited for analyses of brain imaging data. A multivariate method (e.g. partial least squares) is a computational strategy that determines time-varying distributed patterns of the brain (as a function of a cognitive task). Compared to its univariate counterparts, a multivariate approach provides greater levels of sensitivity and reflects cooperative interactions among brain regions. Thus, by considering information across more than one measuring point, additional information on brain function can be revealed. Similarly, by considering information across more than one measuring technique, the nature of underlying cognitive processes become well-understood. Cognitive processes have been investigated in conjunction with multiple neuroimaging modalities (e.g. fMRI, sMRI, EEG, DTI), whereas the typical method has been to analyze each modality separately. Accordingly, little work has been carried out to examine the relation between different modalities. Indeed, due to the interconnected nature of brain processing, it is plausible that changes in one modality locally or distally modulate changes in another modality. This thesis focuses on multivariate and multimodal methods of image analysis applied to various cognitive questions. These methods are used in order to extract features that are inaccessible using univariate / unimodal analytic approaches. To this end, I implemented multivariate partial least squares analysis in study I and II in order to identify neural commonalities and differences between the available and accessible information in memory (study I), and also between episodic encoding and episodic retrieval (study II). Study I provided evidence of a qualitative differences between availability and accessibility signals in memory by linking memory access to modality-independent brain regions, and availability in memory to elevated activity in modality-specific brain regions. Study II provided evidence in support of general and specific memory operations during encoding and retrieval by linking general processes to the joint demands on attentional, executive, and strategic processing, and a process-specific network to core episodic memory function. In study II, III, and IV, I explored whether the age-related changes/differences in one modality were driven by age-related changes/differences in another modality. To this end, study II investigated whether age-related functional differences in hippocampus during an episodic memory task could be accounted for by age-related structural differences. I found that age-related local structural deterioration could partially but not entirely account for age-related diminished hippocampal activation. In study III, I sought to explore whether age-related changes in the prefrontal and occipital cortex during a semantic memory task were driven by local and/or distal gray matter loss. I found that age-related diminished prefrontal activation was driven, at least in part, by local gray matter atrophy, whereas the age-related decline in occipital cortex was accounted for by distal gray matter atrophy. Finally, in study IV, I investigated whether white matter (WM) microstructural differences mediated age-related decline in different cognitive domains. The findings implicated WM as one source of age-related decline on tasks measuring processing speed, but they did not support the view that age-related differences in episodic memory, visuospatial ability, or fluency were strongly driven by age-related differences in white-matter pathways. Taken together, the architecture of different aspects of episodic memory (e.g. encoding vs. retrieval; availability vs. accessibility) was characterized using a multivariate partial least squares. This finding highlights usefulness of multivariate techniques in guiding cognitive theories of episodic memory. Additionally, competing theories of cognitive aging were investigated by multimodal integration of age-related changes in brain structure, function, and behavior. The structure-function relationships were specific to brain regions and cognitive domains. Finally, we urged that contemporary theories on cognitive aging need to be extended to longitudinal measures to be further validated.

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 129-135).
Brain-machine interfaces (BMI) aim to enable motor function in individuals with neurological injury or disease, by recording the neural activity, mapping or 'decoding' it into a motor command, and then controlling a device such as a computer interface or robotic arm. BMI research has largely focused on the problem of restoring the original motor function. The goal therefore has been to achieve a performance close to that of the healthy individual. There have been compelling proof of concept demonstrations of the utility of such BMIs in the past decade. However, performance of these systems needs to be significantly improved before they become clinically viable. Moreover, while developing high-performance BMIs with the goal of matching the original motor function is indeed valuable, a compelling goal is that of designing BMIs that can surpass original motor function. In this thesis, we first develop a novel real-time BMI for restoration of natural motor function. We then introduce a BMI architecture aimed at enhancing original motor function. We implement both our designs in rhesus monkeys. To facilitate the restoration of lost motor function, BMIs have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Moreover, both target and trajectory information are encoded in the motor cortical areas. These suggest that BMIs should be designed to combine these principal aspects of movement. We develop a novel two-stage BMI to decode jointly the target and trajectory of a reaching movement. First, we decode the intended target from neural spiking activity before movement initiation. Second, we combine the decoded target with the spiking activity during movement to estimate the trajectory. To do so, we use an optimal feedback-control design that aims to emulate the sensorimotor processing underlying actual motor control and directly processes the spiking activity using point process modeling in real time. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. This BMI also performs significantly better than linear regression approaches demonstrating the advantage of a design that more closely mimics the sensorimotor system.
(cont.) While restoring the original motor function is indeed important, a compelling goal is the development of a truly "intelligent" BMI that can transcend such function by considering the higherlevel goal of the motor activity, and reformulating the motor plan accordingly. This would allow, for example, a task to be performed more quickly than possible by natural movement, or more efficiently than originally conceived. Since a typical motor activity consists of a sequence of planned movements, such a BMI must be capable of analyzing the complete sequence before action. As such its feasibility hinges fundamentally on whether all elements of the motor plan can be decoded concurrently from working memory. Here we demonstrate that such concurrent decoding is possible. In particular, we develop and implement a real-time BMI that accurately and simultaneously decodes in advance a sequence of planned movements from neural activity in the premotor cortex. In our experiments, monkeys were trained to add to working memory, in order, two distinct target locations on a screen, then move a cursor to each, in sequence. We find that the two elements of the motor plan, corresponding to the two targets, are encoded concurrently during the working memory period. Additionally, and interestingly, our results reveal: that the elements of the plan are encoded by largely disjoint subpopulations of neurons; that surprisingly small subpopulations are sufficient for reliable decoding of the motor plan; and that the subpopulation dedicated to the first target and their responses are largely unchanged when the second target is added to working memory, so that the process of adding information does not compromise the integrity of existing information. The results have significant implications for the architecture and design of future generations of BMIs with enhanced motor function capabilities.
by Maryam Modir Shanechi.
Ph.D.

Machine learning algorithms have been successfully applied to learning classifiers in many domains such as computer vision, fraud detection, and brain image analysis. Typically, classifiers are trained to predict a class value given a set of labeled training data that includes all possible class values, and sometimes additional unlabeled training data. Little research has been performed where the possible values for the class variable include values that have been omitted from the training examples. This is an important problem setting, especially in domains where the class value can take on many values, and the cost of obtaining labeled examples for all values is high. We show that the key to addressing this problem is not predicting the held-out classes directly, but rather by recognizing the semantic properties of the classes such as their physical or functional attributes. We formalize this method as zero-shot learning and show that by utilizing semantic knowledge mined from large text corpora and crowd-sourced humans, we can discriminate classes without explicitly collecting examples of those classes for a training set. As a case study, we consider this problem in the context of thought recognition, where the goal is to classify the pattern of brain activity observed from a non-invasive neural recording device. Specifically, we train classifiers to predict a specific concrete noun that a person is thinking about based on an observed image of that person’s neural activity. We show that by predicting the semantic properties of the nouns such as “is it heavy?” and “is it edible?”, we can discriminate concrete nouns that people are thinking about, even without explicitly collecting examples of those nouns for a training set. Further, this allows discrimination of certain nouns that are within the same category with significantly higher accuracies than previous work. In addition to being an important step forward for neural imaging and braincomputer- interfaces, we show that the zero-shot learning model has important implications for the broader machine learning community by providing a means for learning algorithms to extrapolate beyond their explicit training set.

An ever-increasing number of functional magnetic resonance imaging (fMRI) studies are now using information-based multi-voxel pattern analysis (MVPA) techniques to decode mental states. In doing so, they achieve a significantly greater sensitivity compared to when they use univariate analysis frameworks. Two most prominent MVPA methods for information mapping are searchlight decoding and classifier weight mapping. The new MVPA brain mapping methods, however, have also posed new challenges for analysis and statistical inference on the group level. In this thesis, I discuss why the usual procedure of performing t-tests on MVPA derived information maps across subjects in order to produce a group statistic is inappropriate. I propose a fully nonparametric solution to this problem, which achieves higher sensitivity than the most commonly used t-based procedure. The proposed method is based on resampling methods and preserves the spatial dependencies in the MVPA-derived information maps. This enables to incorporate a cluster size control for the multiple testing problem. Using a volumetric searchlight decoding procedure and classifier weight maps, I demonstrate the validity and sensitivity of the new approach using both simulated and real fMRI data sets. In comparison to the standard t-test procedure implemented in SPM8, the new results showed a higher sensitivity and spatial specificity. The second goal of this thesis is the comparison of the two widely used information mapping approaches -- the searchlight technique and classifier weight mapping. Both methods take into account the spatially distributed patterns of activation in order to predict stimulus conditions, however the searchlight method solely operates on the local scale. The searchlight decoding technique has furthermore been found to be prone to spatial inaccuracies. For instance, the spatial extent of informative areas is generally exaggerated, and their spatial configuration is distorted. In this thesis, I compare searchlight decoding with linear classifier weight mapping, both using the formerly proposed non-parametric statistical framework using a simulation and ultra-high-field 7T experimental data. It was found that the searchlight method led to spatial inaccuracies that are especially noticeable in high-resolution fMRI data. In contrast, the weight mapping method was more spatially precise, revealing both informative anatomical structures as well as the direction by which voxels contribute to the classification. By maximizing the spatial accuracy of ultra-high-field fMRI results, such global multivariate methods provide a substantial improvement for characterizing structure-function relationships.

The human brain acts as an intelligent sensor by helping in effective signal communication and execution of logical functions and instructions, thus, coordinating all functions of the human body. More importantly, it shows the potential to combine prior knowledge with adaptive learning, thus ensuring constant improvement. These qualities help the brain to interact efficiently with both, the body (brain-body) as well as the environment (brain-environment). This dissertation attempts to apply the brain-body-environment interactions (BBEI) to elevate human existence and enhance our day-to-day experiences. For instance, when one stepped out of the house in the past, one had to carry keys (for unlocking), money (for purchasing), and a phone (for communication). With the advent of smartphones, this scenario changed completely and today, it is often enough to carry just one's smartphone because all the above activities can be performed with a single device. In the future, with advanced research and progress in BBEI interactions, one will be able to perform many activities by dictating it in one's mind without any physical involvement. This dissertation aims to shift the paradigm of existing brain-computer-interfaces from just ‘control' to ‘monitor, control, enhance, and restore' in three main areas - healthcare, transportation safety, and cryptography. In healthcare, measures were developed for understanding brain-body interactions by correlating cerebral autoregulation with brain signals. The variation in estimated blood flow of brain (obtained through EEG) was detected with evoked change in blood pressure, thus, enabling EEG metrics to be used as a first hand screening tool to check impaired cerebral autoregulation. To enhance road safety, distracted drivers' behavior in various multitasking scenarios while driving was identified by significant changes in the time-frequency spectrum of the EEG signals. A distraction metric was calculated to rank the severity of a distraction task that can be used as an intuitive measure for distraction in people - analogous to the Richter scale for earthquakes. In cryptography, brain-environment interactions (BBEI) were qualitatively and quantitatively modeled to obtain cancelable biometrics and cryptographic keys using brain signals. Two different datasets were used to analyze the key generation process and it was observed that neurokeys established for every subject-task combination were unique, consistent, and can be revoked and re-issued in case of a breach. This dissertation envisions a future where humans and technology are intuitively connected by a seamless flow of information through ‘the most intelligent sensor', the brain.

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2003.
Includes bibliographical references (p. 100-104).
There is broad agreement that the hippocampus is crucially involved in the formation of richly-detailed, long term memories of events in humans. A key aspect of such memories is the temporal order and spatial context of the events experienced. Evidence from a wide variety of behavioral and electrophysiological experiments indicates that the rodent hippocampal spatial memory system is a model system for studying this type of memory in humans. Here, we develop a new combinatorial method for analyzing sequential firing patterns involving an arbitrary number of neurons based on relative time order. We then apply this method to decode memories of sequential spatial experience in the rat hippocampus during slow wave sleep. Specificaly, rats are trained to repeatedly run through a sequence of spatial receptive fields ("place fields") of hippocampal CA1 "place cells" in a fixed temporal order. The spiking activity of many such individual cells is recorded before (PRE), during (RUN), and after (POST) this experience. By treating each place field traversed as an individual event, the rat's experience in RUN can be represeted by the resulting sequence of place fields traversed, and therefore by the activity of the corresponding place cells. Then to characterize the extent to which the sequential nature of the RUN experience has been encoded into memory, we search for firing patterns related to the RUN sequence in POST. To do so, we develop a method that statistically quantifies the similarity between any desired "reference sequence" (here chosen to be the RUN sequence) and arbitrary temporal firing patterns. We find that the RUN sequence is repeatedly re-expressed during POST slow wave sleep in brief bursts involving four or more cells firing in order, but not so during PRE.
(cont.) This provides direct neural evidence of the rapid learning of extended spatial sequences experienced in RUN. The results may shed light on the encoding of memories of events in time ("episodic memories") in humans. Furthermore, the multiple spike train analysis method developed here is general and could be applied to many other neural systems in many different experimental conditions.
by Albert K. Lee.
Ph.D.

L'attention visuospatiale est un mécanisme de sélection et de traitement d'information qui se manifeste explicitement par l'orientation de la tête ou du regard. En anticipation d'une nouvelle information, le foyer de l'attention s'oriente implicitement en vision périphérique pour dissocier l'orientation du regard et du foyer implicite vers deux emplacements distincts. Dans cette situation, la réaction à une cible qui apparaît à l'emplacement du foyer implicite s'améliore par rapport aux autres cibles qui pourraient s'afficher dans un emplacement non-attendu. La problématique de la thèse est d'étudier comment détecter l'emplacement du foyer de l'attention implicite par décodage de l'activité cérébrale mesurée en électro-encéphalographie (EEG) avant l'affichage d'une cible visuelle dans 3 expériences réalisées chez des sujets sains. La première expérience aborde la problématique dans une condition où l'indication sur l'emplacement de la cible est globalement non-informative pour les sujets. Cependant, leur activité cérébrale suggère que ce type d'indication a tendance à induire un état d'alerte, de préparation ou d'orientation de l'attention dans le temps plutôt que dans l'espace. En lien avec ce résultat, la deuxième expérience aborde la problématique dans une condition ambiguë où l'attention du sujet s'oriente vers un emplacement sans lien systématique avec le contenu des indications
Visuospatial attention is an information selection and processing mechanism whose overt manifestations consist of head or gaze shifts. In anticipation to new information, the focus of attention can also covertly shift to peripheral vision to share attention between two distinct locations: the overt one (center of gaze) and the covert one in periphery. In such a situation, the reaction to a target appearing at the focus of attention is enhanced with respect to targets appearing at unattended locations. This thesis addresses the problem of detecting the location of covert attention by decoding neural activity measured by electroencephalography (EEG) before target onset in 3 experiments on healthy subjects. The first experiment uses visuospatial cues that are non-informative about the target location. However, the neural activity reflects that non-informative cues tend to bring the subjects into a state related to alertness, motor preparation or temporal expectation rather than a spatial shift of attention. According to this result, the second experiment uses an ambiguous precueing condition in which the sujet's attention is shifted to spatial locations which bear a non-systematic relation to the information contained in the cues. With these ambiguous cues, we find that the proportion of targets displayed at unattended locations is equivalent to a non-informative condition, and that reaction speed and accuracy are dramatically impacted

Brain neural activity generates electrical discharges, which manifest as electrical and magnetic potentials around the scalp. Those potentials can be registered with magnetoencephalography (MEG) and electroencephalography (EEG) devices. Data acquired by M/EEG is extremely difficult to work with due to the inherent complexity of underlying brain processes and low signal-to-noise ratio (SNR). Machine learning techniques have to be employed in order to reveal the underlying structure of the signal and to understand the brain state. This thesis explores a diverse range of machine learning techniques which model the structure of M/EEG data in order to decode the mental state. It focuses on measuring a subject's variability and on modeling intrasubject variability. We propose to measure subject variability with a spectral clustering setup. Further, we extend this approach to a unified classification framework based on Laplacian regularized support vector machine (SVM). We solve the issue of intrasubject variability by employing a model with latent variables (based on a latent SVM). Latent variables describe transformations that map samples into a comparable state. We focus mainly on intrasubject experiments to model temporal misalignment.

Le Brain Computer Interfaces (BCI) invasive permettono di restituire la mobilità a pazienti che hanno perso il controllo degli arti: ciò avviene attraverso la decodifica di segnali bioelettrici prelevati da aree corticali di interesse al fine di guidare un arto prostetico. La decodifica dei segnali neurali è quindi un punto critico nelle BCI, richiedendo lo sviluppo di algoritmi performanti, affidabili e robusti. Tali requisiti sono soddisfatti in numerosi campi dalle Deep Neural Networks, algoritmi adattivi le cui performance scalano con la quantità di dati forniti, allineandosi con il crescente numero di elettrodi degli impianti. Impiegando segnali pre-registrati dalla corteccia di due macachi durante movimenti di reach-to-grasp verso 5 oggetti differenti, ho testato tre basilari esempi notevoli di DNN – una rete densa multistrato, una Convolutional Neural Network (CNN) ed una Recurrent NN (RNN) – nel compito di discriminare in maniera continua e real-time l’intenzione di movimento verso ciascun oggetto. In particolare, è stata testata la capacità di ciascun modello di decodificare una generica intenzione (single-class), la performance della migliore rete risultante nel discriminarle (multi-class) con o senza metodi di ensemble learning e la sua risposta ad un degrado del segnale in ingresso. Per agevolarne il confronto, ciascuna rete è stata costruita e sottoposta a ricerca iperparametrica seguendo criteri comuni. L’architettura CNN ha ottenuto risultati particolarmente interessanti, ottenendo F-Score superiori a 0.6 ed AUC superiori a 0.9 nel caso single-class con metà dei parametri delle altre reti e tuttavia maggior robustezza. Ha inoltre mostrato una relazione quasi-lineare con il degrado del segnale, priva di crolli prestazionali imprevedibili. Le DNN impiegate si sono rivelate performanti e robuste malgrado la semplicità, rendendo eventuali architetture progettate ad-hoc promettenti nello stabilire un nuovo stato dell’arte nel controllo neuroprotesico.

Brain decoding involves the determination of a subject’s cognitive state or an associated stimulus from functional neuroimaging data measuring brain activity. In this setting the cognitive state is typically characterized by an element of a finite set, and the neuroimaging data comprise voluminous amounts of spatiotemporal data measuring some aspect of the neural signal. The associated statistical problem is one of classification from high-dimensional data. We explore the use of functional principal component analysis, mutual information networks, and persistent homology for examining the data through exploratory analysis and for constructing features characterizing the neural signal for brain decoding. We review each approach from this perspective, and we incorporate the features into a classifier based on symmetric multinomial logistic regression with elastic net regularization. The approaches are illustrated in an application where the task is to infer from brain activity measured with magnetoencephalography (MEG) the type of video stimulus shown to a subject.
Graduate

Cortical representations of rhythmic and discrete movements are analyzed and used to create a novel neural decoding algorithm for brain-machine interfaces. This algorithm is then implemented to decode both cyclic movements and reach-and-hold movements in awake behaving rhesus macaques using their cortical activity alone. Finally, a healthy macaque wears and controls a lower body exoskeleton using the developed BMIas a proof of concept of a brain-controlled neuroprosthetic device for locomotion.

Dissertation

Through linguistic abilities, our brain can comprehend and produce an infinite number of new sentences constructed from a finite set of words. Although recent research has uncovered the neural representation of semantics during comprehension of isolated words or adjective-noun phrases, the neural representation of the words during utterance planning is less understood. We apply existing machine learning methods to Magnetoencephalography (MEG) data recorded during a picture naming experiment, and predict the semantic properties of uttered words before they are said. We explore the representation of concepts over time, under controlled tasks, with varying compositional requirements. Our results imply that there is enough information in brain activity recorded by MEG to decode the semantic properties of the words during utterance planning. Also, we observe a gradual improvement in the semantic decoding of the first uttered word, as the participant is about to say it. Finally, we show that, compared to non-compositional tasks, planning to compose an adjective-noun phrase is associated with an enhanced and sustained representation of the noun. Our results on the neural mechanisms of basic compositional structures are a small step towards the theory of language in the brain.
Graduate

博士
國立交通大學
資訊科學與工程研究所
104
Visual encoding and decoding are crucial aspects in investigating the neural representation of visual information in the human brain. Once a decoding and encoding model is constructed, it can be used to decode the low-level visual patterns or higher-level visual stimuli such as facial images. Such decoding requires the transformation from a neural representation to a perceptual representation, which lays essential foundation to hierarchical face processing. Previous studies have proposed models for low-level visual processing. However, how the brain process the high-level perception remains an unsolved question. There are three studies in the thesis and three corresponding experiments were conducted for each of the study in the thesis. In the first study, we proposed a bidirectional model for decoding and encoding of visual stimulus based on manifold representation of the temporal and spatial information extracted from magnetoencephalographic (MEG) data. In the proposed decoding process, principal component analysis is applied to extract temporal principal components (TPCs) from the visual cortical activity estimated by a beamforming method. The spatial distribution of each TPC is in a high-dimensional space and can be mapped to the corresponding spatiotemporal component (STC) on a low-dimensional manifold. Once the linear mapping between the STC and the wavelet coefficients of the stimulus image is determined, the decoding process can synthesize an image resembling the stimulus image. The encoding process is performed by reversing the mapping or transformation in the decoding model and can predict the spatiotemporal brain activity from a stimulus image. In our first experiments using visual stimuli containing eleven combinations of checkerboard patches, the information of spatial layout in the stimulus image was revealed in the embedded manifold. The correlation between the reconstructed and original images was 0.71 and the correlation map between the predicted and original brain activity was highly correlated to the map between the original brain activity for different stimuli (r=0.89). In the second study, we applied a decoding method based on the decoding and encoding model to face representations. A low-dimensional neural manifold was constructed using a set of single-trial brain activity data evoked by stimuli with basic face viewpoints and gaze directions. As a perceptual representation with synthesis property, this manifold was able to predict composite viewpoints and directions from brain activity. In the second experiments, when facial images with varying viewpoints and gaze-directions were used as the experimental stimuli, the M170 component in occipital face area and the right superior temporal sulcus gave accurate prediction for face viewpoints and gaze directions, respectively. In the third study, we proposed a supervised locally linear embedding method to construct the embedded manifold from brain activity, taking into account similarities between corresponding stimuli. In our experiments, photographic portraits were used as visual stimuli and brain activity was calculated from MEG data using a source localization method. The results of 10×10-fold cross-validation revealed a strong correlation between manifolds of brain activity and the orientation of faces in the presented images, suggesting that high-level information related to image content can be revealed in the brain responses represented in the manifold. These results suggest that the temporal component is important in visual processing and manifolds can well represent the information related to visual perception. In addition, the proposed neural manifold method can be used to construct an effective perceptual representation for face processing and is applicable to investigation into the inherent patterns of brain activity.

Locked-in syndrome (LIS) is a condition characterized by total or near-total paralysis with preserved cognitive and somatosensory function. For the locked-in, brain-machine interfaces (BMI) provide a level of restored communication and interaction with the world, though this technology has not reached its fullest potential. Several streams of research explore improving BMI performance but very little attention has been given to the paradigms implemented and the resulting constraints imposed on the users. Learning new mental tasks, constant use of external stimuli, and high attentional and cognitive processing loads are common demands imposed by BMI. These paradigm constraints negatively affect BMI performance by locked-in patients. In an effort to develop simpler and more reliable BMI for those suffering from LIS, this dissertation explores using error-related potentials, the neural correlates of error awareness, as an access pathway for adaptive decoding and direct volitional control. In the first part of this thesis we characterize error-related local field potentials (eLFP) and implement a real-time decoder error detection (DED) system using eLFP while non-human primates controlled a saccade BMI. Our results show specific traits in the eLFP that bridge current knowledge of non-BMI evoked error-related potentials with error-potentials evoked during BMI control. Moreover, we successfully perform real-time DED via, to our knowledge, the first real-time LFP-based DED system integrated into an invasive BMI, demonstrating that error-based adaptive decoding can become a standard feature in BMI design. In the second part of this thesis, we focus on employing electroencephalography error-related potentials (ErrP) for direct volitional control. These signals were employed as an indicator of the user’s intentions under a closed-loop binary-choice robot reaching task. Although this approach is technically challenging, our results demonstrate that ErrP can be used for direct control via binary selection and, given the appropriate levels of task engagement and agency, single-trial closed-loop ErrP decoding is possible. Taken together, this work contributes to a deeper understanding of error-related potentials evoked during BMI control and opens new avenues of research for employing ErrP as a direct control signal for BMI. For the locked-in community, these advancements could foster the development of real-time intuitive brain-machine control.

Brain Computer Interfaces (BCI) are devices that translate acquired neural signals to command and control signals. Applications of BCI include neural rehabilitation and neural prosthesis (thought controlled wheelchair, thought controlled speller etc.) to aid patients with disabilities and to augment human computer interaction. A successful practical BCI requires a faithful acquisition modality to record high quality neural signals; a signal processing system to construct appropriate features from these signals; and an algorithm to translate these features to appropriate outputs. Intracortical recordings like local field potentials provide reliable high SNR signals over long periods and suit BCI applications well. However, the non-stationarity of neural signals poses a challenge in robust decoding of subject behavior. Most BCI research focuses either on developing daily re-calibrated decoders that require exhaustive training sessions; or on providing cross-validation results. Such results ignore the variation of signal characteristics over different sessions and provide an optimistic estimate of BCI performance. Specifically, traditional BCI algorithms fail to perform at the same level on chronological data recordings. Neural signals are susceptible to variations in signal characteristics due to changes in subject behavior and learning, and variability in electrode characteristics due to tissue interactions. While training day-specific BCI overcomes signal variability, BCI re-training causes user frustration and exhaustion. This dissertation presents contributions to solve these challenges in BCI research. Specifically, we developed decoders trained on a single recording session and applied them on subsequently recorded sessions. This strategy evaluates BCI in a practical scenario with a potential to alleviate BCI user frustration without compromising performance. The initial part of the dissertation investigates extracting features that remain robust to changes in neural signal over several days of recordings. It presents a qualitative feature extraction technique based on ranking the instantaneous power of multichannel data. These qualitative features remain robust to outliers and changes in the baseline of neural recordings, while extracting discriminative information. These features form the foundation in developing robust decoders. Next, this dissertation presents a novel algorithm based on the hypothesis that multiple neural spatial patterns describe the variation in behavior. The presented algorithm outperforms the traditional methods in decoding over chronological recordings. Adapting such a decoder over multiple recording sessions (over 6 weeks) provided > 90% accuracy in decoding eight movement directions. In comparison, performance of traditional algorithms like Common Spatial Patterns deteriorates to 16% over the same time. Over time, adaptation reinforces some spatial patterns while diminishing others. Characterizing these spatial patterns reduces model complexity without user input, while retaining the same accuracy levels. Lastly, this dissertation provides an algorithm that overcomes the variation in recording quality. Chronic electrode implantation causes changes in signal-to-noise ratio (SNR) of neural signals. Thus, some signals and their corresponding features available during training become unavailable during testing and vice-versa. The proposed algorithm uses prior knowledge on spatial pattern evolution to estimate unknown neural features. This algorithm overcomes SNR variations and provides up to 93% decoding of eight movement directions over 6 weeks. Since model training requires only one session, this strategy reduces user frustration. In a practical closed-loop BCI, the user learns to produce stable spatial patterns, which improves performance of the proposed algorithms.
text

碩士
國立陽明大學
生物醫學工程學系
105
Neuroscience research has been paid more and more attention in recent years, and one of the research is the brain machine interfaces (BMIs). In BMIs, in addition to solve the curse of dimensionality, the accuracy and stability of the decoding algorithm is also important. The architecture of the signal acquisition in BMIs could be divided into invasive and non-invasive. For the invasive BMIs, researches are based on intracranial EEG (iEEG), electrocorticography (ECoG), intracortical local field potentials (LFPs), or neuronal spiking activity (AP). Several neural decoding algorithms have successfully converted brain signals into commands to control a computer cursor and prosthetic devices. A majority of decoding methods, such as population vector algorithm (PVA), optimal linear estimator (OLE), principal component analysis (PCA), partial least squares (PLS), Wiener filter (WF), Kalman filter (KF), Bayesian filter (BF), neural network (NN), and so on. Furthermore, depending on the type of the input, there are spike-driven and LFP-driven decoders. Both of them have high accuracy, and therefore are frequently used. This study adopts kernel sliced inverse regression (kSIR) to predict intended forelimb movement trajectories according to the recorded neurons from primary motor (M1) cortex. kSIR is a novel decoding algorithm, and is useful even when signals obtained from a smaller number of neurons. The LFPs was adopted to improve the performance, and hoped to improve the accuracy and stability. Results showed that the stability and accuracy were significantly improved, where the accuracy (R squared, Mean ± SEM) was improved from ("0.88"±"0.059" ) to ("0.93"±"0.061)" for x-axis and ("0.90"±"0.022)" to ("0.97"±"0.024)" for y-axis. And the adjustment of kernel bandwidth is necessary due to the deviation of observed distribution from Gaussian. Therefore, an adaptive architecture was applied to adjust the parameter using in kSIR. The superiority of this multi-input kSIR was obviously. To sum up, simultaneously using both spikes and LFPs would make the decoder more robust and accurate, even when met conditions with sparse neural information.

An ever-increasing number of functional magnetic resonance imaging (fMRI) studies are now using information-based multi-voxel pattern analysis (MVPA) techniques to decode mental states. In doing so, they achieve a significantly greater sensitivity compared to when they use univariate analysis frameworks. Two most prominent MVPA methods for information mapping are searchlight decoding and classifier weight mapping. The new MVPA brain mapping methods, however, have also posed new challenges for analysis and statistical inference on the group level. In this thesis, I discuss why the usual procedure of performing t-tests on MVPA derived information maps across subjects in order to produce a group statistic is inappropriate. I propose a fully nonparametric solution to this problem, which achieves higher sensitivity than the most commonly used t-based procedure. The proposed method is based on resampling methods and preserves the spatial dependencies in the MVPA-derived information maps. This enables to incorporate a cluster size control for the multiple testing problem. Using a volumetric searchlight decoding procedure and classifier weight maps, I demonstrate the validity and sensitivity of the new approach using both simulated and real fMRI data sets. In comparison to the standard t-test procedure implemented in SPM8, the new results showed a higher sensitivity and spatial specificity. The second goal of this thesis is the comparison of the two widely used information mapping approaches -- the searchlight technique and classifier weight mapping. Both methods take into account the spatially distributed patterns of activation in order to predict stimulus conditions, however the searchlight method solely operates on the local scale. The searchlight decoding technique has furthermore been found to be prone to spatial inaccuracies. For instance, the spatial extent of informative areas is generally exaggerated, and their spatial configuration is distorted. In this thesis, I compare searchlight decoding with linear classifier weight mapping, both using the formerly proposed non-parametric statistical framework using a simulation and ultra-high-field 7T experimental data. It was found that the searchlight method led to spatial inaccuracies that are especially noticeable in high-resolution fMRI data. In contrast, the weight mapping method was more spatially precise, revealing both informative anatomical structures as well as the direction by which voxels contribute to the classification. By maximizing the spatial accuracy of ultra-high-field fMRI results, such global multivariate methods provide a substantial improvement for characterizing structure-function relationships.

abstract: In most of the work using event-related potentials (ERPs), researchers presume the function of specific components based on the careful manipulation of experimental factors, but rarely report direct evidence supporting a relationship between the neural signal and other outcomes. Perhaps most troubling is the lack of evidence that ERPs correlate with related behavioral outcomes which should result, at least in part, from the neural processes that ERPs capture. One such example is the NoGo-N2 component, an ERP component elicited in Go/NoGo paradigms. There are two primary theories regarding the functional significance of this component in this context: that the signal represents response inhibition and that the component reflects conflict. In this paper, a trial-level method of analysis for the relationship between ERP component potentials and downstream behavioral outcomes (in this case, response accuracy) using a multi-level modeling framework is proposed to provide discriminatory evidence for one of these theories. Following a description of the research on the NoGo-N2, preliminary data supporting the conflict monitoring theory are presented, noting important limitations. Next, an EEG simulation study is presented in which NoGo-N2 data are generated with a known relationship to fabricated reaction time data, showing that, with added levels of complexity and noise within the data, the MLM approach is consistently successful at extracting the known relationships that occur in real NoGo-N2 data. Next, using independent components analysis (ICA) to extract spatiotemporal components that best represent the signal of interest, a well-powered analysis of the relationship between the NoGo-N2 and response accuracy is used to provide strong discriminatory evidence for the conflict monitoring theory of the NoGo-N2. Finally, implications for the NoGo-N2, as well as all ERP components, are discussed with a focus on how this approach can and should be used. the paper concludes with potential expansions of this approach to areas beyond identifying the function of ERP components.
Dissertation/Thesis
Doctoral Dissertation Psychology 2019

Tese de doutoramento em Engenharia Biomédica, apresentada à Faculdade de Ciências e Tecnologia da Universidade de Coimbra
The physiological activity of nervous system is constantly changing in response to contextual changes, which shapes our perception, emotions and cognitive reasoning. Many researchers, clinicians and health professionals have become increasingly interested in translating the understanding of the function of human nervous system and behavior into health technology and human performance. A brain-computer interface (BCI) system uses neurophysiological signals originating in the brain to activate or deactivate external devices or computers. In addition to the assistive functions as communication and control to injured subjects, BCIs offer the opportunity for enhancing neural functions and developing therapies for neural disabilities when applied as a neurofeedback (NF) approach. NF technology allows the participants to observe what their brain is doing in real time. This enables to learn, in an operant manner, how to control brain activity, which might have potential therapeutic impact in disorders of anxiety, attention and motor performance, for example. The voluntary control of brain activity is trained by mean of mental tasks typically known to influence a specific brain network, as for example mental calculation, motor imagery and emotional recalling. Methods to better determine the nature of the brain activity dynamics and plasticity through the formation of self-regulation are still under development. Typically, self-regulation of brain activity is based on binary control: the users try to increase or to decrease/abolish a given brain activity modulation. This thesis presents a research effort developed with the main goal of increase the intrinsic levels of control within each class of volitional brain activity control. Here, we explored the hypothesis of achieve up to three control levels of hMT+/V5 activity using visual motion imagery strategies with different number of motion alternations. We took advantage of the specific recruitment of this brain region to visualized and imagined motion features to go from binary to multilevel/parametric voluntary brain activity control. The performed tests were primarily based on 3 Tesla functional magnetic resonance imaging (fMRI) data and then the proposed brain activity self-regulation approach was also studied using electroencephalography (EEG). Furthermore, the high-resolution 7 Tesla fMRI technique was used to explore with more detail the functional properties of the hMT+/V5 brain region during visual motion perception and to propose a future work to study the possibility of multilevel modulation of functional connectivity. iv The strategies to achieve self-regulation of brain activity can be applied not only in NF approaches but also to achieve more direct and flexible assistive BCI systems. Thus, the increase of volitional brain activity control levels might not only allow for more precise NF but also for improved assistive BCI systems. The research work presented in this thesis may contribute to the technical improvements of BCI systems, to increase their application range and to the understanding of neural mechanisms underlying cognitive control. First, the feasibility of training healthy volunteers to up-regulate and down-regulate the activity of the hMT+/V5 complex using fMRI-based NF and visual motion imagery strategies was tested. We show that hMT+/V5 activity can be volitionally modulated by focused imagery and that a specific brain network is recruited during visual motion imagery leading to successful NF training. The results presented contribute also to the debate on the relative value of sensory versus default-mode brain regions in the NF clinical applications. Then, the hypothesis whether more than two modulation levels can be achieved in a single brain region was tested. Participants performed three distinct imagery tasks with different numbers of motion alternation during fMRI-based NF training. Three control levels (two up-regulation levels and one down-regulation level) of brain activity in the hMT+/V5 complex were achieved. Based on our results we suggest that it is possible to design a multilevel system of control based on brain activity self-regulation of a specific brain region and using similar strategies across participants. Empirical contributions to the comparison between the binary and multilevel control processes and between the passive imagery and the active imagery processes assisted by feedback are also provided. The hypothesis whether visual motion imagery can be used as a tool to at least achieve multiclass (>2) EEG-based BCI was tested. The imagery strategies previously studied with fMRI-based NF were applied. We expected that this would be achieved by detection of differential modulations, regardless of their polarity, in the EEG domain. EEG signals were acquired during passive visual motion imagery in order to identify the evoked brain activity patterns by each imagery strategy. Although we did not find levels of brain activity during the different imagery tasks, we suggest visual motion imagery as a simple tool to achieve a multiclass BCI systems. Furthermore, we contribute for the discussion about the role of frontal alpha activity and for the comparison between the univariate and multivariate signal analyses. Finally, we used high-resolution 7 Tesla fMRI to map functional sub-domains in the hMT+/V5 region and to show that the tuning of these sub-domains is for the interpretation v of the perceptually relevant motion features regardless of the physical stimulation. These results contribute for the discussion about the neural correlates of perceptual switches in hMT+/V5 brain region at columnar-level and can be used as the first step for a NF study aiming to modulate brain connectivity as a function of perceptual decision. Keywords: neurofeedback, brain-computer interfaces, visual motion perception, visual motion imagery.
A atividade fisiológica do sistema nervoso varia constantemente em resposta a mudanças contextuais, moldando a nossa perceção, emoções e raciocínio. O sistema nervoso e comportamento humano têm sido alvo de intensa investigação por parte de neurocientistas, médicos e outros profissionais de saúde, nomeadamente para a melhoria de tecnologias da saúde e do desempenho humano. Uma das aplicações que tem despertado mais interesse na área da neuroengenharia baseia-se nos sistemas de interface cérebro-computador (BCI, do inglês Brain-Computer Interface), que usam sinais cerebrais para codificar a ativação ou desativação de comandos de um aparelho externo ou computador. Para além de permitirem auxiliar pessoas com capacidades reduzidas em tarefas de comunicação e controlo, as BCIs podem ser aplicadas como métodos de neurofeedbcak (NF), permitindo a melhoria da função dos circuitos neuronais e o desenvolvimento de terapias para tratamento de problemas neurológicos e psiquiátricos. Os sistemas de NF permitem a observação em tempo real da atividade cerebral o que facilita a aprendizagem do controlo voluntário da atividade cerebral, tendo por base o condicionamento operante. Estes métodos apresentam potencial terapêutico, por exemplo, em perturbações de ansiedade ou da atenção e em patologias que afetem o desempenho motor. Atualmente, estão em desenvolvimento vários estudos que visam uma melhor compreensão da dinâmica da atividade cerebral e da sua plasticidade, recorrendo a métodos focados na capacidade de regulação voluntária da atividade cerebral, também conhecida como capacidade de neuromodulação voluntária. A neuromodulação voluntária pode ser treinada usando tarefas mentais, como por exemplo o cálculo mental, a imaginação motora e recordações emotivas, que influenciam redes neuronais específicas e conhecidas. Tipicamente, esta capacidade permite dois níveis de controlo voluntário: um baseado no aumento e outro na diminuição de um padrão específico de atividade cerebral. Esta tese teve como principal objetivo o estudo da possibilidade de regular voluntariamente mais do que dois níveis de atividade cerebral. Os trabalhos apresentados nesta tese exploram a hipótese de obter até três níveis de controlo a partir da neuromodulação voluntária da região hMT+/V5, recorrendo para isso a estratégias de imaginação da visualização de movimento não-motor baseadas em diferentes quantidades de alternância de movimento. Tirou-se partido do recrutamento desta região viii cerebral especificamente durante a visualização ou imaginação da visualização de movimento, para estudar estratégias de neuromodulação voluntária que permitam ir do controlo binário (dois níveis de atividade por cada classe de controlo) ao multinível (vários níveis de atividade por cada classe de controlo). Inicialmente, as estratégias de neuromodulação voluntária propostas foram testadas usando imagem por ressonância magnética funcional (fMRI, do inglês functional Magnetic Ressonance Imaging) a 3 Tesla e depois usando a eletroencefalografia (EEG). Além disso, foi usada a técnica de fMRI a 7 Tesla para explorar com mais detalhe as propriedades funcionais da região hMT+/V5 e para propor como trabalho futuro o estudo da possibilidade de controlo multinível baseado na conetividade funcional. As estratégias de neuromodulação voluntária podem ser usadas em sistemas BCI de assistência ou em sistemas BCI aplicados como métodos de NF. Assim, ao aumentarmos os níveis de controlo através da neuromodulação voluntária contribuímos para o desenvolvimento de métodos de NF mais precisos e também para a melhoria dos sistemas BCI de assistência. Os trabalhos apresentados contribuem não só para melhorias metodológicas dos sistemas BCI, como também aumentam as suas possibilidades de aplicação e permitem a melhor compreensão dos mecanismos neuronais relacionados com o controlo cognitivo. Primeiro, testou-se a viabilidade de neuromodulação voluntária da atividade na região hMT+/V5, em participantes saudáveis, com recurso a treino NF e fMRI. Os resultados mostraram que a atividade desta região pode ser voluntariamente regulada usando estratégias de imaginação visual de movimento e nos casos de neuromodulação bemsucedida foi recrutada uma rede neuronal específica. Além disso, este estudo contribui para a discussão do potencial dos métodos de NF focados em regiões sensoriais para aplicações clínicas. De seguida, foi testada a hipótese de obter três níveis de controlo baseados na neuromodulação da mesma região cerebral recorrendo a treino com NF. Os participantes usaram três tarefas de imaginação visual baseadas em diferentes quantidades de variação de movimento. Foram conseguidos três níveis de controlo: dois através do aumento e um da diminuição da atividade na região hMT+/V5. Assim, sugere-se que é possível obter controlo multinível baseado na neuromodulação voluntária da atividade de uma região específica e de forma similar entre participantes. Os resultados apresentados contribuem também para a ix comparação empírica entre processos de controlo binário e multinível e entre imaginação passiva e ativa (assistida por feedback). As estratégias de imaginação propostas foram também estudadas com EEG. Explorámos a hipótese de que os diferentes níveis de neuromodulação seriam também encontrados no sinal elétrico cerebral. Os sinais foram registados durante imaginação passiva. Apesar de não terem sido encontrados três níveis de atividade, usando um algoritmo de classificação foi possível distinguir os padrões evocados por cada tarefa de imaginação. Este fato sugere a viabilidade das estratégias de imaginação visual de movimento para obter múltiplas classes de controlo em sistemas BCI. Para além disso, este estudo contribui também para a discussão do papel da atividade alfa frontal e para a comparação entre análise univariada e multivariada de sinais neurofisiológicos na descodificação de padrões de atividade cerebral. Por fim, foram mapeados subdomínios funcionais da região hMT+/V5 usando fMRI a 7 Tesla e foi demonstrado que a resposta preferencial destes subdomínios a diferentes orientações de movimento corresponde ao movimento percebido, independentemente do padrão de movimento real. Estes resultados contribuem para o estudo dos mecanismos neuronais relacionados com mudanças de perceção de movimento ao nível dos subdomínios percetuais da região hMT+/V5. Adicionalmente, de uma forma preliminar, trazem boas perspetivas para o desenvolvimento de uma nova estratégia de neuromodulação multinível baseada na perceção visual de movimento. Palavras-chave: neurofeedback, interfaces cérebro-computador, perceção visual de movimento, imaginação de movimento vizualizado.