Loading

Module 1: Basics of Brain-Computer Interfaces

Notes
Study Reminders
Support
Text Version

Brain-Computer Interface Applications

Set your study reminders

We will email you at these times to remind you to study.
  • Monday

    -

    7am

    +

    Tuesday

    -

    7am

    +

    Wednesday

    -

    7am

    +

    Thursday

    -

    7am

    +

    Friday

    -

    7am

    +

    Saturday

    -

    7am

    +

    Sunday

    -

    7am

    +

So, in this session, we shall consider the different brain-computer interfaces applications in a little more detail and look at tradeoffs between usability, training, and bit-rate. So just to reiterate, so we are not, we are only considering scalp recorded BCI control signals, and the first one is the SSVEP and the number of choices we have is very high. There is no training involved and you have a high information transfer rate. Slow voltage potentials are usually seen in, I mean usually done in patients with quadriplegia, who are, you know, quadriparesis and all four limbs are paralyzed. So here it is difficult, there is a lot of training is required and the information transfer is very low. Then the P300 the number of choices is high and there is not much training involved and the information transfer rate is moderate and finally, sensorimotor rhythms which are, you know where your image, visual imagery, the imagery of motor movements like moving your hand, moving your leg and here the number of choices are low. But you need a lot of training and the information transfer rate is not brilliant.

So, let us consider the P300 BCI Application Speller in detail. So, to reiterate the P300 is a positivity occurring at 300 milliseconds after the brain senses a normal stimulus. For example, you are walking and you suddenly see a snake, you jump; that is a P300, later it may be a rope or something but you still react very fast. So, that is a P300. In the lab, you have an oddball stimulus. You have subjects for presented stimuli of two different categories, a rare category, and a common category. In the rare category, the rare stimuli elicit a P300. So, this was one of the early papers Donchin, Emanuel Donchin in Psychophysiology, it is a surprise. So, this is one of the early descriptions of the P300. So, coming to the right panel, the P300 brain-computer interface on the right, you have rows and columns of a latter matrix which are randomly illuminated either horizontally or vertically. So, suppose you want to say ’please’ the first letter is P. So, the subject focuses on when the P comes and after 6 or so, you know Ps, he will trigger the letter P. As I said earlier this is very painful and slow for us, we speak so fast and we transfer so much information, but for a person who has a stroke or has some severe cerebral dysfunction, this makes all the difference. On the left panel over here, you see the P300 potentials to the desired letter in red compared to the non-desired letters in green. The non-desired letters will just be the standard visual evoked potential. This is the P300. So, laboriously you do P, then you do L, then you do E, then you do A, then you do S, and then you E, and you get ‘please’.

So, the logic is that you, this is using the, you know g.tec, you know the system. So, the stimuli are given. The subject sees them, this random flashing of rows and columns of different letters, then it is amplified and you know, suitable signal conditioning is done with g-tec systems, and signal processing is done, removing the artifacts, epoching, filtering, so on and so forth. But then you have a classifier and the classifier tells you, whether you know, which letter has been chosen and either you can write the classifier yourself or there are open source classifiers. So, you can do this, you know very inexpensively by instead of g-tec, you can use openBCI. g.tec is, can be afforded by research labs and hospitals but individuals can use openBCI. Use the openBCI software and then you use an open-source classifier etc there are many of them, integrate all of them and you have your P300 BCI Speller and if you are an Indian, you could use Indian languages; you know Hindi or Tamil, Kannada for your speller instead of English.

The thing about BCI spellers is that right now the information transfer rate is very slow. So, one of the things which could be useful would be some kind of heuristic word completion, like what you have in cell phones, the typing, and then it completes the word. So, that would be useful to increase the information transfer rate. The other thing is to decrease error rates, you use hyper-scanning. So, simultaneous multisubject recording of different subjects is called hyper-scanning and researchers have done this by using either fMRI or EEG devices which are ideal. fMRI is impractical really and finally, you can decrease the trials, the number of trials by extracting single-trial P300 events using wavelet analysis. So, wavelet analysis allows you to denoise the data so that you can see single trials, P300 in single-trial as we mentioned in the previous lecture. So, instead of waiting 6 times, maybe 2 would be enough. So would ‘please’ would be much faster. You know, it would be 70 percent faster than, you know using 6.

So, coming to the Steady State Visual Evoked Potential, so we introduced you in the previous lecture. So here on the left, on top is your forcing stimulus. So, you have a visual stimulus over here at 11 Hertz, and below is the SSVEP which is also following it at around 11 Hertz and if you do the spectral power spectrum of this stimulus, you have peaked at 11 and if you do a power spectrum of the response you have a peak at 11. So, we can use this peak, you know for the SSVEP signal. The advantage is there is no subject training involved. They straightaway, you know, you just look at the LED, the occipital cortex is forced, to you know, at this frequency and that allows the signal conditioning circuit to do its thing. So, there is a high data transfer rate compared to, you know the other BCI signals and it is much more stable than just the single visual evoked potential and this is a recording of an SSVEP and here you see, you know the heat map or the montage, the color interpolations on the brain of the EEG signal.

So, coming to the robot, so this is an actual experiment where the subject is holding the SSVEP stimulus lightbox and some of this work was done in, you know, in the Department of Applied Physics and Instrumentation at Indian Institute of Science in Professor Sanjiv Sambandan’s Lab, where the top LED blinks at 10 Hertz for upward movement. The right bottom and the left LEDs blink at 11, 12, and 13 Hertz respectively and here you see the stimuli, stimulation at different frequencies on the left and you see the peaks, the appropriate peaks occurring after FFT, you know analysis, real-time analysis happening on the right and these can be used as signals, so you can make the, you cannot see the robot here unfortunately but you can use this to make the robot move whichever direction you want. So, what are the best frequencies, ranges for the SSVEP robot? So, you can go from 5 to 60 but, you know and you can see that at 15 it is good, at 25 it is good. So, you can choose the appropriate frequencies and the, you know for the signal conditioning circuit. Over 60 it is not helpful and less than 5, you know, it can be, you know not so helpful. So, this is the average SSVEP amplitude as a function of stimulus frequency, amplitude, and frequency.

So, coming to mu Rhythms, so in awake people the primary sensorimotor cortical, that is over here, they show an idling kind of an activity like a car or a scooter in neutral between 8 to 12 Hertz, when not engaged in processing sensory input or producing a motor output when you are just still. So, this idling activity is produced by the mid-brain structure, the thalamus and thalamocortical circuits, thalamus which in the center of the brain and it has recurrence circuits with the cortex and this is the idling frequency if you will, of a thalamocortical circuit. The thalamus is very important in both sensory, you know it is a relay station for sensory input, for all the modalities, visual, oratory, somatosensory, etc except olfactory, olfactory does not go to the thalamus and it also controls motor activities, part of the motor circuit and most of the thalamocortical circuit, 90 percent of it is controlled by the cortex on the thalamus. So, these mu rhythms wax and wane, they come up and go down, come up and go down with actual movements or imagination of movement and a linear equation translates the mu rhythm amplitude from one or several scalp locations into cursor movements at 10 times a second, 10 Hertz.

So, when you have this mu Rhythm, so when you think of moving it de-synchronizes. So, it becomes kind of flat, low amplitude high frequency goes into beta, mu rhythm is in the alpha range. So, this is called event-related de-synchronization and its converse is event-related synchronization. We went through this graph previously but I shall go through it again. The dotted line is the alpha range which shows the mu rhythm the best, the dashed line is the beta range where it shows the mu rhythm but it is not as good as the alpha range and the gamma range which we have not discussed, is high frequencies 36 to 40 Hertz and this peaks before your actual movement. We know the movement occurs here at this vertical line because we are also recording the muscle activity, EMG activity.

So, what is the protocol of the mu rhythm? So, initially, users employ some form of motor imagery, like imagining the hand moving or imagining the body moving etc to control the cursor. But at over 2 to 3 weeks and these are 40 minutes to 1-hour sessions with 2 or 3 sessions per week they learn to control the cursor, you know and they cannot say how they do it, but they learn to control the cursor and with imagery, they report that when they move the cursor is just like as if they are moving it. There seems to be no difference in imagining the movement and making the computer move the cursor or them moving the cursor. So, the amplitude and also the variance of the mu rhythm amplitude as well as the time it takes, it depends on the individual. Level of EEG control, it also depends on the pathology in his brain, if he has got a stroke and you know, the stroke affects the left side then you know, the mu rhythm would be, you know diminished over here, that is control over the right side of the body. But it would be normal or fairly normal on the right side.

So BCI, how do you control? So here the thing is you have to move the cursor so that it hits the target over here and this is the raw EEG. It is showing the mu Rhythm and this is the de-synchronized EEG, you know when you are moving or imagine moving a hand and that is a signal for this guy to move up or down and these are the areas of the cortex, the yellow band is the sensory area, then there is a central sulcus and the pinkish band is the motor area. So, the mu Rhythm arises from this area and what we sense in the EEG, so that is why it is the central C3, C4 electrodes and the imagery of the movement de-synchronizes the mu rhythm and the feedback is the moving cursor over here. It is red rectangles, so you have to move it so that it hits this target and this takes about 3 to 4 weeks and as I said you need to do this for 40 to 60 minutes, 2 or 3 sessions a week and then you fi

A better way to do this would be to capture what happens in a first-person shooter game like the old games Doom, Quake, and stuff, and here the subject controls the movement of the scenery by producing low mu and they go left or they go right by producing high mu. So again, it is difficult for them to explain exactly how they do it, but they learn to control mu within 3 to 10 hours, that is it, no weeks, but hours and the consensus is the training is faster because the environment is stimulus-rich, realistic and it is motivating. You are moving, you are shooting, you are killing demons, it is better than this. This is, you know, it is so bland compared to this. So future, you know stimuli paradigms would probably use, you know stimuli like these where you know, it is realistic and you will train much faster. Much of this is in its infancy so there is, you know the place for students like you to, you know design new experiments because of all these first-person shooter games many of them, the original Doom and Quake, they are all open source. So, all you have to do is use it, get the triggers out, use openBCI, and then you are in business. You have to do a little bit of work but an interested individual can do it at home and this is a good paper to start with from Pineda et al from 2003.

Then Slow Cortical Potentials, now this was, you know the first potentials used for BCI by the famous Niels Birbaumer from Max Planck Institute in Tubingen, one of the greats of neurophysiology, cognitive neurophysiology and they used paralyzed subjects, subjects who are quadriplegic, who could not move any of their limbs and again it is very painful and slow, you see these potentials on top and he is looking at, you know different alphabets over here and when he gets the alphabet he wants, there is a slow potential but it is very painful and slow to do this compared to even the SSVEP and the P300. But again, in a person who is completely paralyzed, this is a godsend, without this, he would not be able to communicate at all. So, even though it is painfully slow, it is better than nothing, and of course, we have to talk about the Japanese, they use the fMRI signal, the fMRI looks at the increase in the blood-oxygen-level-dependent signal or the BOLD signal and the argument is, or the logic is that areas of the brain, which are active will have increased blood flow, which you see over here and this is activation and deactivation and this is on the surface while here you see activation, deactivation during, in the basal ganglia and the premotor cortex. So, this is not practical unless you have an fMRI instrument which costs anyway 10 million dollars. So, if you want to do these kinds of things you have to be attached to an Institute which has this equipment.

So, what are the best signals to use for BCI? So, there are these tradeoffs you know, training time versus bit-rate, mu, P300 and SSVEP. So, if you look at the mu rhythm that is the event-related synchronization de-synchronization, the training time takes hours and the communication bit-rate is not brilliant, it is pretty slow. P300 is much faster because you just have to sense what is coming and we assume that, you know everyone knows the alphabet. So, it is in minutes rather than hours. However, the bit-rate again is not brilliant. SSVEP, it is as fast or faster than P300, and the communication, you know the rate is much higher, it is 60, you know bits per minute. So, depending on you know your experiment, depending on your subject, depending on cerebral dysfunction you have a choice. You can use either the mu rhythm, you can use either P300 or you can use either the SSVEP. It also depends on what you want to do, with the P300 and ERDS you can, you know train the person for the whole alphabet, with SSVEP you just have 4 or 5 signals, I mean there is a limit to the number of LEDs you have and you can, you know and so 4 it seems to be optimal, you go up to 6 but that is it.

So, I have to mention that a lot of the BCI research has been, you know stimulated by the Defense interest. So, you have electrodes now, dry electrodes you can see on the leftover here. These have multiple prongs like a comb and even if one of them touches the scalp that is fine for getting a good signal and also you have electrodes from a company called Cognito which is DARPA funded and those fit so snugly and tightly that you can do somersaults and you still get a terrific signal without any interference. Also, algorithms we need really good algorithms so that we can get, you know single events if required. So, one of the interests in BCI is to see if there are any anomalies in the visual environment. For example intruders, you know, what is happening to our borders. If there are any intruders and for that, you need to have a single event, you know pickup. You need to because the intruder is not going to jump across, you know 100 times for your P300 to be average. So, here you use wavelets. Wavelets allow you to denoise, you know the brain potentials. It also allows you to pick up an individual event, which is what you need. So, thank you very much, BCI is a growing field and it is one of the fields which more students are interested in jumping into, because something is fascinating about being able to control your environment just by thinking about it and the story is not over. It has just begun with BCI. There is a lot of place for newcomers, people who think differently you know, both medical students, neurophysiologists, as well as instrumentation people to contribute to the field; all the very best.

We are going to see the introductions which includes the flow of brain computer interfaces and the various clinical or biological applications of brain computer interface. Furthermore, we are going to discuss about the different stimuli that are being generated. So, stimuli is a specifically designed system or specifically designed be it audio or video to get a required response. So, your brain responds to a particular event. So, stimuli is nothing, but a group of events that has been presented in a specific manner. So, how we are going to get that particular stimuli, what are the hardware required to get that stimuli, how we can tweak the parameters related to be it auditory or visual stimuli all these things we are going to discuss in that stimuli generation part. Additionally, there are couple of demonstrations for auditory and visual stimuli which will help you to design any kind of auditory stimuli you want to design. Once the stimuli generation unit is understood we will shift our focus to the experimental setup and biopotential acquisition. So, the experimental setup is one of the very important aspects of any neuroscience experiment.

So, basically how many electrodes you are going to use, where exactly you are going to put your electrodes, at what level or intensity you are going to give your stimuli. What is the rate of stimuli everything like checks the functionality of the entire system with some known things, compare it and then there should be you should then give a signal that now we are good to go with the experimentation. So experimentation setup and finally the biopotential acquisition. Now mostly we are going to record EEG for this course. We are going to discuss about how EEG can be recorded, how it has been processed and also EEG is very, very low in magnitude. It is like microvolt in terms of microvolt range. So, how you are going to exactly get this much small amplitude voltages or this kind of biopotential that will be covered in the third module. And once we acquire the biopotentials, finally how to process it in order to get a particular response that is something called event related potential. So, how you can extract those event related potentials from this obtained EEG. So, all these things will be covered in this particular module. So, considering the first introductions of module. Let us see what we will have in this module.

So this is the outline of the presentation. I have already discussed about auditory stimuli and how we can acquire the biopotential we will see in the details in case of auditory stimuli and biopotential single processing or EEG related single processing or even ERP event related potential related signal processing. We will also demonstrate few of the concepts so it would be easy for you to understand as well as implement. So, I will show you one simplified diagram of any kind of brain computer interface system. So, be it any application of brain computer interface, this is the basic structure which will be followed. So, if you can see this part this is a stimuli generation part or stimuli generation unit. I have shown two of them two times. This is on this top part this is auditory stimuli where two images are there. One is a normal sound image and human speech image. Both kind of auditory stimuli are used. In some of the cases when patient is not that much cooperative in a way patient is not responding he cannot respond properly or let us say the worst case if a patient is in coma. It cannot respond that well to the sound that is being generated by some machines. Mostly this auditory stimulus is used to detect whether you can hear or not. Basically for hearing screening or your till what intensity you can hear.

So, that will decide your subjective threshold that okay we have two subjects out of which, which one can hear it even lower sound or at some places this is a check that whether a person can hear completely or not. So, that time there is something called tones, clicks, chops. This kind of tones are presented to know a hearing threshold for a particular subject. Whereas, for a patient who is not a completely healthy or its neural conditions or he is like kind of as I mentioned if a person is in coma or something at that time it is advisable to present a human speech sound be it is of some relatives or some other sound which is very much familiar to that person. So, it is like this both kind of recorded, prerecorded stimuli, auditory stimuli as well as preprogrammed auditory stimuli can be used as an auditory stimuli generation unit. In the same way there are a different kind of visual stimulus so and it refers in various attributes like which kind of shape you are going to use. So, some standard visual stimuli has round shape and something called blue ball and red ball. So, these different events or different stimuli can form the final stimuli in which this instances the different images are being ordered in specific manner. Other way in most of the eye-related conditions or eye-related disease that is something that you might find character recognition. So, this is a kind of customized visual stimuli as well as there will be stimuli with different intensity.

So, same way like you define a threshold for your auditory screening you can define a threshold for your vision field. So, and one more prominently used visual stimuli is the checkerboard. So, the checkerboard is nothing like a chessboard if you have seen and inverted checkerboard visual stimuli is very prominently used in many of the visually of potential testing. So, once you understood this stimuli generation I am talking flow wise. We will go into the detail of stimuli generation, how we can generate and how we can program it, but for now, once you can understand how stimuli is generated with respect to your particular presented stimuli your brain responds and some form of potential getting elucidate. So, in order to get that biopotential, you need electrodes. So, here in this image, a proper set of EEG gap is shown with electrode based on it. All these electrodes will be recordings will be given to the signal conditioning circuits.

So, I already mentioned that being low in magnitude very critical to obtain all this EEG biopotential, so that is done with some smartly designed electronics consisting of combinations of filters amplifiers and mainly instrumentation amplifier and finally you are going to process it. In order to process it in your computer be at MATLAB at signal processing or any other processing, you are going to convert it into our digital form before transmitting. So, mainly this biopotential acquisition consisting of the instrumentation amplifier, your filters, your filters it will be having both kinds of filters band-pass and band-reject. I will tell you why. So, mostly in this experiment happens in the specifically allotted room, but there also there is a high chances of getting power line interferences. So, in order to remove this we are going to use there is something called band-stop or notch filter with respect to your power law frequency 50 hertz. So, it will have an instrumentation amplifier, it will have your filters both kind of filters band-pass, band-reject. It will have a broadband amplifier and finally, it will have ADC multichannel ADC in order to convert the obtained analog value into digitalize form. Once you convert it into digitalize form you have many wireless transmission protocol. Mostly in today’s practice BLE that is Bluetooth Low Energy modules or else something called RF or this kind of modules are used to transmit the obtained biopotential. And it is like most of the modulus you can have kind of receive the data using dongle into your laptop. So, further, you can process it using any of your software. So, this is the basic flow for brain computer interfaces. So, we will check each module in detail.

So, before checking each module in detail I have listed down some of the key points in each modulus so we will see every key point one by one. So, I have already discussed about auditory-visual stimulus, but there are some parameters with respect to that we can change the particular response and there are some potentials, biopotential that response to a change in your one of the parameter they are being generated with auditory change. So, this auditory change can be characterized by frequency, it can be characterized by duration, it can be characterized by the presentation stimuli rate, it can be characterized a timber. A timber is nothing but if you play a particular chord in your guitar and if you play the same chord in your keyboard you will get a different sound. If you play bishop in guitar and bishop in the keyboard you will get a different sound. So, that is what timber is. So, this kind of specifically designed or customized auditory stimuli can be used. Also in both the cases auditory and visual it is very important that we can find exactly how many number of events are used. So, in most of the cases as I mentioned it responds to the auditory change.

So, we should have multiple auditory stimuli in a proper sequence. So, before starting our experiment you should have a clear idea of how many events you are going to use be it auditory or stimuli. One more evoked potential is the somatosensory evoked potential where auditory and visual both potentials are localized at your ear and your eyes obviously, but somatosensory is global. It is related to your sensations. So, if somebody touches you, you will get a sensation so it is like a global evoked potential. Further, we will have a look at experimental setups. So, before starting any experiment you should know where you are going to put your electrodes, how you are going to record it. Before that is there any requirement or any change that needs to be done for a particular subject, which kind of electrode you are going to use? If it is a dry electrode fine, if it is a wet electrode how you are going to prepare a scheme for that. Every person has a different density of head with different density of hairs so which kind of electrode. There are different types of electrodes I will show it to you there is something called a flat electrode, there is something called a spike electrode, there is something called a comb electrode. So, which kind of electrode you are going to use, where you are going to place it. And all these things should be finalized before starting any experiment. So, first you should know the site of an electrode where you are going to put your electrode on your head. You should also know the montage. Now montage is way to acquire different biopotential based on your electrode. So, if you directly put the recordings from your electrode it is like you are taking a monopolar value of that.

Now if you refer a particular value to some other electrode it is called reference montage and why this referencing and all is required is because if you take any nearby electrode as a reference most of the common noise will get eliminated. We will see this thing into detail. I will also show you an example of the same recording done for two different montages we will see that. And I have already mentioned on that for wet electrodes you need to prepare your screen. So, generally, when a subject comes you have to know it depends on where you are going to take your recording. So, you have to wipe it up with spirit and there are some specifically available paste or creams which is being used. So, that will like when you perform an experiment that we will make sure you will get the best response with your electrode. Be it mostly in case of wet electrode the screen preparation and all will be required. Further, if you move ahead there are few things which we must take care. First and foremost the biggest problem for any biopotential acquisition in open environment would be power line interferences.


So, you should be using a shielding now how to use and exactly which kind of shielding we should use, how it can be used and how it stops your electromagnetic interference we will see. Also it has been recorded, observed many times in case of multi electrode placed on your skin or scalp. One of the electrodes might get little bit disturbed or that electrodes recording will not come properly. So, always make sure before you start an experiment that all electrodes are properly connected to the scalp very important point. One more thing your particular device will have several subunits. Like your acquisition unit will have one function to do, your stimuli generation will have one function to do. All these subunits must have a common ground also we will see when I will show you one example this is a mixed-signal electronic design. So, it will be having analog as well as the digital ground. So, both the ground should be separated and kept at a proper place. So, all this thing will come into the picture when you design an electronic circuit or PCB in order to make your EEG acquisition or let us say ERP abstraction system, and also it is advisable to check your stimulus functionality. We will talk about more in the upcoming slides so and first like finally, you should have if you are performing any experiment this involves EE