Loading
Notes
Study Reminders
Support
Text Version

Introduction to Brain Computer Interfaces

Set your study reminders

We will email you at these times to remind you to study.
  • Monday

    -

    7am

    +

    Tuesday

    -

    7am

    +

    Wednesday

    -

    7am

    +

    Thursday

    -

    7am

    +

    Friday

    -

    7am

    +

    Saturday

    -

    7am

    +

    Sunday

    -

    7am

    +

So, hello in this session we shall introduce brain-computer interfaces and there is a lot of interest in BCI over the last 3-4 years. It has reached the general interest of the general public; but people have been working on it for the last 20-30 years. And essentially BCI and the earlier thing was BMI brain-machine interfaces. They are devices or systems which respond to the neural processes in the brain that generate or modify movement. So, the signal which we use are signals from the neurons or the scalp; so you have the brain. You have signals coming in from the brain you know following or just before any movement. And then you have signal processing and then you have a device a computer or a prosthetic arm for example widget moves. So, you do not move this; this moves based on the signals the system picks up from your brain.

So, what is the goal of BCI? The primary goal of BCI is to suppose you are paralyzed. It should you know or your neural pathways are damaged; so you can use brain signals rather than your paralyzed limbs to control devices. And this could be robotic devices or any device. So, their BCI systems are predicted to have great societal impact. And there is a growing interest from the industry to commercialize and market BCI systems for medical and non-medical applications, especially in Japan. And one of the spin-offs is that you can expand your senses, for example you can feel and touch infrared. So, these are the infrared sensation is not normally present in humans; but with appropriate sensors, you can make an animal a mammal, recognize and respond to infrared. So, who are the innovators in this field?

So, there are a few companies who are pioneers in this field; most of them are in the research stage and there are few products that have been offered commercially. So, one is neural signals they are developing technology to restore speech to disabled people. So, you have this is a brain implant, so you have the sensors in the brain Broca’s area which will transmit signals to a computer and then to a speaker. With suitable training, the subject could think of each of the 39 phonemes in the English language, and speech can be reconstructed through signal conditioning, and you hear it through the speaker. NASA has researched a system for reading electrical signals from the nerves in the mouth and the throat area rather than directly from the brain and they web search by mentally typing the term NASA into Google. Cyberkinetics Neurotechnology Systems in the neural interface which allows disabled people to control a wheelchair, robotic prosthesis or a computer cursor. And finally Japanese researchers and emotive have developed a brain-to-computer interface that allows users to control their avatar in an online world; for example, Second Life or World of Warcraft. But, nothing has been widely deployed they still are in pilot stages and stuff. So, there is still a huge opportunity in this field to make a BCI device which is cost-effective and reliable.

So types of BCI, so BCI can be invasive, so here you have multielectrode arrays of tens to hundreds of electrodes implanted into brain cortical tissue from which movement intent is decoded. You all must have seen the recent demonstration by Elon Musk, where they reconstructed the movements of the hand legs and the four legs in a pig on a treadmill. And this is what exactly they did, so but we are not sure how many electrodes they implanted. We are not where it was implanted; it is still not published. For the non-invasive variety of BCI here we use the scalp signal and we record the changes in EEG state. And this passes through a signal conditioning unit, and you can control a computer cursor or you can control a robotic arm.

So, what are the current trends in worldwide BCI? So, it is extensive and it is rising rapidly especially in China. And BCI technology is rapidly approaching the level of first-generation medical practice. And clinical trials of invasive BCI technologies, home use of non-invasive, EEG-based BCI; example Neuroscan and consumer electronics with emotive and neurosky. Now, emotive and neurosky are gamer systems and the much cheaper than the professional systems. So, any student who is interested in a DIY do it yourself kind of a setup can use this. And I have not mentioned it here software called open BCI; open BCI is contributed by many-many different research groups all over the world. And it is the standard software which you use for BCI research; unless you want to write your own code. But, BCI research worldwide is uneven, so invasive BCI is almost exclusive in the United States North America. Non-invasive BCI systems predominant in Europe and UK. The algorithmic research is being pretty much led by China and robotics where you control robots is mostly happening in Japan the advances.

So, the images over here on the left you see invasive BCI; so they use there is a project called brain gate. And they use an electrode which you see over here which is about 1-centimeter square electrode. It is multiple electrodes in this array, matrix of electrodes and it is inserted into the brain of a person with a major problem. And you do surgery, you insert it, you close up and then you train the person to control through a wire signal conditioning unit; either a cursor or robotics. There is a lot of interest from the defense community in BCI also, and because war and defense are great stimuli for research, unfortunately. So, a lot of advances in electrodes and in algorithms signal conditioning, robotics have been driven by the defense interest in BCI.

So, signals and hardware in human non-invasive BCI, we shall focus on non-invasive BCI; because for invasive BCI it is not practical unless you are a neurosurgeon and have access to patients brains. So, non-invasive BCI occurs you know we use the scalp, recorded EEG. So, what is recorded are electrical signals, they are between 5 to 10 microvolts; they recorded from the scalps. And this could be motor potentials, we saw like for example the CNV or the bridge shaft potential or the mu rhythm. It could be slow wave potentials, again the CNV and slow dc shifts before movement. It could be using the fast Fourier Transform on the mu rhythm; so you can use the peak of the FFT signal to control movement, or it could be cognitive potentials like the good old P300. So, in the middle you see a P300 system, so typically what happens is and this is from GTech this is a good superb German BCI company. It is a little expensive but they sell devices, which you could use at home. Suppose you have patient with stroke and they want to rehabilitate them on or help them to communicate; this is certified for use at home.

So, typically you have different for the P300 stimulus, you have different alphabets being flashed on the screen. And suppose you want to say hello, so you have different alphabets being scalped and each time you see the word h. The first letter of the word hello, you know you have a P300 signal and 6 instances of this and it will get the word h. So, similarly, e, similarly l, again l and then o; it is really painful, the communication rate it is so slow. However, for a person who is kind of locked in and who cannot talk; this is a major breakthrough they can at least communicate with the people. So, even though it is very primitive and stuff for a person who cannot communicate otherwise because of some major dysfunction like stroke, this is really really useful.

What are the other BCI? This is the P300 BCI signal, so the other BCI signals are the visual evoked potential. And you pick up the visual evoked potential from the occipital cortex behind, and you have number of choices. There is no training involved and the transfer rate is pretty high. Actually, there is a particular variation of the visual evoked potential for the somatosensory work potential which is used. The advantage of this is that no training is involved and it is a pretty decent transfer rate compared to the other BCI signals. Then SCP; slow cortical potential, these are slow shifts in brain signals and there are not many choices either 2 or 4, and it is quite difficult. You need a lot of training and the information transfer rate is pretty low. We just saw the P300, these are positive peaks due to infrequent stimuli; and the number of choices are high. No training is involved and the information transfer is (moderate) medium. Then sensorimotor rhythms, the new rhythms, here is modulations and sensory-motor rhythms synchronized to motor activities like you think of moving your hand. So, again here the choices are low and you need training and the information transfer rate is not brilliant, it is pretty low. This depends on the person some people are able to do it very fast with little training, some people take much longer. So, it depends on the brains innate capacity and also how much damage it has suffered due to stroke or whatever.

So, these are the actual signals, so you have these slow cortical potentials on panel A; so negative up and you see this thing going down over here. So, as it reaches a peak, you can use this to move prosthetic arm. Then you have the P300 evoked potential where you have the normal visual evoked potential. And then when you have the pote… a stimulus which you respond to you have a huge P3 over here; and this is a negative absolute P300 is inverted. Then you have sensorimotor rhythms what I said in earlier lectures like the mu rhythm, where you have in the central electrodes, rhythms in the band of alpha 8 to 12 hertz. And as soon as you think of moving your hand or leg, it disappears; so you can use the FFT analysis, you get a peak. And you can use that as BCI signal. And finally intra cortical BCI where you have electrodes like the electrode which showed earlier. This is inserted inside the brain and this is the best because you have the least amount of noise but you cannot do it in humans. But, you can do it in animals and this gives the best signal.

So, this is a sensorimotor rhythm and if you look this is the regular EEG signal, which has been decomposed into its FFT components. So, the dotted line is the line of interest that is the alpha band mu rhythm. The 14 to 18 hertz that is mostly beta and that also has a signal, and the higher bands like the gamma band that is 36 to 40; that occurs before you know the movement or thinking of movement. And here the movement is measured by putting an electro EMG electrode on the hand or the leg. And you see the mu rhythm the one of the alpha band that gives the best signal. So, we use this signal to control our prosthetic arm via signal conditioning unit.

The steady state visual evoked potential, now consider the VEP the visual evoked potential; you have a p1, n1, p2 the initial peaks. Now, you can give them at different frequencies and this is very good because it has a high transfer rate. It is more stable than just the VEP and there is no training required. So, consider on the right over here, you have first let us look at the signal on the left. So, you have signal coming at a high frequency 2 hertz, 4 hertz what have you and you have the VEP following it. Now, consider the circuit over here, the subject is sitting in front of a TV and you have 3 different LEDs I am sorry 4 different LEDs, one above to the right, below and to the left. Now, each of these LEDs flash at different rates; 2 hertz for forward-turn right, 4 hertz-go back, 6 hertz and I turn left 8 hertz or any combination thereof. So, if you want to move forward or you want a robot it is connected to the signal conditioning system to move forward; you look at you just look at the forward LED, and your brain goes into the 2 hertz rhythm. If you want to go back, look at the backward LED; and if it is 4 hertz, it goes at 4 hertz and that is your signal. If you look right it is 6 hertz and left it is 8 hertz. So, these prime and push the brain into these frequencies and looking at these frequencies you can make the using these frequencies again by a signal condition unit. You can make you have 4 signals, you can make robot, go forward, backward, right or left.

So, a lot of work has been done with monkeys; because monkeys you know we can go inside their brain. So, you train a monkey to reach for something, so when it reaches for something you look this at the right is the raster plot. You have electrodes recording units in the brain single units firing; so you have a set a massive set of units in red when the monkeys reaches. When it does not do anything and sitting quietly, you do not have that signal. Now, as soon as you get the signal before it reaches, you can make the robotic arm reach. So, the monkey learns or rather the monkey’s brain learns and then the monkey is able after some training to move the robotic arm just by thinking. So, it is going it thinks of reaching and the robotic arm reaches and this should work similarly in humans too. These are details the monkey sitting and it is got its a joystick or whatever to reach and you have a signal conditioning circuit. And once it senses the reach signal with all these units in the brain firing; it moves robot robotic arm.

So, similarly with rats and with rats a little bit of neuroscience background over here; there is something called the homunculus which is a representation of different sensory areas of the body external areas on the brain and you see how distorted it is. You have a lot of representation for the palm, huge representation for the lips and tongue, and for the feet but not much for the rest of the body. And this is the most important parts of a sensory, the arms, the feet, the lips, the tongue. So, similarly the, this is the human and this is a sensory homunculus; you also motor homunculus which is similar but little different. And that is represented on the motor part of the brain, the pre-central gyros. So, if you see the sensory homunculus the neck, the head shoulder, arm on top then you get the thumb, the face, teeth, tongue, pharynx. So, it is like that and the leg is right on the top. So, in a rat the main sensory organ is the whiskers; so rat uses whiskers to sense obstacles in its environment. And has a specific pattern on either side of its nose, and this pattern is represented on the sensory part of the rats brain. This other distorted image is the rats homunculus just like the human homunculus; it is the very distorted version of the rats body surface with the most important parts being given more a surface area. And this is histology looking at the microscopic area of this visco-cortex if you will and these dark staining areas these little islands represent each hair on the opposite side of its face.

So, very interesting experiments were done by professor Nicolelis Duke and he is one of the leaders in this field where they used an infrared emitter. And in the rats visco-cortex, they implanted and stimulating electrodes which were from a IR detector. So, basically, instead of getting feedback from its whiskers, it is getting input in the whisker area but from an infrared source. So, for the rat, it is feeling an obstacle but actually, the obstacle is infrared emissions. And this is interesting because the rat does not sense infrared but now you are making it sense infrared. And for more details you can go to this paper, it is a seminal paper in this field.

Some details of the infrared detector and the stimulating electrodes. So you have an infrared source. You have this detector on the rats head and then you have the stimulating electrodes which stimulate the whisker are a as soon as you get infrared input. So, the electrodes are very straight forwards biphasic pair of 30 mu stainless steel microwires each about one-third of a millimeter apart 300 mu apart. The IRD or the Infrared Detector is attached to the connector and is powered by two extra reference pins on the connector. And it sense this is the sensitivity, it senses this particular this is an infrared frequency 940 nanometers wavelength. And the pulses once it sends, once it sends senses this. The pulses which are given to the brain are 100 milliseconds in duration with 50 milliseconds between cathodic and anodic phases.

So, this allows the rat after training to respond infrared, then after some time the rat you know it cannot you know there is no distinction between something touching its whiskers or an infrared emitter stimulating the rats brain. So, what professor Nicolelis did was to create an animal, which senses a sensation which it normally does not sense; which to me is kind of mind-blowing. So, implications of BCI: So, one is complete sensory feedback. So, closed a loop bidirectional brain-machine-brain interaction; it significantly improves rt reaction time and accuracy. You can also transcend a species normal perpetual range, like we just saw a cortical neuroprosthesis can expand the species perception to infrared. So, if it can do it for infrared, it is trivial to do it for sensing radiofrequency fields or magnetic fields, etc or what you have. Then the other thing is you can have theoretically a brain-to-brain interface; real-time sharing of sensorimotor information.

So, this is it was done as a proof of principle by you know this company called Inobio. And as Dr. Riera and Dr. Alvaro Pascual-Leone were among the group, who did it. So, basically you have motor imagery happening in one brain, and that signal is picked up. And it is transferred to a TMS a transcranical magnetic stimulator which we studied in previous lecture which is over the occipital cortex. So, as soon as one brain thinks of something interesting the TMS stimulates the other brain. And the other brain you know you see the stimulus as a phosphine a flash of light very primitive. But, it was proved and they did this experiment between Kerala and Barcelona; the TMS subject was in Barcelona while the imagery was done in Kerala in India. And it worked across the internet and it worked just fine. Similar work has been done in monkeys by professor Nicolelis's group, where they have two monkeys or even two rat similarly communicating with each other.

So, in future directions; so we need better sensors, small sensors and so on. So, this is a microscopic sensor designed by Professor Waters group at Stanford. So, this whole panel over here panel A, it is the order of millimeters it shows over here it is about 5 to 10 millimeters attached to the skin. And within this you have all these sensors, you have a strain gauge for seeing deformation or pressure. You have temperature sensors, you have ECG/EMG sensors. You have a wireless power coil, then you have a RF coil to generate radio frequencies and you have an antenna and an LED just to show that it is working. So, this can also be distorted or deformed and they work just fine because they are flexible. So, EEG sensors with the LED coupled to the EEG signal and your grid is 6 to 10 millimeters per sulci. Also you can resolve sulci and gyri in the brain, and it can be used as transducers for haptics to feel a sensation.