Loading
Notes
Study Reminders
Support
Text Version

Brain Computer Interface Stimuli Generation

Set your study reminders

We will email you at these times to remind you to study.
  • Monday

    -

    7am

    +

    Tuesday

    -

    7am

    +

    Wednesday

    -

    7am

    +

    Thursday

    -

    7am

    +

    Friday

    -

    7am

    +

    Saturday

    -

    7am

    +

    Sunday

    -

    7am

    +

In this particular sub module we will focus more on Stimuli Generation. We got the basic introduction in the previous sub module, so in this case, we will see how we can generate auditory stimuli, how we can generate visual stimuli. We are going to a demonstrate cup like 3 examples of visual stimuli and auditory stimuli. Additionally, generation is as important as insertion. So, for further clinical interpretation, we record both biopotential and stimuli generated values. So, generation the stimuli, generating a desired stimuli is one task, whereas you need to exactly give the same timings to your recording device. So, that is another task. So, how we can insert a generated stimuli to a recording device, what are the recording device all these things we will try to cover in this particular sub-module. So, we will go step by step, first we will see auditory stimuli for one event and two events with some variations, in the case of visual stimuli, first we will see for two events then we will for three events. So, let us start the lecture.

Hi, in this presentation, we will look at auditory stimuli generation and we will take three cases of different type of auditory stimuli before I show you the auditory stimuli which we have generated using Arduino UNO, I would like to tell you that you can generate these auditory stimuli using different kind of micro-controller I hope all of you, like most of you, are aware with timers and counters which are mainly being can be programmed in micro-controller. So, I would advise you to select any single board normal micro-controller, it can be Arduino Nano, it can be Raspberry Pi, whatever in which you are comfortable you can select one particular micro-controller and try dumping a code generated by respective software. So, basically, the flow for auditory stimuli generation is you need to program your auditory stimuli using some software and further you have to dump the code into a respective micro-controller. In our case we have generated INO file for our Arduino Nano and Arduino UNO and we have generated all three stimuli which I am showing right, I will showing in Arduino. It was dumped in Arduino UNO or Arduino Nano. One more thing you can use Raspberry Pi, for that you have to program your auditory stimuli in PsychoPy. PsychoPy is open-access software for from the university of Nottingham and it is available online. You can download it in the subsequent slides, I am going to show you a demonstration of auditory stimuli generation using PsychoPy.

So, I already told you that auditory stimuli is characterized by number of events. So, we will see three examples in which one of them will be a single event stimuli, whereas the other would be a two-event stimuli. So, these three are the auditory generated auditory stimuli using Arduino Nano. So, we will talk about each stimuli one by one. So, first is one event stimuli. Mostly this kind of stimuli is used for audiometric purpose where you just have to check whether a person is able to listen properly or not. So, in other words for hearing screening that one event stimuli is used, in terms of cogitative neuroscience, it is called as auditory, evoked potential also one more kind of audiometric test which is been known as an auditory brain stem response in which such kind of train of clicks or train of pulses are being presented. So, this kind of stimuli can be generated and I will show you how to generate one event and two event stimuli. Now if we move ahead to two event stimuli there are two types in the shown two event stimuli. So, basically, there are in both the waveforms if you see that, red ones are the more frequently occurred events and like in all three cases this on time or duration for particular is 100 millisecond whereas, in the first case the off time that is a time between two consecutive stimuli is 400 millisecond in case of one event stimuli. In the case of two event stimuli, the time is 900 milliseconds. So, in other words, the one event stimuli is presented at 2 hertz whereas two events stimuli is presented as 1 hertz. In this case of two event stimuli, if you focus, you can see that there is some regularity in case of that blue colour deviant.

So, red colour is more frequent and is known as standard blue colour is a rare reoccurring like less frequent it is known as deviant. So, in case of two event top stimuli, blue-coloured deviant occurs after 4 standard. So, it is like a constant pattern I have not shown the entire stimuli, but it is a constant it has a regular deviant whereas in case of the bottom two event stimuli, the deviant occurs after irregular number of standard. If you can see this deviant, this deviant occurs after 4 standard whereas this deviant occurs after 8 standard. So, there is not a regularity in terms of deviant occurrence. So, but why it is like that? So, based on some of the cognitive ERPs, Event-Related Potential you have to design your stimuli. So, first this two event stimuli is based on or ERP known as miss-match negativity MMN, whereas the bottom one is for another cognitive ERP known as P300. So, I would like you to encourage you to go through the basics of MMN stimuli on P300 stimuli you will find somewhat similarly similar stimuli to this. One more thing which I would like to mention is this deviant and standard if you can see just a waveform, magnitude remains same, then some of you might ask that why if the magnitude same how can this be deviant? So, it is not required that you should alter the intensity volume or magnitude in this case, you can change your frequency, you can change duration as well but in this case amplitude and duration is the same but as we have generated the stimuli we cap the standard frequency as 500 hertz whereas, deviant frequency as 1000 hertz. One more thing I would like to mention here is that in most audiometric techniques what happens is they will start with a particular reference intensity and they will keep on decreasing the intensity and parallelly they will check the waveforms which should be generated as a reflection of the generated audiometric stimulus. So, the moment that response will stop coming or the moment your brain is not able to perceive that particular sound, it would be that particular level of intensity would be defined as your auditory threshold, your threshold. So, a similar kind of waveform also can be generated using PsychoPy or Arduino Nano anything. So, now what we will see is in the next video or slide what we will see is, how to generate this kind of auditory stimuli using PsychoPy. So, next, we will see the demonstration using PsychoPy.

So, once you successfully download the PsychoPy, I have downloaded PsychoPy 3 you can see 3 windows one would be a coder, one would be a builder and one would be a PsychoPy runner. Mostly we will use PsychoPy builder because its a very friendly GUI you can easily generate the kind of stimuli you want you can use. It has a 3 panel, one is routine, other one is the flow, whereas this one is a component, using which you can generate a different kind of stimuli, you can even get the responses. I will come into that later. We can use images for visual evoked potential, you can use sound, you can give some instructions as well, you can print some text and all, but in our case of auditory stimuli generation, let us say we are using one only one event stimuli, we want to one event stimuli so we will only have a standard. So, let us rename it as a standard and suppose we want to repeat it, let us say five times, so we need to insert a loop from this point to this point and you have to tell them how many times you want to repeat, you can even name this a variable, how many the number of trials you want. Let us say, we say we can tell it repetition reps. So, here if I want it 5 times, I can press okay. So, this particular standard routine will get repeated 5 times. But what is there in standard routine? So, we have to define a standard.

So, this is a input sound which you can generate. You can name it as standard if you want you can, but standard name is already being used by NumPy function. So, we have to give some other name. Let us give it standard, to here you have to enter multiple parameters. What is the start time of that particular tone? So, let us see, let us say we want it to get started at 0.2 second and how much speed of time you want it to be remain on, then it will be 0.1, duration would be 0.1. Duration is the total duration, let us say we will keep it as 1. So, it will have a start time as 0.2 and duration as 1 second. We can also select the frequency of a sound, you can give it in the form of a notation like a, b flat and all. You can even give the number of frequency, generally, we use 500 hertz. So, I will give that 500 hertz, you can give your volume values from 0 to 1. Once you are done, you just press okay.

Now you see the, you can correlate with this that entire stimuli duration was 1 second and start time was 0.2. If I would have given here that my start time is 0.2, but I need, let us say I want it for 1 second, I need 1 event. So, I will just change this thing to 0.4 and I will press. Now what will happen is that if I play this sound, it will be on for 0, off for 0.2 milliseconds and on for 0.2 millisecond. I do not thing it would be convenient for you guys to hear it. So, let us keep it as a half a second. So, it will be on for half a second, off for half a second at 500 hertz. Let us run the program, it should generate tone 5 times, let us see. It will ask where you want to save your project. So, we can save it as here I can make a folder PhycoPy experiments. Here you can name it as auditory stimulus 1. Just save it. It will get take some at least, the first time it will take some time, you can if you want add the name of participation, how many events have generated. Now listen carefully. I hope you heard the sound. The stimuli is generated for 5 times. This is a very basic example of getting auditory stimuli that too also with one event. Similarly, let us try for another stimuli for two events. In this case suppose I need to add some delays in between standard event and deviant event. Let us say in this case only I need to add a deviant after 5 standard.

So, in I need to add one more routine here. I will name it as a deviant, then I should specify where exactly I want to put. It should come after some several number of standard. So, after this loop terminates, I want to put it, so I will put it here. Now, I will reduce this parameter for your better, it will save time. So, let us keep the standard as 3 and after every 3 standards, I need one deviant. So, I have, I should have one more loop which will be covering this and I want that loop to also repeat at 3 sec. Now, one more things, here it is on for 0.5 second and then it is off. So, we have already seen. I have to design a deviant in a similar fashion, so that interval between two consecutive tones will not be altered. So, what I will do? I will insert a sound here, I will use a similar kind of timing specification, 0.5, 0.5. The only thing I will change is previously I have written 500 hertz, here I will write 15000 hertz. So, you can clearly sense the change in sound. There are some another parameter also, you can also change the volume and all. I will come to that in some time, but before that this is kind of a very easy way to illustrate how you can get a regularised deviant in case of two events stimuli. So, I will just press the oaky.

So, here we have used two routine, standard and deviant and as per this repetition, the standard will first come 3 times and the one deviant will come. Again standard will come 3 times, one deviant. Let us hear that, that if we can get any change or anything, let us see. Again I am, listen carefully. So, I hope this thing is clear. Let us make one more modification. Let us say we are not using this kind of two events, but we want even we want to have an even bigger event or let say we want to use something for as I mentioned now for audiometry testing. So, in that case, what is required is you should have a routine which will be, like decrease in, which will decrease in terms of frequency, decrease in terms of intensity rather, so that volume will keep on the decrease. So, what we can do is, it is not like for only once we will give one particular volume, so what we can see is we will discretise the volume in 5 parts. Let us name it to tone 1. So, tone 1 consider, we consider as the highest amplitude, we will put it here, we will repeat it 3 times.

So, in this case, tone 1, we will name this guy as repetitions of 1. Now, what is tone 1, then we have to add the sound, we have to put the sound level here. Here, we will keep the frequency as 500 hertz and we will keep it on similarly, we are not going to change the timings specification, we will keep it as it is, so that you, all of you can hear it properly. So, this sound is 500 hertz and 1 is your volume level. So, what I will do is, once I am done with this thing, I will add one more routine, new routine, let us name it tone 2, where do we want to put it? I want to put it after tone 1 and it want to loop other also in the similar way for 3 iterations like I did before, I will name it as repetition 2, I will keep it here. What is tone 2? So again, I need to generate a tone 2 in order to maintain the delay and timing, everything in the, I want to keep it synchronised, so I am not going to change the timing duration. It was 500 hertz so it was properly in that case, you were able to listen to that. So, I will keep it as now 0.8. So, this is another repetition. Similar way we can go ahead and decrease the intensity. Let me quickly do that.

In this case, we will set the amplitude as 0.6. Last time we have kept it as 0.8. So, here I will put it as repetition 3, what is my tone 3? Then tone 3 is a sound, it is getting generated with 50 percent duty cycle of the square wave at frequency 500 hertz and my volume of this particular tone would be 0, it was 0, I guess. Then nobody would be able to hear anything. It should be 0.6 and this is my generated tone 3. Quickly adding tone 4 at the end of the stimuli, should get repeated thrice as normally it should get repeated thrice for repetition 4. Just to give you a people a proper understanding of what I am doing. First of all I have to define tone 4, tone 4 is the sound with the same timing specifications, my bad, 0.5, 0.5 frequency would be 500 hertz and volume, in this case, would be 0, again it is 0, it should be 0.4. Tone 4 has a volume 0.4. Hope I am right. This guy is 0.6.

So, I will finally I will add just the final routine as I mentioned tone 5, very easy. You just have to keep on adding instances, you can loop it in whichever manner you want and you will get a desired you should and remember all this thing is just one part of the entire cognitive neuroscience experimentation. So, what you are doing is you are generating a particular stimuli based on particular timing. So, I will show you this final one, again the same timing specification we will use, and here I will put it 0.2. So, every tone is getting repeated thrice and is made up of one second. So basically, it is a 15-second auditory stimuli will be played. I want all of you to listen to it carefully that if you are getting any kind of, if you notice there will be a change in terms of intensity and this kind of stimuli is used for threshold detection and one more thing I want to you know, I do not want to invest too much time in this, but if you want, it would not be this like decrement in terms of amplitude will not happen one, they also repeat that to confirm.

So, these trials are very important if you want you can put it as 2 here. Will not put it 2 right now but if you want you can put it. So, now without adding anything in this logic, I will just try to run this stimulus, you guys listen to it carefully, you should be able to identify the change. If you are using headphones, please keep it a little away from your ears or I would advise you to not to use any kind of headphones but use a speaker of your laptop or pc. So, I will play this, I will play these stimuli now auditory stimuli let us see. I hope that change is evident in the generated stimuli. So, that is it for the demonstration for auditory stimuli. I hope you understood how we can generate these auditory stimuli. It is one part, stimuli generation. So, we will see in quick, we will quickly glance through how we can insert these stimuli to the recording device in the subsequent module. So, now next we will discuss visual stimuli followed by a demonstration for visual stimuli. I hope all of you understood how we can generate auditory stimuli using PsychoPy. So, now as we have given a brief intro about auditory stimuli before the demonstration, we will take a look at visual stimulus in the similar fashion. So, how we can generate visual stimuli?

So, there are softwares available, PsychoPy presentation and all. So, this will, uing this software you can program your visual stimuli and further you should have appropriate display like Raspberry Pi display or something like that, in which you can see your generated image. So, these are like very basics of visual stimuli. I will just show you first small two event stimuli. It is a very longer duration, chop it up for 30 seconds. So, I will show you that 30-second stimuli. So, blue square was the standard whereas, red was a rare occurrence. In that case we kept the occurrence rate of standard as 87 and deviant or rare, rare square as 13 percentage. Similarly, in some of the cognitive neuroscience experiments you need a three-event stimuli, a three-event stimuli for visual evoke potential. Now, in that three-event stimuli, the aim is to find out a subject’s response towards the target or subject’s response towards the target with some distracter is there. So, basically there are three events, one is standard, one is target and one is distracter. In case of target, subject was asked to press a button or remember the count of the occurrences of target. Basically, when someone with poor motor control or someone who is paralysed, who cannot press the key, for them they use to ask them to keep a track of number of count.

So, these are like different scenarios of three event stimuli. So, we will quickly see small glimpse or 30 second interval of this three-event stimuli in which blue, blue one is the standard, yellow square is the deviant and checker box is the distracter. So, I will just play the video for you before and will after this video we will see the final video. Now, this video is being used by a researcher John Polidge who has done a lot of work on P-300 event-related potential. So, that is also a visual evoke potential. So, I will play this video one by one and ones that videos the played, we will discuss a further. I will play the other video. In this video instead of a square box, we are using ball or circle which is like a standard component to be used in visual evoked potential and we are again using checker box in this case, the parameter is two blue balls have being used with different size whereas, here two different colour balls were used. So, I will just play this video for you for some time and then we will see the actual generation of stimuli using PsychoPy.

So, let us see how we can generate visual stimuli using PsychoPy software. So, first, we need to add some routine into our flow. (Refer Slide Time: 30:58) Let us add a new routine and name it as image 1. Like we have seen in auditory stimuli generation, we have repeated one particular audio 5 times, we will do the same way just to check the functionality for repetition of first image. So, first image should repeated 5 time but you need to define your image 1 in routine window. So, what is there in first image? So, I will take this image as a component and let us rename it as image 1. You have set your start time and end time. So, start time is nothing but when your image will appear and during like how much duration of time you want it after you, so total duration you want to keep, so that is your when does this component ends. So, that is after 0.5 second. So, it will start at 0.5 second, it should stay there for 0.5 second. Now, which kind of image you want to use? So, you have to define your path, your image here. So , we will see that this image are stored here. So, I just have to copy-paste the path and the image name here to get this particular image as a stimuli. So, let us take an image of blueball.png image which is already stored there. Just press the okay.

So, this timeline will tell you that after half a second the image will appear here, and as I already mentioned that it will get repeated 5 times. So, let us just confirm this once. So, let us save it as a visual stimulus 1. So, once we save it as usual it will ask us about the participants name and session. If you are performing a specific experiment for certain number of trials, you can fill this and check for now I just pressed.As you observed it was 5 repetition of a blue ball image. Now let us say I want to generate a red ball as a deviant or like rare stimuli after 4 blue ball. So, I have to first change this number of repetitions to 4. I need to define one more routine which is like known as a deviant or the other second image. I should also define at what period of time I want that image to occur and also, I should tell what is image 2.

So, I have to the same way I have defined image 1, I have to define image 2. I will name it as image 2. Similarly, we will define the time interval that it should start after 0.5 second and it should stay there for 0.5 second. This time, last time we have used a blue ball. So, this time we will use the other one, red ball which I have already stored as an image. You can see here red ball. So, redball.png. Once you okay it, this name is in use by one of your components. So, this is how you can define the path for the red ball and you can press okay. So, now both the images are defined, let us see the let us repeat this thing like we did for auditory stimulus. Let us repeat it for 4 times. So, you have to create a loop, let us keep it repetitions 2 and repeat it 4 times. So, in a way this loop is getting repeated 4 times and this. So, it will generate a 16 second stimuli with red ball and blue ball. Let us see that.

So, in a way you can design visual stimulus based on your requirements if you want the add some more complex in generally in case of two events like multiple event stimuli, you can use one more stimuli in mostly this kind of checkerboard stimuli is also used with a reversed checkerboard pattern and all. But the idea behind this demonstration is to like you people know that using different stimuli components available in PsychoPy, you can generate visual stimuli based on your requirement and I would like to encourage all of you to go through the PsychoPy documentation and generate more number of, generate different various triggers. Also, I would like to encourage all of you to go through this demonstration and you will come to know that if you use this software properly, then how different kind of stimulus which you can generate. I think it is enough to generate a stimuli for most of the cognitive neuroscience experiments. So, further, we will see that how we can insert the generated stimuli into the acquisition device. Now you guys know how to generate a different kind of stimulus, at least how to write a program for it how it can be generated using PsychoPy. Still, you can go through the documentation and demos, it will give you in detail idea and you can design a stimulus as per your requirement. But like this thing is like according to me is half job done.

Because the thing is once you generate stimuli, the same stimuli should be conveyed to your recording device, because ultimately your biopotential which are recorded with respect to a stimuli that matter. So, you need both information, biopotential through your electrodes, through the channels as well as the stimuli like exactly at what time you have seen which image, exactly at what time a particular tone has changed to which frequency and at what time? So, this temporal synchronisation is very important and the event should be inserted to an acquisition device exactly the same time at which it was heard or it was seen. Both the stimulus, timing changes and everything should be conveyed exactly like you know at the same time when it was observed. We need to use some mechanism using which we can deliver the triggers with precision in timing. So, there is one setup proposed, setup which we have used. So, OpenBCI Ganglion, just consider that as a recording device. It is a four channel device which has like signal conditioning and finally the analog to digital converter followed by a transmitter or due to transmitter. So, what we will do is we whatever stimuli which you have generated in PsychoPy, you can run it on Raspberry Pi, so you can see the whatever your stimuli is and Raspberry Pi are serial output. So, you need to use a parallel port using which you can in case of multiple events, if you need your events to be fed to different data pins or different digital because this, events will be having the form will be having the digital form.

So, 0s and 1. It stays 1 during one event, it will convert to 0 when that event is not there, some other pin should be 1 when the other event is there. So, similarly, provision is being provided by OpenBCI Ganglion board where you can use digital IO pins to insert the event. So, this is one setup. There is one more setup which we have used. So, in case of an auditory experiment, we have generated using Arduino UNO. So, all of you know that you can write INO file and you can dump the code into the Arduino UNO. Similarly, you can talk or you can convert the triggers to openBCI Cyton board. Now, the Ganglion was a smaller recording device. It was four channel whereas, Cyton is eight-channel board of openBCI. It is a standard board which is used to acquire biopotential. Now, how this Arduino performs two task. One, it is generating a sound, at the same time it is conveying your event information to openBCI. So, so far you have seen how to generate a stimuli, but event insertion is equally important a phenomena and it should be taken proper care of, otherwise, it may result, I may see your entire experiment may fail if your triggers are not inserted properly. The very important thing, these are the two setup examples have been provided by me. If you can generate your event information as a part of your digital output through any microcontroller, you are encouraged to use that and you ultimately your stimuli should be seen and transferred exactly at the same time. So, that is all we have in this stimuli generation and insertion module. I hope it clears a lot of questions you have and I hope now you understood and able to make any kind of auditory and visual stimuli. Do not forget to have a look at the demos of PsychoPy and we will see next time, at that time we will discuss experimental protocols and how to bulk potential is acquired.