Virtual Reality Engineering
Dr. M. Manivanan
Department of Biomedical Engineering
Indian Institute of Technology, Madras
Lecture – 72
Haptic Devices and Interfaces
Welcome back. In the last class, we looked at, Haptic Interfaces. Specifically, we focused on the kinesthetic displays, in today's class we look at the, the tactile displays, the other classification of the haptic devices known as the tactile devices ok. To remind you, we are talking about, the human haptics, the first half of the course, we focused on the human haptics.
Now, we are trying to mimic the human haptics in the machine, you can consider our human haptics as a science part of the haptics whereas, the machine haptics which is, which can be considered as a technology part of the haptics.
(Refer Slide Time: 00:57)
So, how do we convert our understanding of the human haptics into engineering and, machines, which will help us to, improve the perception of virtual reality or, or know rehabilitation. There are many applications, which we are going to see later, but right now, how do we convert our understanding of the, our human haptics into machine is what our focus.
In the last class, we looked at the different kind of the grasps and specifically the contact grasp, precision grabs, and power grabs and we are supposed to come up with, a interface, which will help us to have this. Many different type of the grasps of course, it may not be possible to develop one single interface, which will help us to do all the grasp. Probably, you know as many as grabs possible or individually you know, come up with our, devices that will provide individual grasp ok.
And in the last class, we also saw, the kinesthetic displays and the, tactile displays specifically, we saw a kinesthetic displays as a mostly grounded at a kinesthetic display, where at the device, it is a advanced robotic device, where the fingertip or you are holding a pen like device is, which is a end effecter of an advanced robo.
(Refer Slide Time: 02:25)
And, the forces it is resisting your motion of the finger is actually transmitted back to the, the, the, ground. This is called the grounder displays, example is the phantom, we saw. So, many other devices in the earlier class.
So, essentially you can see that the, the device base is grounded. It has serial, linkages, which is connected to the fingertip or the fingers or the palm, which will resist your motion that is the usual, kinesthetic display in the next two classification of the devices. It is, it is the exoskeleton, where the grounding is at the wrist and again the wrist, it has a serial connections to the fingertip. The finger movement is again resisted, um, resisted and the resistive motion is transmitted back to the wrist or somewhere over here, right.
There are a couple of, gloves. We have seen, where this could be the forces. Reactive forces are transmitted to the, the palm or somewhere in the body. These are all called the, you know bodily own, haptic devices. So, some devices transmit the forces on the shoulder or the hip somewhere. It has to transmit the force. So, what we are feeling is the, the force, but there is a equal and opposite force, which has to be transmitted either back to the desk, desktop, sorry ground or, or somewhere on the body ok.
So, in this case it is a wrist or any other, places. Now, the other type of device can be the fingertip device at the fingertip. We do not have to transmit the, um, forces to anywhere else and this device is, it can give you, many, tactile feedback. So, this where there is the, the ground based haptics or exoskeleton, where kinesthetic is involve and a fingertip alone, if you take it, it is mostly tactile not kinesthetic at all right. So, the technologies needed for kinesthetic devices or fingertip devices is very different from the technology needed for exoskeleton or the grounded displays ok.
So, we will focus on the fingertip displays today and then, little bit on the, the, the body displays motion displays or our focus is to develop. These devices is that can be useful in virtual reality, you have put on your goggles HMD, you want to touch certain objects in the virtual reality and we want to get the, you know a force feedback ok. Your combination of this grounded displays or the exoskeletons and the fingertip display is, what, is going to help and today, we will focus on only, on the fingertip displays ok.
(Refer Slide Time: 06:09)
The fingertip displays essentially, you know, called the tactile displays is, again there are many varieties of displays as a part of this course project some of you can attempt to do, you know a few of the devices.
This devices are very-very simple devices ok, usually tactile device means, it is, referred as, an array of, pins ok. An array of pins, which will move up and down, where you can feel, the forces, when you keep the finger or the pump over, there the pins can go up and down and then you can feel the surfaces and, as of now, probably about, you know 200 by 200 has been and a 200 pins and 200 pins have been fabricated, using the mems technologies, but, maybe what we need is 400 by 400, depending upon the density of the fingertips, we, we looked at a lot of, mechanoreceptors.
So, the finger, the density of mechanoreceptors is, are, would require 400 by 400 pins, in the palm area alone, which is probably an 5 years or 10 years, down the line, this is, pin based pin, tactile array. We call it as a tactile array and, each one, each pin is, is called the, taxel, just like pixel in the display. We can call this as a taxel, which either switches on or switches off either on or, or, or increases, the height or decrease the height other than this tactile array a, there are many other, devices is what we are going to. You know focus about ok, the first device is the contact and pressure display, where we are talk. Contact is the very basic, the grasping, task we talked about right.
So, in the contact it is a maybe the simplest one. We can say this is the simplest one, simplest of all the, that, contact task or the you know grasping task, where the fingertip has to be you know applied with certain normal force that is all that is where, you know we are looking at the you know contact, maybe a very simple. You know solenoid or a voice coil or one degree of freedom actuator can be used to give you the contact maybe you can give a very small, you know, finger, gloves where there is only one degree of freedom movement of, this actuator, which will either contact or not, contact either contact or contact disconnect.
So, that will give you the, the sense of contact in a virtual reality, if there are menus over there when you contact the menu, you want to give a feeling of you know very simple contact even though the menus are virtual. There are no displays, there are no real displace, there are all virtual displays, but still when you touch it, touch the menu button, you should, give a feeling of a you are touching the menu by actuating. This one degree of freedom actuator that is a simplest, you know, contact, pressure, display, it is very similar to probably, you know something like this; a very small devices can be mounted on the finger where, and, can be given a contact feedback ok. (Refer Slide Time: 10:04)
Now, the second, device, could be, you know curvature display, curvature display. Suppose, if you have a object, we use, you know, curvature radius is bigger than the, the fingertip itself then what you do is let us say, this is your fingertip. You are contacting it and you are moving this, you are moving this, finger over their display and there are two things happening one is that your, your, your kinesthetic system comes into picture. You are feeling that, you are moving over here and there is also a contact ok.
A contact basically, we are talking about the cutaneous, system a tactile system. So, what is a role of tactile system in the perception of the curvature. We want to display the curvature information of a virtual object ok. So, how is it going to be suppose, again it is based on the, you know psychophysical research, um, the research shows that suppose, if we have a very flat surface and if you want to give a perception of a curvature and, there are two ways of doing it.
One way is that you have this surface normal and then rotate the surface normal. So, that you have this sense of curvature or what do you do is, you change the height of the surface normal without rotation.
Let us say, this is a very smoothly over here, there is also, these were the two ways of doing it. What researchers have found out psychophysical again, study is that this gives you a much more feeling of perception of curvature than this ok. So, not just by altering the height of the, the pins like just like the tactile arrays do, but just by rotating them surface normal. It is very important in order to give the, the curvature, um, information ok. So, again, you can come up with a very simple way of rotating the surface normals. How are you going to rotate the surface normals, we will see it.
So, a simple finger mount, which will rotate the, which will rotate the surface normal as you move over the or the, over the virtual objects that will give you the perception of curvature, without the real objects over there and then the third one is the, the vibration and texture and material of course, vibration. We all know, that is again a very simple, form of a feedback, but in order to feel the texture and materials ok. So, how are you going to use this vibration. Vibration can be useful in order to feel the texture ok.
(Refer Slide Time: 13:36)
Suppose, if we have vibration not just is a you know, vibration you know, we have this amplitude as well as the frequency, which we can modify it. Suppose, if we modify the vibration frequency according to the, you know position of the, according to the position. Let us say the; we move our finger on an, surface is a, at a different velocity. Let us say, we have a velocities increasing and, the vibration frequency, vibration frequency. Let us say this, velocity and a vibration frequency velocity of a finger moments. Let us say and according to the the, velocity we are increasing the, frequency it is not a constant.
So, by modulating the frequency, we can, according to the velocity of the moment, we can, somehow generate the texture feeling. It is not the constant frequency of the vibration, depending upon velocity, you change the frequency of the vibration that gives you the feeling of texture ok. There is a one way of a looking at it of course, texture when you look at it. It is a very complex, multidimensional phenomena, if you look at a texture, what do we mean by texture?
So, are at least I can say there are three dimensions to the texture. Let us, find out the two dimensions and then let us, talk about him other dimensions. So, texture, it can be either rough or smooth right, it can be you know a soft or hard, it can be wet or a dry, it can be, it can be, what, sticky or, slippy slippery. There are many dimensions of a talking about some, many researchers have come up with a many more dimensions, but mainly softness or hardness or, or you know roughness or um, smoothness. These are the main. So, these are the main dimensions two dimension at least..
So, this texture generating the perception giving the perception of this texture in the virtual reality, is a very challenging task ok. So, one way is to modulate the frequency, some way here or, in one researchers. They have modulated the, frequency of the vibration, depending upon the velocity yes.
Student: What does that simulation, which dimension does it simulate.
It is actually you know smooth or rough ok. So, there is a only one dimension they, they have simulated. There are other dimensions also which we are going to see right and, we can also talk about yeah. So, another, the next display is the softness or hardness ok. So, all these tactile devices, if you look at it there is no kinesthetic you know, information at all, no kinesthetic information, it is completely cutaneous or tactile right cutaneous.
How just with cutaneous information? How can we give the softness or hardness, this should be hardness right. That is a challenge, can you imagine how softness can be simulated, just with a hardness, just with the, you know cutaneous information, if you look at the softness or hardness, we have this penetration depth, penetration depth right, depth.
(Refer Slide Time: 18:11)
So, suppose, if there is a spring, we press it and then we know, how much depth, we have, compressed it, but that is a kinesthetic information our challenge is to display the softness, without this kinesthetic information is ok.
So, this penetration depth, the may be related to the, the contact area of the finger ok. So, when you are compressing it more forces, you are applying. So, the more contact area in your fingertip right. So, or, the that in turn relates to the skin deformation ok. So, somehow if we can change the contact area of the finger then probably you know, softness can be you know, simulated without the kinesthetic information ok, that is a challenge. So, how are we going to change the contact area of the, finger again we, we are going to talk about it.
Friction display, in the friction display again, we know friction is a very-very complex, phenomena, simple skin stretch. For example, may be of a 0.25 mm will give you the, you know, perception of the friction regimes, friction perception ok. If you look at the finger pad, what is it's friction coefficient ok. There are friction forces, we need to simulate it ok. So, the friction, this is very highly, non-linear.
For example, in the dry state, the friction coefficient of the, all the finger pad is, you know behaving like an elastic material, in the wet state, the finger pad, it behaves like a plastic material, because how, how the, it is neither or, or the wetness condition of the finger depends upon, how much sweat you have all right. There are sweat glands in your, fingertip or throughout the skin.
So, if you are emotional then more sweat glands are you know in generating sweat that changes the, the friction coefficient, the behavior of the, the finger pad itself and that in turn affects your perception of, of a different objects right. So, the, suppose, if we can, measure the friction coefficient, coefficient of, the finger pad that is not only just a function of a dry or wet condition, but also the, the sliding velocity. So, all the studies have, have been done and, in the haptics literature very well, but our aim is to, how do we make the friction perception in the virtual reality, without the kinesthetic information, just using the, the cutaneous devices, tactile devices.
One way is to do the skin stretch under your fingertip by appropriately, changing the screen stretch. We can give you a perception of friction. So, coming up with a device 1 degree of freedom device, which will give you the skin stretch is going to be, you know very useful the, it can give you, for example, where is yeah, your area? Where you can stretch slightly and the under the fingertip that can give you the, not only the friction, we also talk about the, the, the softness and, we are also going to talk about the indentation in the, in the, in the next slide.
So, it has a lot of applications probably, some of you can and think about developing a skin stretch display. It is not just a vibration. So, we are talking about the tangential motion, tangential motion not all in one direction. It is actually a, you know in the radial direction ok. These are all very simple, simple devices, but the applications could be, you know lot and then we are talking about the, the indentation, ok indentation.
(Refer Slide Time: 23:39)
So, we have a small bump and we want to find out a, we want to find out, we want to display this in the virtual reality, without the real bump. We want to give the feeling of a, real, real bump right.
So, when you move your finger tip over a real bump, what is happening is, there are two things happening, which has a lateral force. Lateral force as well as the, the normal force, normal force in a, cutaneous device. We cannot give the normal force without the normal force. Let us see how to use the lateral force alone to give the skin indentation.
There is a challenge ok. So, this can be easily demonstrated, if anyone of you have a comb, do you have a comb? When you go back to your, hostel you can do this experiment, if you hold this one and then slightly disturb the teeth here, you can feel the bumps in the fingertip ok.
So, this teeth is not moving up and down ok, without moving up and down only by literally moving it, you get the feeling of bumps indentation ok. This is an illusion, but this illusion can be useful to develop a display, virtual display. So, one of you can think about you know, how to make this, display. So, one of you can try it out and then see whether, that is true, can you try it out? Yes or you, you, your yourself can try it out yes, check whether that is true, yes, yes yeah ok.
So, the next, one is going to be push button oops. So, push button again is a challenge is that, when you are pushing the button, there is a movement of the fingertip, without the movement of the fingertip. Can we give the feeling of pushing the button? There is a challenge push buttons again, push buttons means you can see that the contact area is actually increasing contact area, skin stretch is there and the stretch is there and then this is going to be the normal force finger displacement yes, finger displacement.
So, just by stretching the skin, you get an illusion of a you know, increasing a contact area and, because contact area is related to the finger moment. We get the feeling of moving, the finger and really, really pushing the buttons. Now, you can go back and see how Microsoft or a Apple has, has a developed. This is the latest, Apple phone in giving, the feeling of pushing, the button on the phones ok, without the, you know actually, giving the pushing it. How did they give you the feeling of, the push buttons. It will be a very nice homework, I hope you will enjoy it ok.
Surface geometry again, and the combination of all these things as, stretching this skin and, you know modifying the surface, normals rotating the surface, normals can give you the, the surface geometry.
Now, this we have already seen it, these are the few of the devices, which for giving the indentation ok. Remember the, the comb experiment illusion, we talked about, there are already devices, which give you the feeling of a bump of a using their comb illusions and, these are the you know references as you can take a look at it.
(Refer Slide Time: 28:19)
And these are the devices that gives you the lateral stretch very simple devices ok. It is all 3D printed only 1 actuator, but only thing is you need to come up with a very simple mechanism to give you the lateral stretches ok.
(Refer Slide Time: 28:42)
Again view, it is a challenge to give you the kinesthetic perception of kinesthetic stimuli, but using the cutaneous stimuli alone ok. You should feel the kinesthetics stimuli, but you are using the kitchen is again, lateral stretch and then tangential motion can give you the feeling of the kinesthetic stimuli again, there are devices over there.
(Refer Slide Time: 29:01)
You know recently, I think just, 6 months back, there are a lot of startup companies, who made product very successful product. As far as the virtual reality is concerned, that will give you haptics feedback in a virtual reality controller.
The virtual reality controller, you have this controller in your hand, it is not just giving the vibration feedback, but is much more that, and vibration feedback using some of the terminologies or technologies. We talked about for example, striker VR is one of the successful startup company. It gives you the recoil of a gun ok, they give you the library of, you know, designing your gun.
How much recoil should be there and all those things very simple device, such as this one, you take it. It is all, you put it in your own device ok, any device and, it is wirelessly connected and, you design your, recoil and you get a perception, this is one of the successful.
So, how again, what is the technology behind this, device you can, probe into it, a very simple, solenoid ok, check whether you know, they are using the skin stretches, such as what we talked about. Combination of the skin stretches and the, you know, you know solenoid motions can give you, a much impressive, a feedback of um, this, recoil. There is another device again, this is also very successful virtual reality reactive grip from a company called a Tactile haptics. (Refer Slide Time: 38:47)
So, this is a virtual reality, handle, you hold it again here, when you, when you hold it under the finger, you have your skin stretching actuators, you can see that these are all the actually, it moves up and down here, also it moves up and down.
There are actually 4 such 4, 8 actually you, this is the front side, this is the side, this is another side and the similar one at the back side. So, 4 here and 4 down. Combination of these things can give you the feeling of a movements. So, these are all simple skin stretches, there are a very nice, um, a product from reactive grip ok.
(Refer Slide Time: 31:47)
This is the another or again and successful, startup it is called Tactile VR mask ok. You put on this, HMD onto your face. There are actuators over here, there are actuators over here, can you see the actuators? These actuators will give you a feeling of blowing onto the, onto the face.
Suppose, if somebody something strikes on the face, then they want to give a, you know feeling of a blowing onto the face very simple actuators again. These are all without the kinesthetic movements, just with the, cutaneous. How do we, generate illusions is or, is what the challenges.
Recently there is one.
(Refer Slide Time: 32:34)
And, research work, this is not a product yet, but is in the research work, which I thought. It is a very nice, product probably, it will be our product from the University of Tokyo, led by one of the professors in, the university, there haptics shape illusions.
Since, in the, in the last class, we all talked about the illusions, I hope you remember the, the shape and size illusions right. A small shape looks heavier or, or compared to this. You know larger shape of the same weight right. Similar things can be talked about it, how shape can be modified just by you know, rearranging certain things. For example, he is talking about, um, something called a haptic perception model.
(Refer Slide Time: 33:33)
So, you know x axis. Let us say mass is plotted y axis. Let us say shape is plotted, then they come up with a relation saying that some relation. So, as you increase the mass, the shape also is actually increases or the, there is a you know perception between the, the mass of the, the object.
What you hold and the shape of the objects, that is what the size and, a weight illusions, we talked about the same thing is what is implemented over here. In this bat the size is same, but by adding weight into the holes, you are actually changing the, the weight of it is in the virtual reality, when you put on the HMD, the mass, because by changing the mass, you are changing the shape of the object. What you hold in the virtual reality ok.
That is a haptics shape illusion, this is a very nice, project, which I thought may be of interest to a lot of people ok.
Log in to save your progress and obtain a certificate in Alison’s free Diploma in Sensory Virtual Reality online course
Sign up to save your progress and obtain a certificate in Alison’s free Diploma in Sensory Virtual Reality online course
Please enter you email address and we will mail you a link to reset your password.