Virtual Reality Engineering
Dr. M. Manivanan
Department of Biomedical Engineering
Indian Institute of Technology, Madras
Lecture - 66
What is Haptics?
So we will start with a very simple Introduction to What Haptics is? How many of you had heard about this word haptics earlier before? Haptics you are aware of it? Ok. So, what is haptics?
Student: Touch sense of touch.
It is a sense of touch ok. Haptics is a sense of touch is one response, what is haptics? All the sensors in which touch is involved ok, it is the same thing in another way alright anything else.
Student: Experiencing the force.
Experiencing the force ok alright.
Student: Combining both the cutaneous sense and the kinesthetic sense.
Combining both the cutaneous and kinesthetic sense ok alright. Any other answers? By the end of this class we will come again and look at all these your answers whether that is correct or not, whether it is making sense or not ok.
Let us start with what is haptics? For vision, eye is the organ right; what is a basic of our vision. Light is a basic fundamental stimulus right; for auditory, ear is the organ and what is the fundamental stimulus? Pressure wave or sound is the fundamental stimulus.
Similarly for haptics skin is the organ not that is not actually correct, we are going to refine it for the time being; if you can imagine skin is the organ, the fundamental stimulus is a force, the contact force. The other organs they do not need to have a contact, where as here contact is very important ok. This is a crude introduction about what a haptics is; we are going to refine it a little better now. (Refer Slide Time: 02:29)
The word touch we have we have been using it very often ok; what is touch? Forget about the haptics, do we how, understand a what a touch is; when I touch something when do you use touch the word touch.
Student: (Refer Time: 02:55).
When you are?
Student: (Refer Time: 02:57) texture of a surface.
Texture feeling the textures of a surface ok.
Student: Manipulating objects (Refer Time: 03:03) something through have a (Refer Time: 03:04).
Manipulating objects ok.
Student: Change of density.
Density you want to feel it.
Student: (Refer Time: 03:11) I am not feeling anything.
Student: (Refer Time: 03:12).
Vibrations you want to feel it ok.
Pressure do you want to feel it ok right. So, touch is a slightly complicated word, but the end of this class; probably we will refine our understanding of what a touch is.
Only then we can understand what a haptics is; ok. Touch has two aspects of it; one is the sensory point of view of touch ok. I am touching this object, and then feeling the object that is a sensory part of it ok. Another aspect of the touch is that I am manipulating this object ok; not just sensing it I am manipulating object ok; that is a second object of touch is ok.
Together is what we mean by touch. So, in this slide what I have mentioned is only the sensory aspect of the touch. The sensory aspect of the touch again as some of you have mentioned, there are many dimension to it sensory dimension itself ; one of the dimension is the tactile sense; tactile senses that is a technical word for feeling of the pressure, feeling of the you know fine touch.
The secondary dimension is the sense of temperature ok, cold or warm or heat, heat; that is a second dimension. The third dimension is the pain it may be your surprise for many of you; pain is also a part of the touch. Usually we do not consider; pain as a part of the touch right, but it is a third dimension of the touch. The fourth dimension is called the kinesthetic sense. Kinesthetic sense may be very new to many of you. Let me briefly introduce what a kinesthetic sense and why do we need a kinesthetic sense yes Rahul.
Student: Sir what about the internal pain?
What about the?
Student: Internal bens like the muscle pain?
Yes we will come to it ok. Kinesthetic sense that is the sense of position; suppose if I close my eyes and; then I try to touch my nose with my fingertip I try to touch my nose with a fingertip; at every instance I have this sense of position where my tip is there. How do we feel that sense, where is the position since coming from? Without that sense of position I would not reach the target right.
If I reach an object at even though I am not looking at my hands, I have the sense of position where my hand is that is the reason I could reach it without much error in it that sense of position is called a kinesthetic position. We are going to go very deep into what kinesthetic senses; how what are the sensors in our body which gives us these kinesthetic sense very detailed in the later classes; but you should know that, that is also a dimension of the touch ok.
So, we have seen that touch has two dimension: sensory dimension and the and the manipulation dimension. Sensory dimension itself there are four other dimensions one is called the tactile sense, second is called the temperature sense, pain sense and the kinesthetic sense ok.
In all this four dimensions, in this course we are not going to talk about the temperature sense and the pain sense. We are going to talk about in very much detail of tactile sense and the kinesthetic sense, why? Because we have technology to give you the simulate the tactile sense and the kinesthetic sense, but we do not have technology to simulate temperature sense and pain sense; maybe it will take another 5 years or 10 years or who knows ok.
So, as a part of this course we will talk about only the tactile sense and kinesthetic sense. So, sensory aspect of touch is called the passive touch in the literature. Sensory and the manipulation aspect of touch is called the active touch right. So, again usually do use your hands to move things, we use only hands right, but how about other part of the body ? Do you, use your skin to move things do you use your legs to move things? Can you move your skin by the way some of the animals can do.
You can see the deers in the campus, when the flies are sitting on it they move the skin to you know drive it off right. Mostly we use only the hands right. So, use the hands to manipulate things all other part of the organ is actually passively sensing the touch.
For example, you may be sensing the pressure applied by your dress by the skin ok. That is a passive touch ok, it is not just touch is not just the skin level every tissue let us take a for example, muscle.
(Refer Slide Time: 09:34)
Muscle also has a sensor in it the muscle has to elongate when it is elongating how much it is elongating; it needs to know I am extending my hands how much each muscle has to move in order to extend my my hands, whether I have extended my hands or not there are sensors in each of these muscles which gives feedback to the brain.
So, we are going to see how all the sensors are networked in the brain and how brain again processes the information and then commands the muscle back to apply certain pressure, the entire circuit we are going to see how we are actually manipulating it.
The whole skin can only be touched; sensory system is one aspect of it and then motor system is the other aspect of the touch. Hand is the true organs of the touch in fact, one of the theories which is in the haptics literature is that; it is conflicting with the theory of evolution you know the theory of evolution Darwins theory of evolution right. So, that is what we have studied in the textbook.
The haptic researchers claimed that the theory of evolution is not what Darwin proposed. As the animals learned how to use the hands better and better; as the animals have more controll over the fine movements of their hands and fingers, the brain evolves; it is not that brain. Let’s the hand evolve, it is a hand that is evolving the brain; that is a concept they are proposing here.
It is very interesting, we will see some details of this alternative hypothesis of the evolution in the later classes. One major difference between other senses and the touch senses is this, other senses had can only sense eyes can only sense, ears can only sense the sound pressure, taste or others, but only the hands only the touch sense and the hands can not only sense it, but also can manipulate things; it is a bidirectionality. The last sentence in the slide you can see it is a directionality that makes it touch very unique among all the senses and also that is what complicating it ok.
That is one of the reasons haptics has not evolved so much, because there are a lot of challenges; technological challenges because of this bidirectionality, in the internet it is going to be a huge huge problem and the tactile internet, which I mentioned; now we have certain solutions to address this bidirectionality. We will see details later.
(Refer Slide Time: 12:40)
Now, the four dimensions of the touch sense we can see each one of them in detail of course, may be one or two lectures will be on tactile sense, but in this class; I will just introduce what are tactile senses. So, the tactile sense is a sense of pressure fine touch is what we have seen it, but can we have a in a proper scientific definition of the tactile sense. We are going to see such as scientific definition of each of these senses as in the later classes this is just an example.
What are the input to the tactile sense; it is not just the fine pressure it is a mechanical input it’s all called a stimuli stimulus ok. The stimulus can be mechanical, the stimulus can be thermal, chemical or electromagnetic optical also can be there ok. Some of the research students are working in a variety of stimulus now.
Once the stimulus is there that is lead to experience two different experiences like are itch or tickle or pain and pressure pleasure there is a perception level. So, there is a stimulus there is a perception ok. So, how the stimulus is related to the perception is one of the important, that is that is going to be the focus of this course; it is not just to understand and stimulus ok. Tactile sense means it is the totality of the entire system, what is the input to the system? What is the output to the system?
Output is a perception ok.
So, what is perception? How do we measure the perception? Why do we need to measure the perception? Only when we measure the perception we can improve things ok. So, we are going to see a quite a few classes on this perception; how do we measure the perception ; essentially coming from the haptics, how do we measure it and how do we improve on it that the whole feel called psychophysics, which I am going to introduce in one of the classes.
And the focus of this course is psychophysics, this course is you can find it two very different from any other engineering courses in you had had in your a the engineering curriculum, because we are going to talk about what is happening inside the brain. So, we are go to give measurement or we are going to give numbers to the mental events what is happening; that is what the experiences is; that is what the perception is; haptics is all about perception it is not you know just sensing it is not just manipulation it is also about the perception the human experience ok.
This experience is what we are going to measure it, the experience we are going to give numbers, and then we are going to find out how to design systems in order to improve this experience. Only when we measure things only then; we can control things ok, we can design new gadgets in order to improve this experience, that is going to be the focus of this course ok.
So, the scientific view of what is tactile senses all tactile perceptions are triggered by spatiotemporal variations of stimuli, above resulting in higher brain functions experience and emotions. So, this stimuli which is varying in both space as well as time and, that is what letting to the, the different experiences that is a simple definition of what a tactile sense is.
Similar definitions we will see for the other or senses as well ok.
(Refer Slide Time: 17:15)
You have no idea what a tactile sense is; you have some idea about what a kinesthetic senses, that is a sensory aspect of haptics or touch. This along with the manipulation is together is what a haptics is. Haptics our definition is that it is pertinent to the sensing and object manipulation, through touch by the humans not only by the humans it can be between two robos in the future two robos may be no interacting through touch it is not just the robo interacting with a human may be two machines will be interacting with a human haptics.
It can be in the real, it can be in the virtual, it can be in the teleoperated environments; that is a very general definition of haptics is. In fact, if you look at the literature, what haptics is; there will be many definitions you will find that none of the definitions are quite correct with a with a holistic approach of both sensing and manipulation.
In fact, I am going to you know ask you to a do a very small homework ; if you can search Google search it or the other literatures you can search for the definition of the haptics; and then list out at least couple of 5 definitions, and then when you write down the 5 definitions also know provide the references from where you are getting this the definitions.
Compare that definition with this definition what we have origin over here you can see in each of the definition what is missing; sensory aspect is focusing or manipulation is focusing or both are focusing you can check it up.
So in fact, you know we it is our plan to write a full journal paper itself on the definition of haptics, because it is not there. There is so, much of confusion among the haptics researchers, what haptics is; it is not very clear. So, we are going to write a definition paper in the future ok.
So, the haptics is between the human to human, it can be human to machine or machine to human or machine to machine also machine to machine also. So, your brain computer interfaces can be part of this definitions not only brain machine interfaces, brain to brain also it is possible.
So, brain signals can be trapped, and then processed in the computer and then again put it in the another person’s brain. So, that whatever this person thinks that can be conveyed to him without any you know intermediate things. So, brain to brain communication also is possible we have done some experiments we can share it in one of the later classes.
Virtual Reality Engineering
Dr. M. Manivanan
Department of Biomedical Engineering
Indian Institute of Technology, Madras
Lecture – 67
Branches of Haptics
(Refer Slide Time: 00:15)
Before we talk about it the course contents, let us look at how we actually touch the environments, how we interact with the environments. In order to understand and a haptics little more we will look at how we are interacting with a environments. We have the hands when we touch it, because we touch there are some contact forces are developed and this contact forces are sensed by the sensors in our skin. And the sensor convey the tactile and kinesthetic information to the brain and the brain process this information and then ask the muscle to move or do certain things.
And then muscle actually moves and therefore, we are moving the hands therefore, we are manipulating the objects. This is a very rough very crude idea of, what is happening in the haptics right. Let us expand it little further for example; I want to expand what is happening between and a contacting force and what is happening over here just this alone we will take it and then expand it further.
(Refer Slide Time: 01:59)
In a little detail one the stimulus is coming over here all this contact forces right.
Then our receptors or sensors in the skin is converting that contact forces into neural signals right. Neural code, electrical impulses it is generating, that electrical impulses is conveyed to the CNS, and then CNS we have the perception and then the perception leads to cognition, and then cognition we have once we have the cognition we do the motor planning and then once we do the motor planning we will ask the motor system to move it and then motor system moves the hands and then action is on the environment right. The same thing as what we have seen in the previous slide is the same thing ok.
Now each stage is a specialized course available. For example, how this neural coding coming from the sensors is converted into perception, there is a separate course called psychophysics. I am offering a course on psychophysics this semester; we talk about only this stage of the haptics. So, the sensation part alone is called the Neurophysiology for psychophysics haptics neurophysiology is one of the very important topic, that itself is a separate course and we are going to use one of the well known text in the neurophysiology, and we will understand how the sensation is happening, and then sensation how sensation is related to perception is going to be by this psychophysics.
So, the cognition is slight is ah is about the higher functions of the brain ok. Perception is you know perceiving, whether there is a you know texture or not whereas, the cognition is about the memory, whether I like it or not recalling it learning these are the higher aspect of the brain is called the cognition ok. So, once we have cognition, then we do the motor planning. Motor planning itself is a big subjects and then how motor planning is executed into the motor action, again there is a theory of motor action there is again a another specialized subject.
So, how motor system is working in our human body can be you know as a part of the biomechanics that is again an another course which is offered in our department. So, you can see that there are lot of details in it we will in this part of the course, we will focus on a neurophysiology and then psychophysics as a part of the human haptics.
(Refer Slide Time: 05:36)
So, the haptics course is divided into human haptics and then once we understand our human haptics, we may try to simulate in it in a machine in a robotics or we are going to design certain devices and that part of the haptics is called the Machine haptics.
Not only design the machines, but also we need to write softwares in order to give you this experience improve the experience of touch. So, give you a experience of touch in the virtual reality that part of the haptics is called the computer haptics. So, we will have the human haptics, machine haptics, and a computer haptics. These are the three aspects of the haptics; this course will have it ok. So, we will spend about 50 percent of the time in a human haptics another 40 percent of the time in the machine haptics, probably another 10 percent of the time in the computer haptics.
Virtual Reality Engineering
Dr. M. Manivanan
Department of Biomedical Engineering
Indian Institute of Technology, Madras
Lecture - 68
Human Haptics - Tactile System
Welcome back. So, in the last few classes we have been looking at the haptic system. We have looked at the haptic system is a combination of sensory system and their motor systems together. In the previous class, we looked at the psychophysics of the perception touch specifically.
Today, we are going to be look at sensory part of the haptics specifically we are going to focus on tactile part of the haptic sensory part also. We have seen in the last class it has a four modality tactile system, kinaesthetic system, thermal system, and the pain systems right.
We are going to focus on the tactile systems in today’s class we will look at the skin aesthetic system in the next class. Let me start with the basic question. How many of you seen skin? How many of you have seen the skin? The yeah question seems to be now irrelevant right or there is a trivial is not it.
Student: The skin of animals.
No, it is your skin; Have you seen your skin? That is the question I am asking you. So, that question seems to be trivial, but it is not trivial, do we know what is our skin?
So, apparently now what you are seeing is not the skin which we are going to see ok, skin is beneath the outer skin; we will see it. So, essentially we are going to talk about yeah entire process of when touch happens on the skin.
What are the things happening in the central nervous system and then what happens until the brain systems.
(Refer Slide Time: 02:10)
So, for example, here we are talking about the fingertip when fingertips touches something. What is happening in the fingertip? How the signal or stimulus is converted into electrical impulses. And then how it is conveyed it to the central nervous system and what are the different processes happening; until their nervous system is what we are going to see today
(Refer Slide Time: 02:37)
We will start with the answer to the our question. Have we seen your skin? Ok apparently you might have seen it unless you are a doctor where you do the you know surgery. Our skin is about a 100 micron beneath the apparent cover.
So, what you are seeing outside is actually a dead layer of your skin that is not your skin at all it is called the Epidermis. The dermis the skin which is over the skin the cover is what yours what is what you are seeing it. The real skin is buried underneath about 100 microns below that is called the Dermis that is a Greek or Latin name in it.
So, this part is the skin ok, the skin is where there are a lot of sensors over there. So, we are going to see specifically some four different sensors. This is the Merkel disc is one of the sensors, Merkel disc skins are there over here.
So, you have seen your fingerprints under the fingerprints there are ridges what your this is a along zoomed is this is the fingerprint ridges.
Under the ridges there are sensors there very small sensors that is called the Merkel discs, this Merkel is named after a scientists who has invented that in a about 150 years ago, it is a German scientist, it is called a Merkel disc.
And then there is other meissners corpuscles again between those ridges there is a another sensor meissners corpuscular, we are going to see that details of this. And then there are Rufini organs and then Pacinian organs.
So, you can see that this sensors are embedded in different levels, different depth of the our skin. So, the Merkel and meissner you can see that it is in the top layer of the skin, the pacinian and rufini they are in the bottom layer of the skin.
There are little deeper they are not at the surface level by the way do we know why we have the fingerprints? Unique identification, is are the reason fingerprints are unique, but why we have this fingerprints.
Grip alright. Is there any other answers?
Student: To hold the sensors.
To hold the sensors, objects that is what a grip he was talking about ok. There are some researchers recently they looked into the fingerprints why we have the fingerprints. Some people say that at the fingerprints enlarges the any stimulus, amplifies the stimulus.
So, that the sensors here actually now, gets a better signal to noise ratio is improved or whatever it is. But still it is inconclusive why we have their fingerprints you can go back and look at the literatures still scientists do not have clear cut idea why we have the fingerprints ok.
One of you can become a scientist in a haptics. And then you know tell a reason confirmed reason that this is the reason we have fingerprints it is still a open ended research question.
So, this numbers this names and the numbers we will have to now get familiar with we are going to use it again and again throughout the course ok.
They are also named as now SA 1, SA 2, RA 1, RA 2. SA 1 is for slow adapting they are going to see the reasons for or this naming slow adapting. This is slow adapting 1, this is slow adapting 1, this is a slow adapting 2, this is a rapidly adapting. See why it is called the slow adapting why it is called rapidly adapting we will look at it in different details.
So, below that dermis part of it I told you this is the real skin; below that dermis part of it there is also called the hyper dermis, where there is a lot of fats and everything is there. This is about you now 1 mm that is all right and there are a lot of fats over here. This is a representation of the skin.
In your fingertip if you take a small section of the skin from any other parts of the body it is going to be entirely different ok. The thickness of this outer layer is going to be different; the thickness of the dermis layer is going to be different ok. And then the hypodermis layer also is going to be different.
So, the cross section is varied throughout the body why should it be why cannot it be the same everywhere. The skin gets adapted to depending upon what is required ok, because we have going to end up touching many objects very regularly very often.
So, the nature has put up lot of sensors over here at the back of our back we do not need a lot of sensors we do not use our back to touch many of the objects. So, we do not need many sensors at all. So, the skin cross section is vastly different that is one reason.
And yes when we take bath what happens is this is outer layer, this is a epidermis they sheds off the layers, when we take bath the dead cells are washed away that is one of the reasons we have to take bath very regularly ok.
You know one of the main reasons of our skin is not only to protect, but also to cool our body. So, when you take bath the body gets cooled out this way cooling the body is a secondary purpose of for organ every organ has a secondary purpose.
So, eyes is only for seeing that is a primary purpose it has a secondary purpose, liver has one primary purpose and secondary purpose for many of the organs we have not even found out what is the secondary purpose. So, researchers are still looking at now what are the secondary purposes. So, many of the organs we have we only know what is the primary purpose ok.
According to our Indian literature the skin is looked at you know in much more deep. So, whatever I presented now is the is a understanding of the modern sciences of today.
But our system there is a Indian medical system called Ayurveda, how many of you heard about it right ayurveda, Siddha, in southern part of that is specifically in Tamil Nadu, and Kerala.
There is a system called a Siddha medical system and there is a Verma medical system there the skin is looked at you know in much more details and the top layer itself is divided into 7 more layers ok.
Epidermis itself is divided into 7 more layers and then each of the layer is connected to Pancha Boothas; Pancha Bootha so that is our typical Ayurveda science which we will not get into details.
But I just want you to you know remember that our Indian science has much more details than the modern science. We will not go into that Indian science about it this course concern only the modern size, but any of you who are interested you can get in to the Indian science.
And look at maybe some of our findings in the Indian science could be you know could be elucidating some details in the modern science alright. So, this corpuscles this mechanoreceptors we call all this in short mechanoreceptors.
(Refer Slide Time: 11:30)
Specifically cutaneous and subcutaneous mechanoreceptors; So, these are the different names not only the 4 mechanoreceptors, what we have seen meissner, Merkel and pacinian ruffini.
There are also another mechanoreceptors for example, this is also nerve endings. If you look at your air loops air loops does not have any of this 4 mechanoreceptors does not have any of this sensors.
But still we find it now sensing our touch, I am using the pen if you use your finger probably the sensors when your finger may be sensing the forces right. If you use your pen or a pencil and then try to stroke your ears you can see that you still get sense of touch how is it?
There are very small nerve endings over there and then nerve endings since a see touch ok. These are all specialized receptors whereas, a nerve ending is a very basic receptors are you aware that now the blood vessels has the sensors entire blood vessels even the minute blood vessels has the sensors mechanoreceptors.
So, essentially entire body is full of mechanoreceptors, entire body is full of the start end receptors touches your bodies and it is called a Somatosense. Have you heard about somatosense? Somatosense somato means body bodily senses.
The entire body is made up of the touch sense other sensors are just located in certain parts of the body it is all now restricted to certain are not an average the somatosense the touch sense is there throughout the body.
So, when you and I were in the mother’s womb the first sense developed is the touch sense, there is a body sense, that is throughout the sense, only then other senses started developing that is one of the reasons we call touch sense as a fundamental sense without the touch sense other sense cannot be developed at all.
So, we will find the details of this sense in the coming days. So, each of the sensors are also having a different nerve fibers we will see. But the modalities is what is what is it is a sensing each sensor is a specialized sensor it is specializing for certain stimulus energy ok.
So, for example, meissner if you look at it is sensing the stroking stroking is a very slow on a movement and then fluttering is a very slow vibrations ok. Merkel is sensing the pressure and then texture your shirt is applying a certain pressure on your skin and that is actually felt by Merkel you may not you know realize it all the time.
Nous enverrons les instructions pour reinitialiser votre mot de passe à votre adresse e-mail associée. Veuillez marquer votre adresse e-mail actuelle.