Notes
Study Reminders
Support
Text Version

### Manipulation and Locomotion

We will email you at these times to remind you to study.
• Monday

Tuesday

Wednesday

Thursday

Friday

Saturday

Sunday

Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary

Lecture - 19-1
Interfaces (manipulation)

And so selections one big thing one big thing part of manipulation. After selecting an object you may determine if you wanted to grab it. The next thing is placing the object back or at least doing some kind of positioning of the object. I guess we could get very complicated and talk about grabbing a bunch of objects at the same time and doing some kind of putting together with your hands and go we could go down that path I am going to try to keep away from that that is a bit more complex and I want to cover today. Let us just talk about positioning. So, I would like to place the object back.

So, how far am I moving the object and where do I have to bring it with respect to myself? And one thing that becomes very important is the positioning, precision requirement, right. So, when I place the object do I have to get everything down and down to the level of sub millimeter accuracy in the virtual world assuming we have perfect one to one correspondence. Does the object have a basin of attraction right?
Maybe that I just leave the object in some location and the software that you are right knows that it needs to snap into a particular location. And you are always happy that it snaps into the right place right and there is no confusion with that. You have seen these kind of techniques being used with just simple line drawing programs right to make the vertices match if you are drawing a nice polygon as a as a closed polygonal curve you may have some settings. So, that it just snaps the ends of the polygon together perfectly.
And so, similar kind of thing will happen here with positioning is there a basin of attraction that will just bring the object to the right place is that reasonable to do for your application. If so, you should not be burdening the user with performing very precise placements unless that is the purpose of it could be a game where you try to see who can position objects the best with the most accuracy.

When we perform manipulation let us think about different types of input device categories. So, input device categories I will divide them into two I will say there are metric where the motions are tracked through space. We should think about the degrees of freedom for these. So, it could be the case that you have a maybe just a simple mouse on a desktop and so you have 2 degrees of freedom perhaps 3, if you have a more sophisticated mouse and it can handle the rotations as well.

One general thing that is very interesting and appropriate important to pay attention to here; note that both of these are essentially all of these input devices that we consider have what I call scaling issues over space and over time. And so we see this already with a computer mouse right there is a speed setting. So, you move your mouse a tiny bit and the pointer on the screen moves some amount and people have different preferences for that, but I just want to point out that you do not need to have perfect one to one correspondence.

Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary

Lecture - 19
Interfaces (locomotion)

First one was locomotion, second one is manipulation, third one is system control, fourth one is social or social interaction and I made a fifth category which I called other, which correspond to specialized kinds of interfaces and methods for one particular kind of application whereas, the first four categories correspond to fundamentals, these may show up in category 5 as you develop particular systems, but you may also find very unique issues.
For example in category 5 maybe you want to develop software or write a textbook, maybe you want to play some kind of a hand to hand combat game with your friends, with some kind of actual interaction at a physical level. So, different interfaces would arise; obviously, for problems such as that, they are very specialized and yet some of the general principles may still apply.

So, locomotion which I will call traveling in this alternate, world that we are making could be virtual could be some kind of capture of the real world. And in order to understand locomotion, I think we should make two different extremes and reason about them. So, one of them I will call all physical and the other extreme I will call all virtual and then we will have this kind of spectrum between them and let us go to the all physical case first.
So, in the all physical case, suppose we are in a cave like environment and we can walk around and there is no other way to induce a motion beyond the actual physical walking that you are doing, and the moving of your head to change the viewpoint and that is all or if you put on a head mounted display. Suppose you are in an entire and an entire cleared room in the physical world, and you can walk around, you can look at things, you can get down on the floor and lie down and look up to anything you want. All of the physical motions are being captured and the viewpoint that you observe everything from is being tracked and transformed into the alternate world so, that is there is no mismatch of any kind right.
So, that is a very nice extreme that is the most comfortable case assuming there is no latencies or other kinds of tracking artifacts and display artifacts and this is the most comfortable case. So, let us say walk around in an open space. If you had like to walk around in an enormous open space, then you will need a lot of physical space for this.
So, it is not always very practical.

But one easier alternative is if I am in a rotating chair, I could actually just rotate my body like this and then push forward, and then I can rotate back some other direction and then push forward right. If I did that then I am using one more degree of freedom from the physical world, I am actually doing the all correctly, but the mismatch then corresponds only to the additional components the pitch in the role and of course, the translation or positional changes. So, this by rotating in the chair it is more comfortable for the for the user, because it involves less of this rotational vection may eliminate the rotational vection altogether if the tracking and everything else works correctly.
So, that is interesting; however, not everyone going to be sitting in a rotating chair and if you rotate too much, if you have a cable system then you become tangled up in the cables. So, these are the interesting kinds of tradeoffs that we have when thinking about locomotion. So, over here closer to all physical is you could rotate with a swivel chair or office chair and over here you rotate this is just the yaw component. So, I should put this yaw with controller or buttons on the keyboard or the keyboard is not at all ideal.

How to apply controller based locomotion, when you may recall the chain of transformations T viewport, T canonical view, T eye which we broke into two components for left and right eyes in the stereo case, but I am not going to worry about that today and T rigid body right. So, these are the transforms that we did before worrying about rendering just trying to make the geometric models end up in the right place. So, they end up being in the right place on the screen corresponding to the viewpoint of the eye.

So, the all physical case T eye obtained by tracking in the all virtual case right I mean the virtual being just a locomotion part. So, if the locomotion part is entirely virtual, then T eye is entirely determined, by the controller operation and so, you may be press buttons move knobs on the controller.

So, there are several choices for the center of rotation and, while it is a great source of debate, I think I found that various people will have varying opinions about what is the best and most comfortable way. So, I am trying to think, I think I think I think my favorite was having it be the center of the chair. So, when you are leaning it feels like you are rotating like this, but I cannot even recall any more so and so, which one was my favorite.
So, remember as you as you go forward, you get optical flow present it to your eyes that goes outward, when you go backwards it goes inward and we talked about the other cases when you are rotating or moving sideways. So, problem is the vestibular mismatch, or conflict and, you may recall that I mentioned the case where, if you instantaneously start moving that is actually more comfortable right.
If you instantaneously go from being stopped to being at a full normal walking speed, that is actually more comfortable than gradually ramping up because, when you gradually ramp up you may have for example, 2 or 3 seconds of vestibular mismatch because, your eyes are perceiving an acceleration, but your vestibular organs that you do not have that right, then we said that just doing a quick impulse is better is more comfortable, you can do the same thing for orientation.
So, you could you can make it. So, that every time you tap a button on the controller for example, you may rotate by 5 or 10 degrees rather than doing a gradual circular motion. And this also seems to help with this vestibular mismatch.
Now, if every time you tap the controller you rotate 90 degrees, you may have a lot of confusion about which way you are facing, you get a kind of disorientation from that if you every time you tap the controller you only move 2 degrees, then you end up having to contact a controller many times really fast. In which case you will experience vection again, you will have the rotational vection.
So, there is a kind of balance you have to strike between having them be very small motions and being very large motions, they each have an unfortunate side effect and whether, or not some number in between ends up working really well for everyone, I think it remains to be seen.
So, in the 10 degree range 10 to 15 degrees perhaps that is a good number to try and every time you hit the button you rotate 10 degrees or. So, and that is still not going to be as comfortable as sitting in the chair and, rotating your body in the physical world if you have a rotating chair and, do not get tangled up in the cables right.
And you understand that in the case of circular motion even at a constant angular velocity, there is vestibular conflict right that makes sense right you may remember, that let me just write it quickly.
(Refer Slide Time: 26:51)

So, rotation or circular virtual motion; So, suppose this is the radius vector, in the virtual world or alternate world I am moving along in a circle, at constant angular velocity omega and from basic physics the linear velocity is equal to omega cross r, the omega component is along the Z axis, if you write it as a three dimensional vector has 1, outward component right maybe it is not the Z axis here maybe the Y axis.
So, because of the axis that we have chosen to use, In physics and most of engineering out would be the Z axis. So, it is the outward axis to keep from avoiding confusion. So, this would give you a linear velocity vector V that seems appropriate because, as this if you are on maybe go around staying around, this is the instantaneous direction you are moving you are about to be hurled in that direction ok. Except for on the forces of that are due to accelerations which an acceleration is using basic formulas from physics is omega cross V, which is then equal to omega cross omega cross r, which gives you a component like this of acceleration right so, inward.
So, there is an acceleration component and, if you present visual stimulation that convinces your brain that you are in fact, rotating in a circular arc like this at constant angular velocity, then you are perceiving and acceleration again with your eyes non 0 and your vestibular organ knows that you are not rotating in this way and not does not perceive this acceleration. So, you have a mismatch.
So, in the straight walking case, if you are walking in constant velocity there is no mismatch right, there are no accelerations going on, but in the case of turning just be very careful this is a very uncomfortable case and, I think it ends up being one ends up being effectively the worst case these kind of rotations. So, we need to do something better to handle this difficulty.
And I will say one final thing even, if you are moving along a constant velocity, I still do not think it is completely comfortable. So, you start looking around in different directions you see optical flow the system may be working perfectly, or nearly perfectly I still sometimes experience discomfort and, I am not completely sure why, I think it is left for researchers to figure that out perhaps you will figure out why that is happening.
I do not think there is any case in the real world, where we are moving along truly at constant velocity with no additional forces right, even if you are on a train, or in a car wherever you might be doing I am coasting on along on a bicycle, there are always more vibrations more motions going on more stimulation to the vestibular oregano along different directions and to have perfectly smooth straight motions with no other stimulation going on is I think odd, it is a bit unusual and I find that uncomfortable in virtual reality myself is.
So, even at constant speed; So, just something to pay attention to; however, we know that when there is a mismatch, it leads to discomfort for large fraction of people, if there is no mismatch may still cause some discomfort for some people and we are trying to figure out why all right.

So, in response to this what you should be thinking about, in the design of a virtual reality system is the following. So, it is continuous self motion important or critical all right.
So, for the application that you are building the experience that you are making is it important that you drag your virtual self, through all the gradual motions that we would be doing in the real world right, what if I can just point somewhere where I had like to go perhaps on some kind of map that pops up right, where even if I am in a big room I can just point and click and say I want to be there.
So, just something to pay attention to you can do, when I called teleporting just point and click where you had like to go, if you have a virtual concert you had a virtual you had a sporting event maybe you are in a virtual movie theater could be watching a live sporting event perhaps and, there may be cameras located at different places inside of an actual stadium, do you want to try to do all of the intermediate motions, or would you rather just have a seating chart pop up and, then you just click where you want to sit very often when you buy tickets for a concert you have a seating chart anyway. So, you might as well just have it set as soon as you click and buy the ticket you just appear there and you are in there place.
So, you may decide select a seat from a chart, or map right perhaps, you want to navigate in a city and you have panoramas that have been captured like, in the case of Google street view you may have a bunch of panoramas of a city, you do not even have intermediate video that will transition you from place to place people may eventually make all these things and connect everything together. But while you do not have intermediate video, why not just pop up a little map and you can see where the different places are you can stand and, you just click where you want to go makes sense.
So, even if you do not have the data you may you may you may guide you towards a more comfortable experience because, standing and looking at a panorama is very comfortable because, it does not have this kind of vestibular mismatch. There is one interesting general topic called way finding and, this goes back to what I mentioned in the very first lecture, when we as humans or other organisms explore physical spaces, we end up developing play cells in our brains part of the neural structure, we start allocating neurons towards particular places and they fire the neurons fire, whenever we arrive back in those places again.