Loading
Notes
Study Reminders
Support
Text Version

Set your study reminders

We will email you at these times to remind you to study.
  • Monday

    -

    7am

    +

    Tuesday

    -

    7am

    +

    Wednesday

    -

    7am

    +

    Thursday

    -

    7am

    +

    Friday

    -

    7am

    +

    Saturday

    -

    7am

    +

    Sunday

    -

    7am

    +

Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary
Indian Institute of Technology, Madras

Lecture - 18-3
Interfaces (overview)


There is a nice book called 3D User Interfaces by a Bowman et al which you might find interesting reading.
Locomotion very important in virtual reality, which is moving oneself from place to place right and we have talked about the issues here of this vestibular mismatch with vision. So, you had like to move around you had like to move in a much larger world in virtual reality, then you are moving in the physical space right. Because if you have to match them perfectly one to one then you need an enormous physical space to hold and preserve entirely for your virtual reality experience which is not feasible in many cases most cases.
Another general category is manipulation. It corresponds to all kinds of things. You are reaching into the world, touching, feeling, grabbing maybe grabbing an object, perhaps carrying an object, discarding or dropping an object right. So, all of this interaction with objects in the world, I will put objects here. So, what happens how do we select objects in the world right, and what kind of feedback to we need if we want to be able to grab them? And manipulate them in some way that is comfortable. I would be willing to bet that depth perception becomes important right how far away is that object that I want to grab. So, these are interesting kinds of questions.
Another category is a system control. So, just standard how should menus look, and how to select in menus, do we want some kind of windows. In other words, what is the replacement for the desktop right we have a desktop with windows and icons and we we do our work our smartphones have some sort of version of that with a touchscreen. Again we have icons and we open them up and we do various things.
So, what should the system control look like exactly, how do we interact, and just do our work in a virtual reality environment, it took us a long time to get to the type of desktop environments that we have now or smartphone type environments that we have now what should that look like for virtual reality remains to be seen even just text editing right. So, how should the text appear, readability, comfort, speed things like that so if I have you know fonts all these kind of things and how do I interact with the system in some comfortable way.
For social interaction interfaces social interfaces so chatting, texting, interacting with people, we end up with some difficult let me put face to face, we end up with some difficult challenges, because if you and your friend want to interact in virtual reality, and you are each wearing a black objects in front of your face, you cannot see each other’s eyes.

Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary
Indian Institute of Technology, Madras

Lecture - 20-2
Evaluation of VR Systems
Evaluation of VR systems:
Recall neurobiology where VR was applied in science lab:
Separation of 1) Scientist and 2) Lab

So, what options do you have? I will mention two options. One train yourself to be a critic, I say good critic or maybe a bad critic in the sense of train yourself to be very negative and hostile towards your own designs it is difficult to do maybe. Two, design an experiment and recruit test subjects right. So, recruit people to try. Which one should you choose? I suggest choosing both of them right
So, obviously, the first one is the laziest one right. So, train yourself and then you can evaluate what you are doing with very quick frequency and without doing a lot of extra work. Number 2 may require significant amount of extra effort and work and so you end up doing this depending on how much your life depends on it right. So, what exactly you are doing, you build entire company around this, you had better do a lot of experimentation and testing are you building your entire career basing it on this then do a lot of testing. If it is just something for fun to share with your classmates then fine let a few of your classmates try it, and get some feedback and then may be enough to make you happy. So, choose both.

Let me give some advice under these two cases. So, what if we choose number 1? So for yourself, we give some key points one thing is recognized your own bias, we recognize your own bias. So, one of the things I will say about that is you are going to know more about what you are trying to do, what is the intention of your experience than anybody else right.
So, if you know what you are trying to do, it may be the case that you have some kind of motion occurring, and you know that the object is moving not you. But if someone tries it, and they do not know either way, they may think that they are the one that is moving, and then they get sick and you complain why are you getting sick and then and then and then you say well you know it looked like I was moving and then the creator might say well, well duh of course, you are not moving the object was moving did not you get it you know. So, it will actually affect the outcome of sickness here in the simulator sickness based on what interpretation your brain is giving. So, you have to be very careful about that.
The other thing which I already mentioned is you might be very proud of what you have made or lazy all right. So, you have done, you want to be done with it, you do not want a bunch of feedback that says do it all over again right or re-iterate. So, you had like to think that you are done right, so that is your bias right. You are either proud or lazy, you do not want to redo it or you are happy of what you have made you are biased by that.
And only you understand what it is supposed to do.


Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary
Indian Institute of Technology, Madras

Lecture - 20-1
Interfaces (social interaction)


Many years ago the idea of virtual communities was developed. This can be we considered as a part of sociology. And how do we, how do we interact and socialize build societies together? The term virtual communities came out of a book by Howard Rheingold. It is a long history of virtual communities. So, one of them is called Plato which is from the 1970, while see 60s and 70s which came from the University of Illinois and was a system for doing for education.
So, in that time there were forums appearing. So, I was no worth computers brought together and people were interacting in the context of classes now a forum like the on the forum we have for the class the online forum seems very natural this is developing a kind of community for learning, right. So, you do not have to have a completely immersive virtual reality experience to develop a community of people who have some kind of common goals or interests. So, I just want to point that out.
So, in addition there are chat rooms, then after a long time email lists, perhaps not as interactive, but we tend to use email lists all the time with threads going and discussions Usenet, Multi User Dungeons been around for many years. So, this idea of virtual communities has developed for quite a while and it is quite an independent thread of virtual reality right. You do not need virtual reality to build virtual communities and have some kind of social interaction right; human beings will do this in many other formats or many other kinds of media.
So, we have it is already been established that they do not have to be physically present in the same physical space to form a community correct, alright. So, a special case of that is through the some kind of virtual the some kind of virtual worlds.
We have seen this idea all over in science fiction tell me just kind of when you list a few books that have inspired a lot of people in this space usually, they involve some kind of dystopian future society or people connected together through virtual reality um.
So, you can look these up if you have not seen them before Neuromancer from the 80s snow crash, from the 90 and ready player one from a few years ago. If you want to read about all of the things that might happened to us one day if we immerse ourselves and virtual reality almost all the time or all the time.
Coming back to the real world MMORPGs is worth looking up if you are not familiar with that Massively Multiplayer Online Role Playing Games right, any of you involved in those, all right.
So, you have heard of this before it. So, it just depends on the on the audience, and these online games they involve some kind of teamwork and interaction perhaps with a bunch of strangers right, all around the world. So, it is fascinating I would say just corresponds to some kind of virtual worlds with social interaction occurring.
And second life this platform started around 2003 grew to have many users 10s of 1000s were connected together at the same time, building some kind of virtual worlds for interaction. The real people who are interacting through avatars so we have cartoon like representations of themselves, they are wearing clothes, they go to the shopping mall, and buy other buy outfits they socialize in virtual of pubs. They started buying real estate there became a currency called Linden dollars that was actually exchangeable for real currencies like you could exchange it for US dollars there is an exchange rate for that.
This is one of the earlier examples of a kind of independent currency. You see bit coin today very often. So, this was a that the currency that evolved in here was a precursor to that. All you need is some kind of social interaction community and people will start exchanging money that becomes effectively real money at least as real as the money we use that are backed by governments. Eventually it got into a lot of trouble in a lot of ways because there is no government inside of here really. And so it became hard to enforce rules and laws and theft and other kinds of things became a problem people would start to trick others out of money and its actually changeable for real money. So, it is a fascinating what happened inside of here.
(Refer Slide Time: 06:04)

So, the creators of second life are were big fans of virtual reality they just did the best they could using the technology of the time which means do it all on a screen do not worry about the problems of head tracking and immersion and simulator sickness. They have worried about the social interaction part and so the people who the creators of second life are continuing to be involved in the virtual reality community and want to see it brought to the next level, where you feel like you are completely inhabiting these worlds, alright.
So, virtual reality is increasing the amount of immersion this feeling of being present together is that important for social interaction or not. I do not know which we started let us say a long time ago with telephones for talking to each other, and then we carry around these things we call smart phones how often are we phoning each other versus I am sending text messages, all right. So, which what is more immersive and why give you more of a feeling of presence? When you are talking to someone on the phone or when your text messaging them? I would think the phone feels a little more immersive.
In fact, I will go the other direction I will add video to that, wow now I have a video conference going with my friend How often do you do that versus just sending a text message all right. So, it is very interesting right.
So, you feel like you get pretty good interactional text. So, I do not think it’s always very clear you know it is not necessarily the case that more immersion more feeling of presence somehow improves the interaction, right. You would like to just roll out of bed and start sending text messages and not really worry about how you look for example, right maybe that is an issue. So, something to pay attention to.
(Refer Slide Time: 07:51)

On the subject of face to face interaction which again might not be necessary at all [FL] maybe we put on our virtual reality headsets and then we just read text messages from each other right. Why would you do that? I do not know.
But if you are spending all of your time inside of virtual reality you might still be texting with your friends then right. And in second life all the interaction was done by texting. And I asked people who are who are who are in that very active in that community there is also an open source of version then I encourage you to try called open sim or open simulator. So, if you are all especially if you are on Linux platforms you may find that very interesting and hopefully people are running servers on that.
So, I asked them why do not you just transmit the voice and they care a lot about the anonymity of it they like to be someone else. You know they become their avatar, and so the texting part let us them communicable with each other and they would walk around in these virtual worlds. And only when they get closed enough within what would be normal hearing distance then you can see the texts of each other. So, the texting replace that.
If you wanted to do an audio track version you may need something like a voice disguise to listen right, because they are already running around with avatars that disguise their appearance. So, you would need to disguise the voice to have it will be comparable, right. So, that that could work you could make a real time voice warbler or some kind of distortion transform to the to the voice, but texting works just finer.
So, when we get to face to face to face interaction we have on the one hand avatars which is some graphical representation of oneself. If you want to look like a monkey it is up to you can be anything you like, you could look like uh, a piece of fruit it does not matter really, but so you decide to have some kind of other representation. Probably something that animates itself now a piece of fruit put in you know you can trying to give you extremes versus real faces.
In other words yourself captured as well as you possibly you can capture it. Maybe just capture with a video camera and transmit, maybe you do something more sophisticated to provide 3 dimensional information. And for the audio part maybe you have text versus the real audio, all right. So, that is the visual part in the first line on the hearing part in the second line.
(Refer Slide Time: 10:27)

So, on this side you are approaching realism you are trying to match the physical world experience, on this side you are approaching fantasy right. You having a chance to escape the real world yet still have some kind of interesting social interaction. So, these are both the interesting and useful extremes. Think about your task think about the experience you are trying to make.
(Refer Slide Time: 10:56)

So, regarding achieving realism how might I do that, right. So, maybe I just placed in the environment like in this classroom today I just put an how many directional camera, all right.
So, using multiple cameras that are synchronized, using lenses, optical techniques, for a wide field of view; however, you want to put it together build a coherent panoramic video and then transmit that. And someone we can look around and feel like they are here that should be very realistic may be better to capture all of the information in stereo or capture the entire light filled somehow. So, they can move their head around, get parallax, get everything working just right that would be an extreme level of realism.
And you know maybe not too difficult if you just want to do a monoscopic, panorama and transmit the live video. You could replace your face to face on a screen video conferencing with full live panorama, all right. So, that is not too bad.
One problem is that if each of you is wearing a head mounted display then in the panorama you will be wearing a head mounted display. So, then I cannot see your eyes, all right. So, we have a great conversation, but every time we meet we have black bricks on our faces right. So, it is kind of unfortunate, but that is one of the outcomes all right. So, what do you want to do about that? You can put little cameras inside. So, I could at least try to capture the part of your face that is being obscured by the headset. But I still have to reconstruct the rim parts that I cannot see any way you know it is a max number how you do it? So what exactly you are you going to reconstruct?
Another interesting problem is that what if I would like to extract the person out of the real world and put them into a virtual world. Maybe I would like to move if you and I are interacting maybe I would like to move us to some virtual world or perhaps we would like to be standing in Paris talking together right, make it appear like that. Now, you have to do a lot of difficult work to extract some representation of our bodies and move us to that location right the moving parts not too hard once you have a good representation of the entire person.
So, it is a lot of work and it is going to be very challenging to get that accurate without having access to your own private motion capture studio or you can put markers all over the body and try to completely and perfectly extract representation of the person. So, that is a very difficult problem. So, it was very easy if we all stay in the same place and we just captured with omnidirectional video becomes very challenging if we want to extract just the body maybe put a blue screen behind and extract the body and move it somewhere else. So, make sense and always take into account problems of delays if we want to move ourselves to another location on the earth for example, or some other virtual world there may be problems with synchronization. And if you see my lips moving, it might not be synchronized perfectly with the audio. So, another issue maybe it is better to delay them both just, so that they are synchronized, all right.


Virtual Reality
Prof. Steve LaValle
Department of Multidisciplinary
Indian Institute of Technology, Madras

Lecture - 20
Interfaces (system control)


So, one of the most common interfaces that we use for system control with regard to our computers, smartphones, laptops, pc with a screen is a graphical menu. And you of course, have the usual questions of you know how readable are the fonts, you keep the number of colors to a minimum and so forth so some basic style and comfort issues. You know how else could you select options from a menu, you could do it entirely by which way your head is looking I guess you could look at a particular menu option, you could have a menu appear up in front of you, try to select things that way or you may use some controller buttons or keys if you have a keyboard.
How big should the menu appear to be? So, if you have a kind of menu, and this also could apply by the way to just making a maybe I want to make a virtual or a web browser appear as well right. So, I just want to start reading a newspaper articles could be anything like that in addition to menus, how large should this appear. It is best to have it be about one-third of your field of view. If it gets much larger than that then you will be moving your head back and forth to read and that may be uncomfortable. Why are you using your neck muscles, you would not be using that in the real world if you are reading a magazine article or a newspaper for example. So, these are some subtleties to pay attention to for making system control.
Another general thing to think about is what if I need to enter a lot of text. So, it may be that in system control or box appears I need to type something. What if I want to write an entire paper, how do I do that? If you just have the keyboard, then you need to remember that there was a table in the physical world, you put your hands on it, you cannot see the keys anymore, but if you are good enough typist maybe it does not matter too much little frustrating I think. You can feel the bumps on the keyboards in some cases on some of the keys to try to find your location.
But generally speaking if it were up to me to completely reengineer the way we type then at least the way we commonly type they are certainly all sorts of different keyboards and things, then I would recommend separating the keyboard into two parts. And then just putting my hands one on each part and I would like to train myself to just sit and type like this, so that I do not have to reach around for some table and remind myself of that exists in the virtual world. Maybe I can even not have to press buttons and just rig something up to my finger, so I can just very easily do some motions and the typing happens.