Tag Archives: Virtual Reality

IVAS eXPeriment

Thirty-five people took part in the Immersive Virtual Architecture Studio (IVAS) between 26th February and 9th March 2018. The experiment was simply presented as a room scale VR experience where you would have to solve a few jigsaw puzzles inside different rooms. This was one of the most important and exciting parts of my PhD research project. Follows a short explanation of context, purpose and approach.

Soapbox VR room running IVAS with Dean.

Context

Homo Sapiens’ biggest achievement, to a certain extent, as a global civilization, has been to transform and adapt the environment to his needs. The main strength to achieve this outcome is Sapiens “spatial awareness”: the ability to perceive and make sense of his spatial environment and the intrinsic sense of agency that it affords. Sapiens developed this ability following different trait, the most recognizable one being known under the field of “architecture”. For more than five thousand years, using bricks and mortar, he built places to fulfill all the different functions required by society: services, religions, politics and other cultural activities. In the 21st century, Virtual Reality (VR), an inherently spatial technology, offers us the perfect medium to test and apply some architectural principles developed over the centuries to structure and navigate today’s overwhelming digital landscape.

Purpose

The built environment has a significant effect on humans behavior in the physical world (1). How does that translate in VR? The overall aim of this project is to establish the foundations of a framework to support the design of Immersive Virtual Environments. Such a framework will have benefits not only for scientists but in every field VR is disrupting such as game design, industrial design, data visualization and learning applications to name just a few.

Approach

This study is exploring ways to evaluate how different architectural elements affect human’s spatial cognition performance using the IVAS. The following steps will be to apply those findings to support specific cognitive tasks for specific users. This particular iteration of the project is looking at two architectural elements arranged following two spatial characteristics. Those fours conditions are tested using three cognitive tasks. Follows a short description of the setup.

Physical Space – Hardware – Mode of Interaction

For most of our history, natural movement has been the only way to navigate our environment and to experience “architecture”, therefore, it is the primary mode of interaction used in this experiment. To accompany this principle, a room-scale VR environment is set up with a minimum of 9 sqm (3mx3m) of navigable space. In this instance, the IVAS exp. happened in two different rooms, at two different sites: Goldsmiths, Hatcham house, 1 and Soapbox, Old Street 68.
The second mode of interaction is the VR system which is composed of an HTC Vive head-mounted display with two wireless hand-held controllers allowing together 18 degrees of freedom (18 DOF) of movement. The headset is tethered to a powerful laptop that runs the simulation.

Prt Screen from Unity running IVAS A1

Virtual Space – Software – 3D Models

The room with approximately the same dimensions as the physical room is modeled in 3D and will serve as the base for the different conditions (architectural scenes) that will be tested. All other 3D assets are modeled using 3Dsmax before being imported in Unity3D where the interactivity is programmed.

Spatial Conditions

Two architectural elements, wall and columns, were studied following two spatial characteristics: enclosure and complexity (3)

  • A1 : Close Columns
  • A2 : Open Columns
  • B1 : Close Walls
  • B2 : Open Walls
Layout of the different conditions.

Three Tasks involving Spatial Cognition

Solving a Jigsaw Puzzle

This task was design to encourage participants to navigate the space in search for all the items needed to solve the puzzle. A stopwatch was encouraging them to do so as fast as possible – a way to measure performance. VR allows to easily track user’s movement: time, position and rotation. Everybody seems to have enjoyed solving the jigsaw puzzle and were very focused on the task. I had to remind them to explore the space before starting the task. Once the puzzle was solved, the participant was automatically transferred to a transition area where he had to rate two experiential qualities.

Rating of Experiential Qualities (REQ)

The spatial analysis can only be meaningful in regards to an equivalent evaluation from a human experience point of view. Evaluating “lived space” (2) can be done by asking participants to rate their experience with each spatial characteristics. This task brings the qualitative human evaluation into the equation. Using a semantic differential scaling technique, subjects were able to differentiate their appraisal using a five-step Likert-like scale. The rating categories were selected to represent previously mentioned properties: enclosure and complexity.

Perspective Taking Task (PTT)

Once out of the IVAS, participants had to answer a few questions on the online questionnaire before completing this last cognitive task. The main purpose of this task is to measure the memorability of each scene (4). It consists of a sequence of 16 pairs of images. For each pair, one of the images was taken from one of the explored room, the other image was taken from a room not visited. The participant had to identify which image relates to one of the scenes he had experienced.

Perspective Taking Task – All the views
Perspective Taking Task – Pair 03
Perspective Taking Task – Pair 11

Space Syntax Design Analysis

The design analysis using Space Syntax approach will give us an objective measure of each considered spatial characteristics. By combining both “Isovist” and “Visibility Graph” techniques, we obtain a number of measurands (3). In this case, we will be using the following measurands to represent the best predictor variables for the spatial characteristic considered:

The spatial qualities and their related measurands are :

  • Enclosure: “isovist openness” and “jaggedness”;
  • Complexity : “number of vertices”, “vertex density”, “roundness” and “jaggedness”;

Early Observations

A quick glance at the data shows that participants experienced spatial complexity as intended in the scenes designed with the columns. Their average best performance comes out of the scene with the closest room with columns. However, the feeling of openness doesn’t seem to be related to the number of windows in the room. One explanation for this is most probably because there was a texture on the glass. It wasn’t completely transparent. A participant even said: “I didn’t realize that there were glass panel walls!”
This is just a short intro of the kind of conclusions I am working on. This experiment is bringing plenty of good data to dig into, some with positive results some negatives. I have a few pages to fill with that discussion (check further posts).


References:
  1. Arthur E. Stamps. Mystery, complexity, legibility and coherence: A meta-analysis. Journal of Environmental Psychology, 24(1):1–16, 3 2004
  2. Annemarie S. Dosen and Michael J. Ostwald. Lived space and geometric space: comparing people’s perceptions of spatial enclosure and exposure with metric room properties and isovist measures. Architectural Science Review, 60(1):62–77
  3. Jan M. Wiener and Gerald Franz. Isovists as a Means to Predict Spatial Experience and Behavior. pages 42–57. Springer Berlin Heidelberg, 2005.
  4. Barbara Tversky and Bridgette Martin Hard. Embodied and disembodied cognition: Spatial perspective-taking. Cognition, 110(1):124–129, 1 2009.

 

Report from the VR World Congress 2017

Bristol was hosting this three days congress. What a good excuse to explore the West Coast of England. Loved it! How to report from such an big event pact of keynotes, talks, debates and demos about Virtual and Augmented Reality, interactive storytelling and immersive art, architecture visualisation and video game development, to name just a few of the field involved ? I will start with  the general trends, then some key points from the talks, to finish with what really got me hooked.

This event was a solid push from AMD. As far as I can remember, AMD had always an edge to better process 3D on his most known rival Intel. Well, it looks like they are still in the fight, but now to process VR wirelessly with their Nitero tech. And this is important because, being in a virtual environment is pretty cool, however, without any cable in our way, it will be much better.  Hololens has taken that mobility party from the start. They made quite a demo there, bringing forward the Mixed Reality concept. That being said, I am still not convinced with this small field of view and the basic interaction where you pinch things around.

Mk2 in Paris
SoReal in china

In the meantime, the big hype is around VR location-based experiences. Mk2 in Paris looks very neat, curating only high quality content and SoReal, a VR theme Park in China sounds epic. On a hardware note, I am curious to see what kind of headset the Chinese will bring on the market with their DeePoon!

Another main trend is the good old haptic feedback. They are working hard to bring the third sense into the game. We are not sure what shape it will take:  gloves, waves, arms, sensors,…but it was explore in every corner of the congress.

Most important is the effort given to produce high quality content. At this stage, only great content will bring VR to the mass.

Follows bullets points of my tweets and comments of the different talks I followed:

On Wednesday:

  • Vive and the Evolving Ecosystem of VR” with Graham Breen from HTC.
    HTC Vive – Haptic

    What’s next for Vive and his ecosystem evolution? Not much on the hardware side, a part of those haptic gloves shown there. They are focus on helping and promoting content creators with their Viveport platform and the ViveX accelerator.

  • Streaming VR/AR Content over the Web”  with Jamieson Brettle from Google. That’s where it get really exciting! AR/VR through the browser. He was telling about pretty good compression performances for cloud point 3D with Draco. For sound volumetric or spatial audio, they are using Opus with ambisonic compression.
  • “Ghost in the Shell VR – A deep Dive” with Sol Rogers from Rewind.
    He delivered a great talk about how he and his team made Ghost in the Shell. He gave no numbers and ask very nicely not to take a photo!

    That’s all I got from Ghost in the Shell
  • “The Importance of touch: Mid-air haptic feedback for VR & AR” with Tom Carter from Ultrahaptics.
    How cool is that, conducting emotions through mid air haptic feedback? Because, at the end, it is the sense of touch that makes things real.
  • Perception = Reality: Exploiting Perceptual Illusions for Design of Compelling AR & VR Interfaces” with Hrvoje Benko from Microsoft Research. Don’t try to be perfect. Use tricks and perception illusion.

Using perception illusion to extend the experience; Passive haptic; Perspective Projection Mapping;  The rubber hand effect revisited as Haptic Retargetting; Body warping and world warping are very promising technic that make you  believe you interacting with objects and that gives you this haptic feedback.

  • “Virtual Futures Track: The Ethics of VR Risks and Recommendations” with Michael Madar from University of Mainz
    • Why should we care?  Our environment influence our behaviour.
    • VR brings the illusions of place, embodiment and agency.
    • Beneficial vs manipulative in VR, tricky balance to make.
    • Are we going to create a rating system for VR experience?
    • “Users should be made aware that there is evidence that experience in VR can have a lasting influence on behaviour after leaving the virtual environment.”
    • Users should also be made aware that we do not yet know the effects of long-term immersion.”

On Thursday, I went to only one talk from the lovely blue hair Mary Kassin from Google who explained her day-to-day process. She prototypes a lot. Trial, user feedback and iteration make the most of her time. She also mentioned the Tango ready phone to play with AR.

Alright, this is all very exciting, full of ideas, however, what made the biggest impression on me was the couple of VR visualisation agencies showing off their architectural Immersive and Interactive walk-through made with Unreal engine.  With this very realistic real-time rendering, we are getting closer to eliminate the long hours of rendering time for still images for a full immersive experience in the early stage of the process.  That is happening right now!

London VR Meetup Special Education Rocked.

Monday evening was my first VR meetup for developers, special education; right down my alley.  Really dynamic, full of enthusiastic developers and makers, a few exciting demos on the first floor and loads of speakers on the ground floor.  All this in a good old fashion London building, the Hackney House.  This blog will cover mostly my experiences with all the demos I have experienced.

The speaker room

First thing I tried was the full body VR immersion kit. It took some time to adjust all the straps around every main bones. Once calibration is done,  fit the Oculus Rift on your head and there you are looking in the mirror at your avatar,  a character attached to your  body and movements. Despite the lack of space (the casual black environment),  a sense of scale was given by a couple of statues. A few tweaks using UX interface designed by Dr Harry Brenton, and you have a tail, a gigantic arm or an alien head. The level of presence get higher with every step. I can’t wait to have full hand and fingers tracked.

Harry Brenton slide on Character Design

The next experience was a real-time holograph in VR. You get the headset on to meet the holographic projection of someone in Lithuania in real-time. She couldn’t see me, though, I could see her, and talk to her over the phone saying “hello” and she would wave her hands. Not yet like in Ghost in the Shell but it works, telepresence, yes!  However, it would be nice to have a sense of  geographic location in the environment, wouldn’t it?

The most exciting demo was about making VR in VR by Constructive Labs.  Still in an early stage, the demo let you manipulate objects in VR using HTC Vive and controllers like you would use the mouse and keyboard in 3Dsmax or Unity. On top, we were able to do that in interaction with another person. Their idea is to develop a VROS, Virtual Reality Operating system.  Pretty cool stuff! However, again, the environment was really poor. They just used a model of a random brick tower as a gimmick surrounding.

The last demo I tried was more on the interactive storytelling treat made by Bandits Collective. Following the hot panel discussion about 360 video a bit earlier in the evening, I think they nailed it quite well. Their intro for a short movie brings you in a computer generated 360 environment animation based and stylish. It is the environment that make you stay where you are and look where the action happens, though you can check all around and even move. There is no interaction. The action is happening around you. Very promising!

There were a couple of other demos with cardboard and other mobile VR. We know what we get there. I am much more interested with what we don’t know. There is still so much to explore, mostly for me, as you can tell, about spatial environment in VR.   On those four demos, only the 360 story has a designed spatial environment that support the experience.

How Much the Mind Can Bend?

If an artist is able to transform the essence of a place using visual illusions, I am wondering how far we can go by exploring the potential of Immersive Virtual Environments (IVE).

Peter Kogler’s installations absorbs the physical constraints of the places he is investigating into a new space/time paradigm. Those constraints are made of walls, floors and ceilings – the three fundamental elements of architecture. By mapping those curvy textures and lines all over those elements, he creates the illusion of flow, of oneness. Usual sense of up and down, right and left, front and back, vanished under the optical illusion of interlaced black lines. This artefact forces the participant to reevaluate his understanding of what his mind takes for granted. Walls appears as swirling vortex . Distances are altered. Floors rise and ceilings bend.

Peter Kogler Room Installation Illusion – Image Courtesy of MyModernMet.com

Now, transposing this into the realm of IVE, where the only constraints are coming from the electronic limitations of hardware to process the information, what kind of strange world will we be able to imagine?

Peter Kogler Room Installation Illusion – Image Courtesy of MyModernMet.com

This is an artist point of view, it is supposed to question the possibilities. However, from an Enactive Approach, the process he is using is very relevant to the design of meaningful IVE. This artwork can only exist through the interaction between the build environment and the moving body.

Similarly, it is by copying those constraints into the electronic realm, that we are allowing a smooth transition to the infinite possibilities of the virtual. As our senses will adapt to this new realm, we will be able to create complete out of this world environments serving the only purpose of the virtual experience. That being said, how will the embodied mind adapt to such an alien environment ? Is the body a constant  or will it change like the astronauts body into space?

Virtual Reality Immersion for Christmas.

Over Christmas, my son and I had the opportunity to play around with the HTC Vive. Straightforward to set up, you need at least 2 by 2 meters of free space to enjoy the full range of motion. Overall, that was a couple of weeks of awesomeness. Follows a few lines on the games and experiences we have tried out.
  • “The Lab”: After the tutorial, this is the first place from where you start and learn the trick of the trade of the Vive. The main thing is to get use to moving around by pointing, with one of the controller, a ray of light  on a virtual grid on the floor. Then, pressing the trigger will teleport you to the pointed location. I am not totally convince by this method. It still gives me a kind of thickness and definitely reduce the feeling of presence.

    The Lab – Where you start to use the grid to teleport around
  • “Rec Room” : my son’s favorite. He  played in there for hours. The stranger thing in Rec Room is that you play with strangers; you talk to them like they are just in front of you in the room. Well in a way that’s where they are,  virtually next to you. Then you make a team and go playing a exciting paintball or football game, even though they actually are in another country.

    Rec Room, playing paint ball game with some frenchy.
  • Portal Stories: VR”:  love it, my personal favorite game. Really good adaptation of an already so clever and fun game. A perfect match for the HTC vive. In one sentence, the game is about getting out of a room to the next. It is all about the room. Your living room suddenly develop an infinite potential.

    Portal Stories: VR
  • “Irrational Exuberance: Prologue” : this introduction to a new kind of experience is really well design with subtlety. It starts from a very dark environment with just the title and some strange sphere of energy in your hands. They are encouraging you to use those to follow the sphere that goes along the title itself. Then you start to perceive some pieces of stone floating around. Instinctively, you want to push them away, then they break in smaller pieces. Doing the same movement a bit further, at some point you will hit something bigger which will break as well to bring you a glimpse of the outer space. This is the actual wall that surrounds you. By break the wall, you realise that you are standing inside a meteorite. The experience is very immersive, and taking you to a place you couldn’t have imagine. We want to see more of that!

    Irrational Exuberance: Prologue by Buffalo Vision Immersion:
  •  “Google Earth VR” :  bring you as a giant on earth. It is a fantastic feeling to be able to jump from one city to the next. I loved landing effortlessly at the top of the world on Mount Everest

    Google Earth VR
  •  “Engage” is a Educational Virtual Learning Platform. That is where I get very exited. That is the core of my research project: how can we use this amazing technology to enhance the learning experience, to change education for ever?

    Engage – Main hall with the whale
We tried a few other applications as well like this architectural walkthrough “Bund88” which was not easy to load up probably due to the very quality of the render. However, the experience was so realistic. This is definitely a game changer for the architecture and real estate industry. The Roller coaster just made me sick, no comment. No problem for my son though.

The HTC vive is just one more convincing step towards the development of a new medium with a huge potential not only for entertaining  but more for learning and sharing new skills and knowledge between people all over the world.

I was in those virtual worlds more than two weeks ago and still, I can close my eyes and vividly remember the different environments and interactions I lived then as it was this morning.  We learn through experience. That is what VR is all about. We are barely scratching the surface of this new tool that has the potential to positively impact the world.