Tag Archives: Interaction

Report from the VR World Congress 2017

Bristol was hosting this three days congress. What a good excuse to explore the West Coast of England. Loved it! How to report from such an big event pact of keynotes, talks, debates and demos about Virtual and Augmented Reality, interactive storytelling and immersive art, architecture visualisation and video game development, to name just a few of the field involved ? I will start with  the general trends, then some key points from the talks, to finish with what really got me hooked.

This event was a solid push from AMD. As far as I can remember, AMD had always an edge to better process 3D on his most known rival Intel. Well, it looks like they are still in the fight, but now to process VR wirelessly with their Nitero tech. And this is important because, being in a virtual environment is pretty cool, however, without any cable in our way, it will be much better.  Hololens has taken that mobility party from the start. They made quite a demo there, bringing forward the Mixed Reality concept. That being said, I am still not convinced with this small field of view and the basic interaction where you pinch things around.

Mk2 in Paris
SoReal in china

In the meantime, the big hype is around VR location-based experiences. Mk2 in Paris looks very neat, curating only high quality content and SoReal, a VR theme Park in China sounds epic. On a hardware note, I am curious to see what kind of headset the Chinese will bring on the market with their DeePoon!

Another main trend is the good old haptic feedback. They are working hard to bring the third sense into the game. We are not sure what shape it will take:  gloves, waves, arms, sensors,…but it was explore in every corner of the congress.

Most important is the effort given to produce high quality content. At this stage, only great content will bring VR to the mass.

Follows bullets points of my tweets and comments of the different talks I followed:

On Wednesday:

  • Vive and the Evolving Ecosystem of VR” with Graham Breen from HTC.
    HTC Vive – Haptic

    What’s next for Vive and his ecosystem evolution? Not much on the hardware side, a part of those haptic gloves shown there. They are focus on helping and promoting content creators with their Viveport platform and the ViveX accelerator.

  • Streaming VR/AR Content over the Web”  with Jamieson Brettle from Google. That’s where it get really exciting! AR/VR through the browser. He was telling about pretty good compression performances for cloud point 3D with Draco. For sound volumetric or spatial audio, they are using Opus with ambisonic compression.
  • “Ghost in the Shell VR – A deep Dive” with Sol Rogers from Rewind.
    He delivered a great talk about how he and his team made Ghost in the Shell. He gave no numbers and ask very nicely not to take a photo!

    That’s all I got from Ghost in the Shell
  • “The Importance of touch: Mid-air haptic feedback for VR & AR” with Tom Carter from Ultrahaptics.
    How cool is that, conducting emotions through mid air haptic feedback? Because, at the end, it is the sense of touch that makes things real.
  • Perception = Reality: Exploiting Perceptual Illusions for Design of Compelling AR & VR Interfaces” with Hrvoje Benko from Microsoft Research. Don’t try to be perfect. Use tricks and perception illusion.

Using perception illusion to extend the experience; Passive haptic; Perspective Projection Mapping;  The rubber hand effect revisited as Haptic Retargetting; Body warping and world warping are very promising technic that make you  believe you interacting with objects and that gives you this haptic feedback.

  • “Virtual Futures Track: The Ethics of VR Risks and Recommendations” with Michael Madar from University of Mainz
    • Why should we care?  Our environment influence our behaviour.
    • VR brings the illusions of place, embodiment and agency.
    • Beneficial vs manipulative in VR, tricky balance to make.
    • Are we going to create a rating system for VR experience?
    • “Users should be made aware that there is evidence that experience in VR can have a lasting influence on behaviour after leaving the virtual environment.”
    • Users should also be made aware that we do not yet know the effects of long-term immersion.”

On Thursday, I went to only one talk from the lovely blue hair Mary Kassin from Google who explained her day-to-day process. She prototypes a lot. Trial, user feedback and iteration make the most of her time. She also mentioned the Tango ready phone to play with AR.

Alright, this is all very exciting, full of ideas, however, what made the biggest impression on me was the couple of VR visualisation agencies showing off their architectural Immersive and Interactive walk-through made with Unreal engine.  With this very realistic real-time rendering, we are getting closer to eliminate the long hours of rendering time for still images for a full immersive experience in the early stage of the process.  That is happening right now!

Virtual Reality Immersion for Christmas.

Over Christmas, my son and I had the opportunity to play around with the HTC Vive. Straightforward to set up, you need at least 2 by 2 meters of free space to enjoy the full range of motion. Overall, that was a couple of weeks of awesomeness. Follows a few lines on the games and experiences we have tried out.
  • “The Lab”: After the tutorial, this is the first place from where you start and learn the trick of the trade of the Vive. The main thing is to get use to moving around by pointing, with one of the controller, a ray of light  on a virtual grid on the floor. Then, pressing the trigger will teleport you to the pointed location. I am not totally convince by this method. It still gives me a kind of thickness and definitely reduce the feeling of presence.

    The Lab – Where you start to use the grid to teleport around
  • “Rec Room” : my son’s favorite. He  played in there for hours. The stranger thing in Rec Room is that you play with strangers; you talk to them like they are just in front of you in the room. Well in a way that’s where they are,  virtually next to you. Then you make a team and go playing a exciting paintball or football game, even though they actually are in another country.

    Rec Room, playing paint ball game with some frenchy.
  • Portal Stories: VR”:  love it, my personal favorite game. Really good adaptation of an already so clever and fun game. A perfect match for the HTC vive. In one sentence, the game is about getting out of a room to the next. It is all about the room. Your living room suddenly develop an infinite potential.

    Portal Stories: VR
  • “Irrational Exuberance: Prologue” : this introduction to a new kind of experience is really well design with subtlety. It starts from a very dark environment with just the title and some strange sphere of energy in your hands. They are encouraging you to use those to follow the sphere that goes along the title itself. Then you start to perceive some pieces of stone floating around. Instinctively, you want to push them away, then they break in smaller pieces. Doing the same movement a bit further, at some point you will hit something bigger which will break as well to bring you a glimpse of the outer space. This is the actual wall that surrounds you. By break the wall, you realise that you are standing inside a meteorite. The experience is very immersive, and taking you to a place you couldn’t have imagine. We want to see more of that!

    Irrational Exuberance: Prologue by Buffalo Vision Immersion:
  •  “Google Earth VR” :  bring you as a giant on earth. It is a fantastic feeling to be able to jump from one city to the next. I loved landing effortlessly at the top of the world on Mount Everest

    Google Earth VR
  •  “Engage” is a Educational Virtual Learning Platform. That is where I get very exited. That is the core of my research project: how can we use this amazing technology to enhance the learning experience, to change education for ever?

    Engage – Main hall with the whale
We tried a few other applications as well like this architectural walkthrough “Bund88” which was not easy to load up probably due to the very quality of the render. However, the experience was so realistic. This is definitely a game changer for the architecture and real estate industry. The Roller coaster just made me sick, no comment. No problem for my son though.

The HTC vive is just one more convincing step towards the development of a new medium with a huge potential not only for entertaining  but more for learning and sharing new skills and knowledge between people all over the world.

I was in those virtual worlds more than two weeks ago and still, I can close my eyes and vividly remember the different environments and interactions I lived then as it was this morning.  We learn through experience. That is what VR is all about. We are barely scratching the surface of this new tool that has the potential to positively impact the world.

360 Panoramic Views with WebVR – Production Workflow

Optimising work is part of me. I hate repeating the same task again and again. We are using computers, that’s what they are good at, repeating a task for us. When it comes to work in a team, the workflow get even more important so that two different persons don’t overlap each other work or come back with always the same questions of “how you doing this?” or “where is this file?”. This is all good sense in theory, very far for being the norm in practice, at least from my experience, working mostly with architects and designers. Furthermore, as a freelance, months will pass without using a specific workflow,  working on a different project with different software. Being able to look back at how we did something  is a must if we don’t want to repeat the same mistake again.
On the Saydnaya project I was working on with Forensic Architecture for Amnesty International, we produce a series of interactive 360 panoramic views developed using the fresh WebVR. We didn’t get enough time to develop it in VR, although it is real-time 3D online in a browser, no plugin, no viewer needed. Pretty awesome stuff! Here is the workflow we used to get those 8 Interactive 360 Panoramic views online:
Saydnaya Circulation in 3dsMax
  1. Some of the 3d models were made with Rhino then exported to 3dsMax, most of the models were made only with 3dsMax avoiding any mistranslation between Rhino and 3dsMax, two very different modellers.
  2. All the texturing and lighting was done in 3dsMax, then baked to texture.
  3. Optimization was a big part of the process.. Models were  exported in OBJ (only meshes) and optimised with MeshLab. Maps were optimized with Photoshop.
  4. Everything was tested in a sandbox in WebGL using MAMP as a Apache Local Server, Atom to edit the PhD and Python files.
Saydnaya Circulation WebGL

If you are interested  and would like more details, let me know in the comment below or via email.

What is WebVR? It is 3 things, see this Reddit post here for more details:
  • WebVR is configuring the VR HMDs
  • WebGL is about drawing in WebVR using mainly three.js
  • HTML5 is the sandbox to play in.

Saydnaya Project link

VR for the mass market with Google Daydream?

The next move in the VR world is the Google Daydream initiative. It sounds more like a big marketing push to take advantage of Google Cardboard success though. Nothing is really new in that product. The neat fabric covered headset comes with a remote that nicely fits inside. Take the remote out, place your phone instead and there you  go in VR. As we can all do already with any cheaper cardboard like version. So what is the big deal?
Daydream Home
Daydream Home
Well, it is about the experience, the user experience (UX). A comfortable headset, a powerful phone and a VR app interface will allow you to navigate between your different apps without the need to take the headset off. They want to keep you inside for longer. Sounds good to me.
However, the first question that comes in mind is what phones are Daydream-compatible? Obviously, the recently released Google Pixel phone is. How clever, right? But what qualifies a phone to be Daydream-ready or compatible? This is not so clear yet and it gets too technical anyway. (check this article if you want to know a bit more). My LG G5 with his LCD screen will probably not make it because of the too high persistence. Only the low-persistence of a Full HD OLED display allows to reduce motion blur enough to make the VR experience long lasting (explanation on low-persistence here).
Anyway, it is good to propose a better VR UX at that affordable scale. Clearly the “future proche” of VR is within the mobile market. Now, do we have the content to keep people inside for longer? And do we have the platform to develop more content? Perhaps we do with the WebVR. I will dig that up in a future post.

Is Augmented Reality the next design tool?

What role Augmented Reality (AR) will be playing in the architecture practice and the design of our built environment? How will companies use this technology to attract more consumers? Lets look at those two very different ways to use the power of AR:  as a design tool and as a consumerism device.

Looks like Microsoft is taking the first route with the Hololens. With a variety of polished video presentations, they are showing the Hololens as this amazing new tool to help design interiors, buildings or even a whole new world. It even let you communicate in real time with your teammates from another continent. As an architect, Greg Lynn seems enthusiastic after having use the HoloLens at the Venice Architecture Biennale 2016 (interview from Dezeen here).   For interior designer lacking of vision, this video shows the potential. And to be be really blown away and transported in the future, check out this TED talk from the visionary Alex Kipman.

Of course, I can’t agree more with him on the fact that all those 2D interfaces, displays and screens that have invades our life are born obsolete. We are moving creatures always exploring around us, using our whole body with all our senses to interact with our environment. We are not supposed to be locked behind a screen all day. But, I am digressing onto my research topic here. Even though this is very exciting topic,  we still don’t really know how we are actually interacting with those virtual objects embedded in our physical surroundings. I will look into VR and user experience in another post.

In the meantime, looking at the other side of the coin, the consumerism version of AR, perhaps Keiichi Matsuda vision with his “Hyper-Reality” production is closer to what will happen sooner than later.

HYPER-REALITY from Keiichi Matsuda on Vimeo.

If VR is not the answer, AR is not neither. What I am really looking forward to is when those two technologies will merged into a Mixed Reality (MR) integrated into some sort of contact lenses.