Tag Archives: Embodiment

Creating Memories in Virtual Reality.

I love  Metaworld ‘s introduction about creating memories despite the fact they are using the social presence as their main attraction, which I completely disregard in my research, … for now. Anyway, you are wondering why would we want to create memories in a virtual environment at the first place? Isn’t it better to go out in the physical world and meet real people?

metaworld-chess
Playing chess inside Metaworld

Well, it doesn’t compare. The whole idea of using VR to create memories is based on a couple of principles that are much more powerful to set up in VR : associations and embodiment.

Firstly, memories love associations. It is by associating one new information in a network of previously formed memories, that we are going to make sense of it and remember it. We will be able to retrieve the memory more efficiently later on by accessing the same network.

Secondly, memories are embodied. We don’t work like computers. We don’t have a hard drive or a cloud to engrave information one by one in a linear manner. We are storing information on a multi modal level within a mind-body system . So the more senses we are using when accessing a new information, which call upon the previous principle of association, the better we will be able to form new memories.

By proposing a persistent online world where we’re entering by immersing ourself with a HMD (Head Mounted Display), a head phone and a couple of controllers in our hands, we are already covering most of our senses. However, in VR we don’t have the same constraints as in the physical world. Anything is possible that can serve the purpose the participant is after. We can fly, we can travel into space, swim in the ocean or go inside a volcano.

Now, one of the debate about embodiment is the question of the avatar. When we enter into VR as ourself, we can see our hands, we don’t want to see a cartoonist body of ours. Although, if we want to interact with other people, we need to see them, somehow. Metaworld solves this issue by using a quite cartoonist avatars with non attached detailed hands. That way, you can see your own hands and other people hands with their avatar.  Let’s try it.

Is Augmented Reality the next design tool?

What role Augmented Reality (AR) will be playing in the architecture practice and the design of our built environment? How will companies use this technology to attract more consumers? Lets look at those two very different ways to use the power of AR:  as a design tool and as a consumerism device.

Looks like Microsoft is taking the first route with the Hololens. With a variety of polished video presentations, they are showing the Hololens as this amazing new tool to help design interiors, buildings or even a whole new world. It even let you communicate in real time with your teammates from another continent. As an architect, Greg Lynn seems enthusiastic after having use the HoloLens at the Venice Architecture Biennale 2016 (interview from Dezeen here).   For interior designer lacking of vision, this video shows the potential. And to be be really blown away and transported in the future, check out this TED talk from the visionary Alex Kipman.

Of course, I can’t agree more with him on the fact that all those 2D interfaces, displays and screens that have invades our life are born obsolete. We are moving creatures always exploring around us, using our whole body with all our senses to interact with our environment. We are not supposed to be locked behind a screen all day. But, I am digressing onto my research topic here. Even though this is very exciting topic,  we still don’t really know how we are actually interacting with those virtual objects embedded in our physical surroundings. I will look into VR and user experience in another post.

In the meantime, looking at the other side of the coin, the consumerism version of AR, perhaps Keiichi Matsuda vision with his “Hyper-Reality” production is closer to what will happen sooner than later.

HYPER-REALITY from Keiichi Matsuda on Vimeo.

If VR is not the answer, AR is not neither. What I am really looking forward to is when those two technologies will merged into a Mixed Reality (MR) integrated into some sort of contact lenses.

Testing Oculus DK2 – Leap Motion Virtual Reality Blocks Demo.

Last week, I had the opportunity to try out the Oculus Rift – Leap Motion Demo in Goldsmiths Digital Studio. Here is the set up: with the Oculus Rift on your head,  you stand in the middle of the room, a couple of wire climbing up to the ceiling. There is enough space around you to take a few steps in every directions. A small camera sitting on the front wall will capture the movement of your head  in real time: rotating, tilting and panning. Stick on the face of the Head Mounted Display (HMD) is the Leap Motion sensor which will capture your hands movements in front of you. An ambient music is running in the background for everyone in the room to listen to. Sounds, triggered by your movement as well as your reactive expressions will give any observer present in the room a great incentive to try the device for themself. That’s all they perceive from the outside though.

From your point of view (the participant), it is a very different situation. You are immersed in a simple world made of a grid like grey plane levelled up with your chest and surrounded by a black environment. A variety of solids (3D geometric shapes) like cubes, cuboids and other polyhedra randomly spread on the grid surface will attract your attention. Right in front of you is a little character guiding you through available gestures you can use to interact with those solids. By pinching with both hands, you can create more solids, by opening one hand you are able to point at different shapes to choose from with a finger of your other hand. You can actually interact with all those solids around you, pick them up, build walls or even throw them away. Then,  with both hands flat out, palms up, moving gently up, gravity will go away, and all the polyhedra will float above you. The opposite gesture, palms down, will bring gravity back and all the solids down to the ground.

This interaction makes you feel a sense of power and control over things you can only dream of. By playing for a few minutes, because the interactions with those solids is so subtle, your sense of touch will rapidly becoming more accurate and develop some level of haptic feedback.

So, yes, great experience overall, being able to see your hands in Virtual Reality is definitely adding up to the feeling of immersion. It is not just looking around in a 360 panoramic view. By being able to interact with your environment directly using your hands and moving things around, the sensation of presence in another space is undeniable.

Hustling Space-Body Phenomenon

AVPD Hitchcock-hallway
AVPD Hitchcock-hallway
Experiencing with space, placing the participant in an environment that will shift his perspective, is a powerful way to question who we are and how much we relate to our surroundings. AVPD, an artistic spatial laboratory from Copenhagen,  is using architecture language to explore and “rethink the triangular constellation of the subject, the object and the context”. Their work evolves by hustling the space-body phenomenon. It makes you realised how much our perception of the world relates on our acquired experiences and emotions.
Once aware of this phenomenon, we can use it at our advantage by reconfiguring our spatial understanding of the world. What I am interested in is to extract from those experiments and art installations some data that could lead to quantify the usefulness of specific architectural features at helping someone to navigate the physical world as well as the virtual world.

Report: The VI International Conference on Spatial Cognition 

birds

This article was originally posted at Goldsmiths Department of Computing’s Blog here.

PhD student Pierre-François Gerard reports on the International Conference on Spatial Cognition, which took place in Rome on 6-11 September 2015.


pfgIt felt good to leave the already cold London weather early this September and land in Roma’s Mediterranean climate for a whole week of International Conference on Spatial Cognition. What a city, what a history! My daytime was packed with talks and lectures, each giving a different take on situated cognition. My nights were dedicated to applying all those theories along hours of walking and navigating this old city filled with memories.

The conference venue was an interesting building to start with. Situated in an old neighbourhood east of Termini Station, part of Sapienza – Università di Roma, the faculty of psychology was barely recognisable from the street. However, once you got inside, the space was quite remarkable. There was this wide and long mildly inclined ramp punctuated by little steps distributing students and conference participants alike to the four levels of rooms and auditorium. At the back, there was a large court yard to share thoughts and eat lunch in the sunlight.

Keynote speakers – Scientific Method

The main keynotes were given on the top floor. Arrived only on Tuesday afternoon, I missed Monday keynote lecture with Kevin O’Regan: Constructing space: A theoretical basis for how naive artificial or biological agents can construct spatial notions . A couple of early friends informed me that although O’Regan is a quite prolific author, this presentation wasn’t that great. There is plenty to catch up online anyway. On Tuesday was Vittorio Gallese important keynote on Embodied Simulation and the Space around us. He explained the main concept of inter- and extra-personal space used by a lot of authors that week.

The first keynote I attended was quite enlightening on the scientific methodology developed by psychologists. Yan Bao, associate professor from Peking University, explained step by step what is “attention” and how does it work through human’s eyes. To do so, she scientifically answered one very specific question by doing one very specific experiment. From there, a new question arise that lead to the next experiment that will bring a new answer and suggest the following question, and so forth. She presented a cascade of 12 studies based on a cueing task, mainly to demonstrate the effect of “Inhibition of Return”; the ecological significance of this mechanism being that it favours novelty and curiosity.

The next keynote, Inter-subjective relations in lived space and instituted space was given by Shaun Gallagher, an interesting fellow American philosopher. He is actually working with astronauts, trying to understand their feelings in space travel using Virtual Reality. Where it really pumped me up was when he started to talk about how architecture shapes our experience and how we can modulate the way we are experiencing things by modifying our environment. He also coined a powerful concept which is the “affordance landscape”. I will follow him closely.

minority

My favourite lecture was given by Sergei Gepshtein on Solid field of sensitivity: Perceptual structure of immersive space. He works with two well known people, Alex Mc Dowel and Greg Lynn. Alex Mc Dowel was the art producer behind the Minority Report interface and many other film productions. Greg Lynn represent the avant garde of the digital turn in architecture in the nineties with the concept of folding and topological geometry. After a quick reminder of what is the “perspective” we’ve been living by since the 15th century (thanks to Alberti’s “The Pictura”), Gepshtein went on to explain how he is working to change this paradigm; to get past the restriction of this portable window. Their project uses different techniques to go from cinema to immersion by creating maps that can be transferred in solid space: sensory mapping, multi modal design, adaptive smart environment, mixed realities are just a few of the concept mentioned during this really exciting lecture.

Talks – eclectic topics

Every day started with three simultaneous symposium. A lot to choose, from a large panel of research. A variety of fields were represented: psychology, neuroscience, computer science, architecture, social studies and philosophy, to name just the main ones.

Starting from a philosophical point of view to explain “space concept”, T. Holichka defined what is a virtual place at the confluence of possible worlds and fictional worlds. The importance of the concept of affordances (Warren 1984) and places were then discussed by Jonietz & Timpf from an GIS (Geographic Information System) perspective. After that, some phenomenology were involved by Nitsche to analyse impressionist paintings with the notion of distancing instead of understanding. The main question really was: “How do we perceive space?” which brought the notion of situated knowledge.

The next talk centred on a potential new field of research which I am relating to – Computing Embodied Architecture. Prof. E. Ackerman’s talk on developing more appropriate self-directed learning space for children definitely pulled some strings. Then came Paloma G. Rojas, student from MIT, with a methodical approach applying computational model to analyse our perception of space; best hint so far directly related to my own research.

Wednesday’s symposium on Body & Space, explored the concept of peripersonal space and relative perception through the lens of Virtual Reality (VR). You can find more about this research by searching for the CS-IVR Lab, the Immersive Virtual Reality Laboratory and for Mirage Lab, the Multi Sensory Illusions Laboratory.

Thursday’s theme was Navigation. Researchers presented studies exploring how blind people, children, men and women were using different mechanism to find their way around.

On Friday, we returned to the theme of Embodied Space in Architecture with a neuroscience twist. Peri and extra-personal space were still leading the dance backed up with neurophysiological correlations. One specially engaging presentation on central and peripheral vision by Rooney, brought a striking point on how vision drives two different type of embodiment: projected and extended. The last talk I followed, before having to catch the train, was given by Van der Ham, on human navigation in real and virtual environments, and the role of locomotion. The experiment she presented showed the closest settings to mine: participants have to remember their way inside a 3D virtual environment with landmark images hanging on the the wall.

To summarise, an eclectic field of research was represented at this conference. A variety of presentations were showing a large range of disciplines to study the relation between humans and their surroundings and how we are perceiving our body into space. Embodiment, inter and extra -personal space, affordances and places are the main concepts embedded in the body of those researches. VR is used throughout those studies. It is a fantastic tool of research that allows to approach real world situations yet offering a wide range of control, flexibility and scalability on the designed VEs.

Posters – People from all over the world

My poster made a good impression. It gave me the opportunity to start a few conversations with different people from all over the world: Columbian, Swedish, Turkish, French and a couple of Italians. Each had their own take on my project, so being here really brought me a lot of value and new perspectives on what I am working on.

Overall, I had a really great experience. From a scientific point of view, it made me realised how narrow the field of research has to be to conduct valid case studies that answer one specific question at a time. Psychology research definitely set definitely the measure on that matter, by processing one variable in a very controlled environment. However, a new trend is to take research much closer to real life situations. It also brings a lot more data to manage. That is where computation power become really handy for two mains reasons: on one hand, computational models help to simulate systems and behaviours, on the other hand, visualisation engine and Virtual Reality help simulate close to real life environments to work with.

The interdisciplinarity of research also aroused my attention – psychologists trying to make sense of architects’ approach about space perception and embodiment, computer scientists questioning behavioural methodology, or even psychologists between themselves – these were all common discussions during the conference. Only a few presentations were really cross-disciplinary; this is a very interesting subject way easier to debate than to put into real practice.

Most of all, it is the excitement and the enthusiasm of those dozen of researchers from every corner of the earth, supported by the strong foundations of Romans walls and history, that I will carry along my own pursuit of adding a stone to existing knowledge.


Pierre-François Gerard is studying for a PhD in Computer Science

This Post has been published on the Goldsmiths Computing Department Blog on 23/10/2015