Hybrid VR Video Solutions – Best of Both Virtual Worlds

One of the more challenging areas regarding 360 VR video relate to the format’s lack of interactivity; beyond the ability to look around VR films are typically a fairly passive experience. Whilst this is still an extremely powerful immersive experience, adding interactivity can certainly enhance the users overall engagement with the virtual reality environment.

INTERACTIVITY IN VR VIDEO

There are a number of ways this can be tackled and more recently a project I produced along with my programming partners at Multiply developed an interesting hybrid 360 VR video + fully interactive programmed VR experience. This allowed us to deliver a best of both virtual worlds solution.

For non-interactive sequences, content could be filmed in 360 stereoscopic 3D using industry standard equipment from Insta360 – this delivers photorealistic environments filled with living creatures and characters for a fraction of the budget of attempting to develop this as animated 3d scenes especially without entering the “uncanny valley”.

For interactive sequences we pause the video and cut to an associated interactive 360 environment where the users can move and interact freely around and with the objects in the scene, obviously based on the limitations of the interactivity programmed. Once the users has completed the interaction we then cut to the next video in the sequence. The primary limitation of the interactive scenes is generating and displaying photorealistic environments and/or characters within the limited computing power of a stand-alone VR headset and on a reasonable budget.

There are simpler solutions to this, for example using fairly basic interactive video technology, in essence pausing the video to request user input on the direction the narrative would follow in a similar manner to the recent interactive “Bandersnatch” episode of Black Mirror or more historically from the “make your own adventure books”. These can be simple, although managing branching narratives can quickly become cumbersome in themselves. They are simpler and less costly produce as they do not require any 3D modelling nor programming beyond the simplest of button press interactions.

FIRE SAFETY TRAINING IN VR BASIC EXAMPLE

For example in a basic interactive video VR experience for a theoretical fire safety training project, the user could choose which fire extinguisher would be the correct one for a specific type of fire; choosing the wrong one would have dire consequences.

In this basic interaction case the user couldn’t actually (virtually) pick up the fire extinguisher and point/activate it – they would simply click the one they thought was correct and the application would then respond by playing the resulting video – either safely continuing the users training or terminating it as the environment went up in flames.

FIRE SAFETY TRAINING IN VR HYBRID EXAMPLE

In the hybrid version of the same VR training, we could use 360 video to engage the viewer to the point of the interactivity then switch from video to a fully interactive VR environment where they can move around to locate and virtually pick up the fire extinguisher, pull out the safety pin and press the handle to engage it – a more complex and realistic experience all around.

Once they’ve pulled the trigger to engage the fire extinguisher we can again jump immediately to the next VR 360 video and either continue or end the training session as above.

HYBRID VR VIDEO + INTERACTIVE SOLUTION

This type of solution delivers a good level of interactivity and immersion but without the costs of having to model & animate every scene. This hybrid models uses VR film & VR Interactivity to deliver to the key strengths of each.

The video sequences benefit from being photorealistic and can feature anything a typical VR video does in terms of audio whether voice-over or character dialogues and music/effects. What the scenes lack in interactivity is offset by the absolute realism of the film.

The interactive sequences can then deliver completely immersive and fully interactive environment and objects. The visual limitations of the interactive sequences is offset by the advanced interactive possibilities.

The best of both virtual worlds.

The uncanny valley is a hypothesized relation between an object's degree of resemblance to a human being and the emotional response to the object. The concept suggests that humanoid objects that imperfectly resemble actual human beings provoke uncanny or strangely familiar feelings of uneasiness and revulsion in observers. "Valley" denotes a dip in the human observer's affinity for the replica, a relation that otherwise increases with the replica's human likeness. https://en.wikipedia.org/wiki/Uncanny_valley

Photo by Andrei Slobtsov & Matt C on Unsplash