User plays: How to tell stories in a new medium

Virtual reality was one of the biggest tech trends of 2016. But, as with any new medium, storytellers are still experimenting with the technology and attempting to throw off the shackles of previous mediums. So what are the new rules of engagement when it comes to designing experiences for VR? And what does the future hold for those who hope to turn passive observers into active participants in a narrative? Maya Breen immerses herself in the work of Conical’s Alejandro Davila, the creator of the Green Fairy

We all like to hear a good story. Or read it. Or watch it. But what about going a massive step further, where you’re actually in the story and experiencing it first-hand? Well now you can. Virtual reality storytelling is here and it’s making stories come alive as never before. 

Storytelling goes back a long way, before written records. Think gatherings around a fire where bards would ignite the imaginations of their listeners. Or rock art etched onto cave walls tens of thousands of years ago. Those stories were made to last. 

Move forward through time and you get the written word, novels, theatre, photography, film and games. There are so many different ways to tell a story now. And VR is at the forefront of new-age storytelling. Unlike other mediums, where you hear, read, watch or play a story, the author’s expression is inevitably in-between you and the story. But the full immersion of VR makes it possible to experience the tale first-hand; to play an active part in the story, rather than simply being a passive observer. 

Humans have a long history of embracing new mediums. But it takes a long time to understand their nuances. When television first burst onto the scene, it was common for creatives to use the same approach they’d used when making radio shows. Specific expertise has been developed over the decades, of course, that play to television's strengths, but the same thing is now happening with VR and content creators are now tackling the challenges of designing stories for a new medium.

It’s a kind of magic 

One pioneering New Zealand VR studio, Conical, has managed to capture the hearts of young Kiwis – and teach them a thing or two along the way – with The Green Fairy, New Zealand’s first series of story-driven VR experiences. 

For VR producer and Conical founder Alejandro Davila, it’s always been about storytelling and when VR emerged he saw it was the perfect tool to tell them. 

“We’re living in an era where we’ve gone from the information age and apparently now we’re going into the experiential age,” he says. “That’s all about experiences and people buy experiences instead of products.”

The pilot episode was exclusively showcased throughout last year, with the Green Fairy 2.0 experience set for an October 2017 release. 

With VR gaining traction, Davila spotted two opportunities in the market: 1) to use the tech to create immersive stories; 2) creating fantasy-based yet educational stories for kids.

“The gap in the market was VR and beyond and I noticed that in VR a lot of people were creating content that involved shooting or zombies or science fiction and then I thought maybe children. Even though it has not been trialled a lot yet it’ll be definitely something that’ll pick up in the future.

Alejandro Davila.

“2016 has been a year where it has brought awareness to a lot of people – not just the content creators but the public. [VR] has gone from something that only very few people were working on quietly in the background and suddenly it became important.”

But with new tools come new challenges in designing with them. It’s a complex frontier where you must change and adapt, tweak and innovate, to realise your vision.

Telling tales

The idea for the Green Fairy, where fairies create the glow in traffic lights, was dreamt up by Davila about seven years ago when he was studying a road safety book for his driver’s licence. “I wanted to tell this story and couldn’t find the right platform to tell it until VR came along.” 

The project is proving it’s possible to tell absorbing stories in VR and aims to have engaging, episodic content readily available as VR becomes more widespread. But creating a realistic and comfortable VR experience for whoever dons the goggles means looking at elements of design and storytelling in a whole new way. 

“All these questions arrived in terms of design,” Davila explains. “At the beginning when I was trying to find a design methodology for this I was struggling a lot because I was stumbling upon the same roadblocks that a lot of VR developers find at the beginning - which is they try and make it like a game.” 

A game character will often take a long time to get from one place to the next as the player ‘plays’ their way through an environment. But from a storytelling perspective, he says, this approach kills the momentum and people detach from the storyline. 

So the Conical team decided the user could ‘jump’ with the fairy characters to the next location and so maintain the momentum of the Green Fairy storyline. “Having the storyteller be the one who transitions us to a different place - it’s just huge,” he points out. “Especially because it’s something to do with magic – she can easily just use some fairy dust and transform this whole place, which magically becomes something else.”

New rules of engagement

Although VR is a new medium, the Green Fairy embraces traditional storytelling elements. It’s a fairytale, albeit set in a modern world, and it also has a storyteller (the Green Fairy) leading the way within the experience. 

“If you really want to engage people emotionally with a simple story just keep it linear and add certain moments where the user can look around and interact.” 

Sound design is also critical in VR given viewers are interacting with the environment and spatial audio is an effective way to influence where and when they look around. At one point in the Green Fairy, the user’s attention is drawn from the Green fairy towards her sister, the Red Fairy, as they hear her talking nearby (only via their left ear) and so look around in that direction to find out who it is. 

Visual elements such as the size of the character and how close they appear to the user can make them lean in or back away if they feel as though their space is being invaded. Colour and lighting can be used to direct attention and set the mood by brightening or darkening certain areas. Getting the realism of a character’s eyes right is very important too, says Davila, while their ability to track the user’s movements adds another layer of realism. 

“If you have realistic eyes on your animated characters or CGI characters - eyes really will engage people.” 

But when it comes to the user’s eyes, Davila likens their line of sight to an invisible ‘laser’ within the VR system, which will hit invisible hotspots and trigger a reaction. 

“The user has to look at certain areas of the virtual arena to activate some of the hotspots that we have hidden. So they have to, in a very seamless way, trigger these animations to advance the story.”

Having the story move forward by laser-beaming these hidden hotspots frees the user to experience the Green Fairy story without controllers. Davila says many VR experiences need controllers as a means to interact and advance the story or game, but for the Green Fairy, it could risk hampering the experience for its young target audience who may get overwhelmed. “Especially for kids, they may not be able to reach the end if they have to interact with the controller, so as long as it’s just a gaze-based experience it’s fine.” 

Motion and emotion

Duration is another important consideration when designing a VR experience, Davila says. As VR is fully immersive, without distraction, it is intensely absorbing, which can be fatiguing for the user after a while. It feels longer than it actually is, he says, so the Green Fairy adventure was kept under five minutes. This worked well for episodic content for five to ten year old kids who never like to sit still for too long.

And where would fairytales be if it they weren’t filled with great characters? They have to reach beyond the technology, which is ultimately a tool to tell the story, and connect with the user. 

“The IP and content are king,” explains Davila. “Mobile app games – people download those games just because of the characters. So it was very important for me to have the user engaged and create this relationship with this character, especially if they’re going to be in that strange world with her for about five minutes. They need to trust her and they need to feel that she’s going to look after them – especially if they’re kids.”

The first iteration of the Green Fairy was trialled in a number of Auckland and Christchurch Westfield shopping malls over two weeks last year, which further influenced the design in making it ‘kid friendly’. About 10,000 families had a go in the Green Fairy experience using Oculus VR goggles and as a result, the Conical team discovered that the duration of the adventure was on the mark and also found out that some kids (and adults) were spooked during the cut to black in scene transitions as they were afraid of the dark. 

It was also a great opportunity to see how the user responded to vertigo when peering over the edge of a high platform at one point in the VR journey. Davila says the vertigo element was intentional to maximise what people could experience in VR. 

“We didn’t want to have everything so safe and stationary that it just became boring. Mostly it’s adults who would feel the vertigo rather than the kids. Kids love it, they love to feel in a different place. It has to feel like a roller coaster that has a lot of variations. It cannot just be all movement, it has to have that contrast so I think in design, contrast is a huge thing.”

Where to from here? 

2017 is set to be a watershed year for VR, according to predictions from industry experts. As the gear becomes easier to handle and cheaper for users, consumer interest will soar, more capital will be pumped into the industry and heavier IP will emerge. 

“I think [2016] has brought awareness to VR and people will be using it in different disciplines in [2017],” says Davila. “Having different types of content already available for people to be able to experience as they use it as a tool for other things will help reach more audiences.” 

Davila thinks the evolution of VR will be similar to the evolution of mobile phones, which gradually morphed from being a single function calling device to the smartphones that we do almost everything on today. 

Chris Milk, the CEO of leading Los Angeles-based VR story production studio Within, has said virtual reality will be the last medium. And an article in The New Yorker posited that interactive storytelling would be the defining art form of the 21st century. Like the fondly remembered pick-a-path books, "they could deepen a viewer’s sense of responsibility for a story’s outcome" and while failed attempts have been made in the past (like Mr Payday, from 1995) and the execution is still often lacking now, "the requisite artistic sophistication would emerge". And, with experiences like The Green Fairy, we can see that starting to happen.  

To stay up-to-date with the Green Fairy’s journey, check out followthefairy.com.

Beyond entertainment

Applications for AR/VR abound, serving practical functions that could infiltrate a range of industries in the long run. Here are some of the most promising.

1. Healthcare

From surgical training in VR to meditative experiences helping those with depression, VR has myriad applications in the medical world. In New York, a mother was put into a guided VR meditative experience during childbirth to ease the pain after it was too late to have an epidural and a Mexican surgeon is experimenting with VR as an anaesthetic when sedatives aren't available. 

2. building (and buying)

VR makes it possible to virtually walk through a building under plan, at a human scale, which is great for figuring out where to put pipes, wires, and lightswitches etc. It's also great for buyers. You can walk through buildings that don't exist yet and if you’re wondering what that couch will look like in your living room, various AR apps allow you to see items in situ before you buy.

3. Space travel

NASA has looked at VR tech to control robots on Mars and provide astronauts with stress relief. 

4. Creating empathy

The first-hand experience in VR has the ability to make a user feel empathy towards a subject unlike any other media. Charities and progressive news organisations (and, yes, brands) have picked up on this as a powerful motivator for potential donors as they can gain a deeper understanding of the plight of others. One project drops the user into a Lebanon refugee camp so they can experience the conditions and struggles of the people there. 

Out of the trough

2016 was the year virtual reality really came bleeping onto the consumer’s radar. Three immersive VR headsets, HTC Vive, Oculus Rift, Sony Playstation VR, were unleashed onto the market and cheaper smartphone VR headsets like Samsung Gear VR and Google Cardboard offered a cheaper, more accessible version of the technology.   

2016 was the year virtual reality really came bleeping onto the consumer’s radar. Three immersive VR headsets, HTC Vive, Oculus Rift, Sony Playstation VR, were unleashed onto the market and cheaper smartphone VR headsets like Samsung Gear VR and Google Cardboard offered a cheaper, more accessible version of the technology. 

Augmented reality also had a breakthrough year in 2016, with Pokemon Go becoming an international phenomenon. While virtual reality and augmented reality are often described together as they are both immersive technologies, the two are actually quite different. AR quite literally blurs the boundaries between the real world and a virtual one as users can interact with virtual imagery imposed onto a real world view, whereas VR places the user in a completely virtual environment, wearing a headset that disconnects them visually from the real world. 

These technologies may have just caught your attention in the last year or two, but they have been developing for a long time. The first VR head mounted display appeared back in 1960 but the term virtual reality wasn’t coined until 1987. Headway continued to be made throughout the ‘90s but the technology struggled to match up to the vision and so it faltered. But rapid advancements in technology since the 2000s have brought us to where we are now and the ‘trough of disillusionment’.

This is a phase in the Hype Cycle, which shows where emerging technologies are heading, depicted by tech research firm Gartner, Inc.

According to Gartner, VR is ahead of AR on the evolutionary scale but both have 5-10 years to go until they reach mainstream adoption. The ‘trough of disillusionment’ is the stage after a new technology has passed the peak of inflated expectations but before it is accepted by the masses.

2017 is expected to be a year where it will take an important step towards the people, but a few issues need to be ironed out before they can go truly mainstream. At present, the headsets are bulky and require wires. Wireless, more comfortable headsets aren’t far away, and faster internet speeds are helping to eliminate lag for higher resolutions and frame rates, which is a key cause of drawbacks like nausea and headaches.

Distribution power is another key factor in virtual reality as a platform reaching mainstream status and becoming commercially sustainable. Not only does the hardware need to be affordable for the average consumer but the software and content needs to be easily accessible too. Also there isn’t a great deal of data on consumer preferences yet and the majority of VR content (apart from mobile VR) is exclusive to one of the high-end devices.  

Speaking of the business end, there are lingering questions over who’s going to pay for the content to be created. Will consumers pay for a VR movie the same way as they pay to go to the cinema? There are a number of business models VR content creators are exploring in this early stage market while funding from the big tech companies, traditional investment or crowd-funding campaigns keeps them afloat.

Because of its immersive nature, selling VR games is seen as a prime area of opportunity. Many brands are funding their own promotional VR experiences and creating event-based campaigns and sales from advertising placed inside VR content could also be a valuable way to generate revenue. As successful as they have been for mobile and web, in-app purchases could also be a big deal for VR developers as could subscription-based business models giving consumers access to a range of VR content.