Hands on with Sony’S VR headset
Technology writers Ben Grubb and Tim Biggs give Sony’s latest virtual reality headset an “immersion test” which works in tandem with a PlayStation 4 to display and control games.
Virtual reality — once the stuff of science fiction — is still in its infancy. But there’s already a gold rush around the technology, which plunges viewers into a simulated 3D environment and lets them explore their surroundings as if they were really there.
Technology and entertainment giants are betting billions that virtual reality is much more than a passing fad, one that will revolutionise the way we experience movies, news, sporting events, video games and more.
Meanwhile, filmmakers and other creators are grappling with an entirely new storytelling language and dealing with some formidable challenges — claustrophobic headsets that can make people cybersick.
Technology and entertainment giants are betting billions that virtual reality is much more than a passing fad, one that will revolutionize the way we experience movies, news, sporting events, video games and more. Photo: Viktor Koen / The New York Times
The competition to dominate this space begins in earnest these next few months, with the arrival of newfangled, affordably priced headsets from Samsung, Sony, HTC and Facebook (which paid $US2 billion last year for a virtual reality start-up called Oculus VR). And Disney, Comcast, Time Warner and Legendary Entertainment are just a few of the entertainment companies plunking down millions of dollars in a mad dash to create content for these machines. By 2025, the market for virtual reality content will be $US5.4 billion, according to the Piper Jaffray investment bank [PDF]. The hardware component will be worth $US62 billion.
“We’re at the brick-size cellphone days of VR,” said Ted Schilowitz, the in-house futurist at 20th Century Fox. “The technology works. It’s remarkable. But it is nowhere near good enough, on any front, to take on mass, mass adoption.”
Yet, he added, “every few months, we’re reaching closer to the target.”
The Oculus Rift, owned by Facebook, must be powered by a robust PC and releases in its consumer form in 2016.
Without compelling content, even the most impressive piece of technology won’t appeal to more than a hardy band of early adopters. One of the more high-profile experiments at filling that void is taking place at Schilowitz’s studio, where director Robert Stromberg (Maleficent), Ridley Scott and the Fox Innovation Lab are putting the final touches on a virtual reality companion to The Martian, Scott’s hit film. In the 15- to 20-minute film, to be released early next year, viewers will become the stranded astronaut (played by Matt Damon in the feature film) as they navigate the planet and attempt tasks to stay alive. They will even get to experience zero gravity in space and drive the rover on Mars.
Here, some of the other pioneers in film, journalism, sports and gaming talk about the potential and struggles of building a new art form from the ground up.
Better Than Backstage
The HTC Vive comes with two base stations, which encourage users to move around a 15-foot by 15-foot area.
For director Mark Romanek (One Hour Photo, Never Let Me Go), virtual reality has long been a fixation. About 25 years ago, he tried on a VR rig at a convention sponsored by the early cyberculture magazine Mondo 2000 and was disappointed by how huge and uncomfortable it was.
So in 1991, when he was directing the music video for Love Conquers All for the British pop band ABC, he jury-rigged his own contraption.
He covered a scuba mask in black spandex, paired that with a fake interface glove, and then concocted a 15-second piece of computer graphic imagery.
Paul McCartney performing Live and Let Die, with Jaunt VR’s camera sharing the stage. Photo: MJ Kim / Jaunt VR
“I just sort of wished it into reality, even though the technology wasn’t there yet,” he said.
Last year reality caught up, when he was involved in an actual VR project with none other than Paul McCartney.
While he was talking to McCartney about collaborating on a short music film, the conversation turned to virtual reality. The former Beatle had never seen any footage, so Romanek asked the people at Jaunt VR, where Romanek is on the board of advisers, to supply a demonstration. McCartney was so enthralled, he urged them to film the concert he was performing the next day, at Candlestick Park in San Francisco.
Samsung’s Gear VR virtual reality headset works with high-end Samsung Galaxy phones.
Romanek, in Europe at the time, suggested camera angles to the Jaunt VR crew via FaceTime, and a performance of Live and Let Die was captured in 360-degree glory. Viewers start off right next to Sir Paul’s piano, where they can swivel to take in the rest of the stage or glance upward to find a sky filled with pyrotechnics. Moments later, they’re in the front row, but a glimpse behind reveals a crowd 70,000-strong.
Now, as he preps more ambitious mini-films, Romanek said he’s grappling with how to allow the viewer to feel that he or she is affecting the story in some way (often called “agency” in VR circles) while retaining some semblance of directorial control.
“When the viewer can look anywhere at any time, composition and montage goes out the window,” Romanek said. “Do you want Steven Spielberg or Alfred Hitchcock modulating your viewing experience, or your Uncle Morty?”
Google’s Cardboard is the cheapest and easiest VR viewer. A phone is insterted in to the carboard viewer and does all the work.
But a director’s willing and enthusiastic ceding of control may come to define the medium.
“I don’t think the question is: How do we make The Godfather or Jaws in VR?” Romanek said. “I think it’s something else.”
“Who knows?” he added. “In the end, we may discover that VR will turn out to be an essentially ineffective medium for narrative and be better suited to gaming, live events, news coverage or more purely ambient or fine art experiences.”
Sony’s PlayStation VR works in tandem with a PlayStation 4, and produces a flat version of the VR image on the television for all to see.
Inventing New Tricks of the Trade
As film students at Concordia University in Montreal last decade, Félix Lajeunesse and Paul Raphael studied the tools of the director’s trade. The zoom pan, the Dutch tilt, the extreme close-up.
Now, as virtual reality filmmakers whose Félix & Paul Studios has created works for Cirque du Soleil, Universal Pictures and LeBron James, they’ve had to forget, or retrofit, most of those techniques.
Cirque du Soleil made the virtual reality film Inside the Box of Kurios with Félix & Paul Studios. Photo: Félix & Paul Studios
People inside a virtual reality universe are able to look in any direction, a freedom that can be disorienting at first. So filmmakers need to find new tricks to guide their gaze, essentially sprinkling breadcrumbs of sounds, images and transitions throughout the films.
“It’s like playing musical notes that didn’t exist up until a few years ago,” Raphael said.
Sound is the nudge in Wild — The Experience, a three-minute VR film that cemented the duo’s status as groundbreakers in the fledgling field. Based on the 2014 movie Wild, about a grieving woman hiking the Pacific Crest National Scenic Trail, the film places viewers alone in the wilderness until Reese Witherspoon’s character appears. Then, a voice materialises off to the side, out of sight. Prompted to turn, viewers will discover Laura Dern, who plays the mother of Witherspoon’s character. If you keep your eye on her for the rest of the experience, she’ll stick around. But turn back to Witherspoon, and Dern will be gone the next time you look over.
The virtual reality filmmakers Félix Lajeunesse and Paul Raphael. Photo: Alexi Hobbs / The New York Times
It’s a subtle bit of interactivity.
“We didn’t want to see her walk in or dissolve in,” Raphael said, referring to Dern’s character. “We wanted to hide that cut in a way you can’t do in a film.”
The filmmakers warn against succumbing to the temptations of the 360-degree frame and staging constant action all around. Grounding viewers in their surroundings, engineering a sense of presence, is more important. Also, a frantic, jittery camera, particularly when it moves out of sync with a viewers’ head motion, can induce nausea.
Nonny de la Peña, pointing, during a virtual reality demonstration with journalists. Photo: Monica Almeida / The New York Times
“An experience that’s not great in a movie theatre is just a boring movie; an experience that’s not great in virtual reality can ruin the rest of your afternoon,” said filmmaker Chris Milk, a leader in the field.
Journalists and the ‘Empathy Machine’
Nonny de la Peña was a correspondent for Newsweek. She’s written for The Los Angeles Times. She’s made documentary films.
But nothing in her journalism career had quite satiated the need that drew her to the profession in the first place: telling stories that would inspire people to truly care about inequality and human rights abuses. Then she discovered virtual reality.
Her first project, Hunger in Los Angeles, explored food insecurity through the scene at a church’s overwhelmed food bank. She had originally hoped to capture what it would be like for a mother in line to discover that the food had run out. But when an intern recorded harrowing audio of a man collapsing into a diabetic coma while waiting in line, she opted to tell that story instead, in a novel way.
Using about $US700 of her own money, teaching herself computer coding and cadging favours from friends, she spent about two years recreating the scene in a seven-minute virtual reality experience. Hunger melded computer-generated animation of the people and environs with real audio, and allowed viewers to move around inside the story, even to kneel down to (futilely) help the collapsed man. In 2012, it made a splash at the Sundance Film Festival, where it was selected for the New Frontier showcase. De la Peña has focused on virtual reality ever since, with projects on the war in Syria and the plight of a migrant beaten and Tasered to death by the US Border Patrol.
The theory behind this sort of immersive journalism, as it’s become known in academic and journalistic circles, is that the visceral nature of the experience makes a viewer a new kind of witness.
“You really engage on scene in a way that gives you this incredible connection to where you are,” de la Peña said. “And that’s why, early on, I was calling it an empathy generator, an empathy machine.”
De la Peña may have been one of the first journalists to branch out into virtual reality, but she now has plenty of company. News organisations, including The Wall Street Journal, ABC, CNN, The Associated Press and Vice, have all done virtual reality projects.
De la Peña believes the possibilities, though, are freighted with journalistic peril. “As much as or more than anything, this medium allows for propaganda and mistruth,” she said.
Filmmaker and subject often have more of a symbiotic relationship than in traditional video journalism, as the technical logistics require more coordination. And viewers, feeling as if they’re on the scene they’re watching, give virtual reality a credibility they may not give other media.
“What does transparency look like when you have goggles on?” she said. “I don’t know the answer, but it is something I think about a lot.”
Courtside Seats and Immersive Games
Late last month the reigning NBA champion Golden State Warriors opened their season at a sold-out Oracle Arena in Oakland, California. Sitting courtside were the wealthy and connected who had shelled out thousands of dollars for the most coveted seats. And thanks to a funky looking double-lensed camera sharing that real estate, some hard-core, tech-savvy fans at home got to enjoy the action as if they were right there, too.
That was the first time a professional sporting event was broadcast live and nationwide in virtual reality. And it was no accident that the Warriors were playing.
The team’s co-owner, the longtime entertainment executive Peter Guber, is also a major investor in NextVR, the virtual reality company that filmed the home opener. He, like the owner of the Sacramento Kings, is betting that fans who crave live experiences and sponsors looking to tap that passion, will be willing to pay for the experience. What form those experiences will take — through broadcasts of complete games or packages of highlights, on a subscription basis or pay-per-view — will become clear in time.
“There are untold ways to mix the brew,” said Guber, a former chief at Sony Pictures and PolyGram before setting up his own company, Mandalay Entertainment. “I just drank the Kool-Aid.”
The lucrative future isn’t here yet, of course. People watching the Warriors game on their smartphones while wearing Gear VR headsets complained about jerky reception, no virtual scoreboard, and the sound and visuals sometimes failing to match up.
But perhaps the biggest hurdle to overcome is the uncomfortable headgear. The arriving batch of contraptions, while sure to be more comfortable, will not yet be conducive to wearing for entire sporting events — or for video games, another market that deep-pocketed companies are beginning to flood.
Guber and Tim Sweeney, the founder and chief executive of Epic Games, which in September released a virtual reality demo called Bullet Train, believe that it won’t take long for today’s oversized goggles to morph into glasses no bigger than a pair of Oakleys.
(For Sweeney, the biggest challenge for virtual reality game designers is creating incredibly photorealistic scenes. “When you’re looking at a computer screen, you have a high tolerance for cartoony artwork,” he said. “But when you’re in it, it just feels wrong.”)
Improving the actual viewing experience of virtual reality can only help exploit the possibilities in live events besides sports, according to Guber. Like awards shows.
In August, MTV live streamed the Video Music Awards red carpet show in virtual reality and followed that up in October with the entire European Music Awards. The American Music Awards will probably experiment with virtual reality soon. Not coincidentally, Guber owns part of the company producing that show, Dick Clark Productions, which itself is also an investor in NextVR.
When it comes to virtual reality, Guber — and his wallet — are all in.
“It may not have the same trajectory that we plan for,” Guber said. “It may not become omnipresent like we believe it will. But it’s here, and it’s going to play a prominent role in more than film and entertainment.”
Virtual Reality: The Headsets to Experience It
In the coming months, virtual reality will get its biggest mainstream push yet, as companies like Samsung, Sony and HTC release new headsets likely to be accompanied by huge marketing campaigns. Here’s a look at some of the major players’ offerings.
A collaboration between Oculus, the virtual reality company bought by Facebook, and Samsung. Powered by smartphones (though just high-end Samsung Galaxy models), the Gear VR headset offers a portable virtual reality experience, and is available now.
Powered by a PC, making it more robust than the Gear VR. Its positional technology gives users a wider range of physical interaction with the virtual environment, allowing them to crouch down and dodge bullets, for example. Available first quarter of 2016, pricing to be announced.
Developed in conjunction with Valve, the company behind digital game store Steam and the creators of video games like Portal. The headset plugs into a PC, and two base stations encourage users to move around a 15-foot by 15-foot area as their actions are replicated in the virtual environment. Available first quarter 2016, pricing to be announced.
Designed to work with the PlayStation 4. Unlike the other entries, it creates two sets of images: one for the headset and one for a TV, so virtual reality can be more of a communal experience. Available first half of 2016, pricing to be announced.
The simplest — and most affordable — way to experience virtual reality. A foldable cardboard mount with plastic lenses and a fastening device, into which a smartphone is slotted horizontally, it requires a compatible app. Available now for as low as a few dollars, or you can make your own.
The New York Times