The New Film, Video Experience Begins To Take Hold
By Andy Marken
“We’re nothing but a simulation on some computer.”
Douglas Hall, “The Thirteenth Floor,” Columbia Pictures, 1999
Industry analysts, and the media folks who interpret what they say, swear Pokémon Go is proving that AR (augmented reality) is real. Jeeez folks, it isn’t really AR, more like an anemic subset. And like most subsets, it will come and go and I will almost have totally missed it.
Actually, that’s not true. Because it turns out, I did the same thing Nick Bicanic, director and consultant on VR (virtual reality)/AR projects, did when it came out:
– Installed it.
– Caught my Pokémon.
– Uninstalled it.
It was almost as much fun as watching a lady take a foodie of her Bubba Burger as I did awhile back in Kauai. Mildly good AR, mildly good burger. Oh well, Pokémon has gotten folks to climb/get stuck in trees, drive into police cars and walk off hillsides; so I guess it’s good clean exercise.
The Difference
Lots of folks like to use AR and VR interchangeably, but about the only thing they share is reality. VR takes you into a filmed or computer generated world, while AR adds something into your world. There are a lot of techie differences, but with VR, filmmakers really use very wide and/or fisheye lenses, and there’s no real focal length. With AR, you’re simply dropping images into a scene and they have to be in focus.
VR puts you in the center of an artificial environment and totally obscures the real world. For the time being, VR requires head mounted displays (HMDs) such as Oculus, HTC Vive, Samsung Gear, Google cardboard, etc. Work is being done to change that but for the time being, you’re stuck looking like a Minion or pervert.
Hands Off – The one issue I have with HMDs is that they tend to make VR film viewing a personal activity and when you’re sitting there quietly enjoying a trip to a faraway place, there is always someone wondering if you aren’t really visiting some exotic place.
Adult VR searches are rising 9,000 percent, according to Google Trends (yes, they track you there too) and Cory Price, Vice President of one of the most visited adult sites on the web. Not that we can speak from experience, but Price says it can give you a mesmeric experience unlike anything you’ve ever seen before and he expects the industry to be worth $1B by 2025. Hey, whatever floats your boat. After all, there are 1,000s of meaningful/profitable marketing, business, entertainment and education applications for VR emerging.
[Tweet “”George Lucas has been quoted as saying that VR would ruin the storytelling part of filmmaking.””]
VR represents a media inflection point the likes of which the world has never seen and it has the potential to be bigger than TV, the Internet or smartphones. Tens of billions of dollars have been invested in hardware or platform tools, but almost nothing has been spent on the most critical element … content!
Somehow, we got the cart before the horse with people rushing product to market before folks have figured out how to use it to tell stories for the masses. True, we’re still in the very nascent age of VR filmmaking; but fortunately Fox, Sony and every major studio are investing millions into short and long form VR content. However, greater attention has to be paid to creating the narrative in multiple ways before the hardware will reach its fullest potential.
Crafted content like Fox’s Martian VR Experience, Sony’s Ghostbusters: Dimension, a spin-off of Jurassic World and the hundreds of VR films in various stages of production are creating the excitement. Google and Facebook like to point to these great works and encourage kids to jump in and get a leading position, even if they haven’t mastered 4K and HDR (high dynamic range) production.
Unfortunately, there always seems to be a chasm between the world according to marketing sound bites and what’s being done in the trenches. You know, how folks doing the creative work … testing new gear, new techniques and busting gear/workflow by pushing them beyond their limits. People like Fox’s Futurist, Ted Schilowitz, have access to teams of experts (and money) to solve these problems. Indie filmmakers work through the hard way by sharing their experiences, expertise, successes and failures.
In checking the facts, I asked filmmakers to share stuff with me:
– Marcelo Lewin, founder of Digmedia Pros
– Lewis Smithingham, CTO of 30Ninjas
– Nick Bicanic, CEO, RVLVR
– George Krieger, TME at Sphericam, SoundVisionsMedia
George Lucas has been quoted as saying that VR would ruin the storytelling part of filmmaking because the viewer can go anywhere he/she wants, which makes it tough to “guide and manage” the plot. I agreed with the point until Smithingham showed me a 30-second video with a stupid cat chasing a laser light around the room.
Oh Lookie – When you see Smithingham shooting a video of a cat chasing a laser pointer, you think another stupid cat trick; but think like a VR filmmaker and you can envision how the content creator can actually guide the viewer through the film rather than whipping his/her head this way and that (getting motion sickness or whiplash).
Filmmakers do something very similar today by placing visual breadcrumbs for the viewer to follow through various scenes, a technique Schilowitz probably put to great use in The Maze Runner movie. You’re free to move around and make your own movie experience, but that damn creative person will still suck you back. But as Lewin and others have hammered into me over the past few months, visual is only half of the film.
[Tweet “”The mere rotation of the head while watching is a form of user input.””]
Woody Woodhall of Allied Post Audio, said he’s been busy with “conventional” projects but the audio techniques he has perfected over the years will work beautifully in VR films. He believes psychoacoustics will play an important part in the new stuff you experience. And Google agrees with their release of Omnitone, spatial audio for the web. While filmmakers are just mastering the visual part of a film experience, Lewin speculates how we hear and the impact of music and sound will really enrich the VR experience in the years ahead. And if you’ve ever “felt” a beam bend from your right side, through your head and land on the right side, you know it’s freakin’ awesome!
First Things First
Sure, a good film starts with a good storyline but let’s skip that and go to the hardware we’re using today. At best, it’s just adequate. Smithingham and Bicanic have broken more of the early GoPro rigs than they’d care to count; but they did a decent job early on: 16 rugged 4K cameras spread around filling loads of SD (secure digital) cards with great 4K content again … and again … and again.
Underwater Adventure – We admit it, diving into waters where you know there are going to be sharks isn’t for everyone. Smithingham’s shark adventure gives people an up close and personal experience of almost interacting with sharks without getting wet.
Smithingham was one of the first filmmakers to blend his vocation with his avocation and produce a shark dive so landlubbers could experience underwater action first hand. Krieger took to the air flying mid-sized drones to give people a new perspective of events and locations, including the first HD live drone stream at Burning Man 2014.
Cool, but still cumbersome.
Billions are being invested today to develop cameras that can deliver 360 degree immersive experiences: single-lens, two-lens and multi-lens approaches and they’re all sorta’ kinda’ good in their own way. For the filmmaker though, one thing is missing. The viewing screen. Which means they also bring along a notebook computer so they can capture and review.
[Tweet “”We’re not simply documenting events and telling stories anymore.””]
New and established camera folks have been rushing out cheap and expensive camera gear for the market and Fox’s Schilowitz has been working with Samsung to deliver what he thinks will be the definitive camera gear for VR. Bicanic and Smithingham have been more pragmatic in their efforts, working with firms like Jaunt, Nokia and others to change the cameras so they work in the real filmmakers’ world to capture live action video and create 360-degree, stereo 3-D VR experiences. Bicanic divides his time between designing/building VR applications and shooting.
Game After – Until recently, Cuba was “off limits” to Americans, which has made it an even more enticing holiday location. Bicanic was one of the first filmmakers to visit the country and was one of the first to give you a feeling for the sights, sounds and people as they go about their lives. Even something as unexciting as a game of backgammon becomes interesting in VR.
A guy who has also covered armed conflicts, Nick Bicanic was one of the first to give us a chance to visit Cuba in the comfort of our own HMD. He shot everything for you from the Cubans’ love of baseball to a thoughtful, relaxing game of backgammon. Nick feels UX (user experience) is something creative types often discount in storytelling. With VR, even though there are no game buttons to push for interaction, the mere rotation of the head while watching is a form of user input that the filmmaker needs to take into consideration when crafting the story.
[Tweet “”including thinking how to incorporate the viewer into the story at the beginning of the creative process.””]
Instead of opting for a larger, more tethered stereoscopic system like the Nokia Ozo or the Jaunt, for the Cuba shoot he used an Izugar Z4X. He ripped off the stock GoPro Hero Black lenses and replaced them with modified fisheye lenses. Not necessarily a “gentle guy,” he pushes his camera systems: movement, proximity to action, positions for coverage, and then aggressively edits his content during post. The small size of the camera systems allows for movement to happen as close as 6 inches to the camera, which, in a fisheye lens, affords an increased level of intimacy with performers and spaces. As they say, your experience may vary as to which you like best.
Krieger stumbled on what he feels is a better solution with Sphericam’s 4K, 60fps compact camera.
New Toy – Krieger couldn’t wait to try out his new “toy” and had his wife drive him around Las Vegas slowly so he was able to shoot a VR short film of the city by day. Even for those who have to visit the city for mind/feet numbing tradeshows, it can (almost) be a fun experience when captured in VR.
Most of Krieger’s income involving drone film work was from teaching others how to capture and produce aerial content. Along the way, he also garnered a number of honors and prizes for his aerial creative work. For a guy who 11 years ago was given a nine percent chance of surviving a head-on with his van and a 150,000 lb. crane, capturing all that excitement might seem a little weird; but what the heck.
His aerial passion therapy of unmanned systems and cameras became even more passionate when he started messing with 360VR and spherical imagery. “We’re not simply documenting events and telling stories anymore,” Krieger commented. “We’re creating experiences people can enjoy; and while the industry is in its infancy, we’re closer than a lot of folks seem to realize. It won’t be long and people won’t be constrained by a square box.”
What filmmakers will be constrained by though is that they’ll have to think totally differently about their content. How they handle it. How they treat it. How they store it. How they share it, and give the viewer more control over the creative experience; including thinking how to incorporate the viewer into the story at the beginning of the creative process.
We’re way past the point where Detective McBain said, “When you get back to wherever it is that you come from, just leave us all the hell alone down here.” All four of my chosen experts, and hundreds (thousands?) of other filmmakers, are firmly committed to changing the status quo. Bicanic uses the example of a sunset off one of “his” Southern California beaches where someone is painting it, another photographing it, and he’s there shooting it in VR. It’s another way to capture the moment, the emotion and either keeping it to yourself or sharing.
As a friend likes to say, “It’s all good.”
Andy Marken is the owner of Marken Communications.
