Ang Lee invents the "realies" with Billy Lynn's Halftime Walk
IBC2016’s Big Screen keynote session featured a sneak peek at Ang Lee’s new film Billy Lynn’s Long Halftime Walk. It was a tour de force of HDR, 3D and Dolby Atmos at a 120fps frame rate. Is it the first step in an entirely new kind of filmmaking?
Ang Lee was the star of last year's IBC Big Screen keynote lineup, and the revered director brought with him a remarkable technology demo. The director elevated stereoscopic With his Life of Pi, the director elevated 3D to an art form. Now he has taken high frame rates (plus 3D, plus high dynamic range, plus Dolby Atmos sound) to a new level with his new feature Billy Lynn’s Long Halftime Walk.
IBC Big Screen head Julian Pinn interviewed Lee in front of a packed IBC auditorium, which was treated to an 11 minute clip from Billy Lynn’s Long Halftime Walk, intercutting the character’s combat experience during the occupation of Iraq and his platoon’s participation in a frenzied Super Bowl halftime show.
This was the first public showing of the footage outside the US, and using the highest-end set up imaginable: 3D HDR projection by twin 4K Christie Mirage laser projectors at 28 foot-lamberts each, at 120fps. And topping the NAB showing, the IBC Auditorium was specially rigged with a Dolby Atmos sound system.
The screening of the 11 minute Billy Lynn clip left the delegate stunned with a vague sense that they were looking into at least one of the possible futures of filmmaking. Peter Jackson’s Hobbit experiments in high frame rates had left some sceptical about its effectiveness in narrative storytelling, with the greater image information in high frame footage revealing too much detail, underlining the artifice of moviemaking and distancing viewers rather than drawing them in. But the Billy Lynn screening showed that in the hands of an artist high frame rates - and at 120fps twice the frame rate of The Hobbit - can create an entirely new palette for moving picture storytelling.
IBC is by and large a technology conference. The ideal would be to analyse Ang Lee’s Billy Lynn HFR footage (in high dynamic range stereoscopic 3D) with hard-headed pragmatism, coldly evaluating its pros and cons as a possible future format.
But the fact was that Ang Lee’s direction of the technology was so masterful, that assessing its technical merits was virtually impossible. It was like listening to Yo Yo Ma play the theremin. You were stuck wondering: Is this new technology truly the future of music, or am I just being dazzled by a genius who can make anything sound good?
The HFR experience
For the first minute or two of watching the 120fps footage, it was hard to shake that “video look”, the sense that you were watching something cheap and ordinary. Whether that effect is the result of a lifetime of indoctrination into the “magic” of 24fps, or our brain’s assigning “commonplace-ness” in direct proportion to the rising frame rate, is still an open question.
But eventually, the “video” effect gave way to the sense of looking through the screen at another reality. It wasn’t quite a feeling of looking through a window – although with a lesser talent at the helm, that might have been a natural conclusion, and high frame rates in sports coverage will capitalise on this “window” analogy – but of seeing another real, flesh and blood world carefully manipulated by the storyteller.
Capturing images at 120fps, has some interesting implications for content creation. The higher frame rate – once the heights of 120fps are reached - essentially eliminates all motion blur, so onscreen action and camera movements are perfectly smooth and the images have perfect clarity.
The traditional cinematic frame rate of 24fps requires the brain to do a lot of interpolating of the missing motion. Taking into account motion blur as well, a significant portion of a 24 or 30fps film is literally left to our imaginations. But 120 fps, which offers many times more visual information, produces an unsettling sensation of being pulled into a real event.
To get even a simple shot to look good takes a lot of work
Ang Lee’s film school
Ang Lee was as impressed as the IBC delegates when he first began to experiment with high frame rates for Billy Lynn. On Life Of Pi, he had become frustrated with how strobing and motion blur detracted from the performances, and so he began to look toward higher frame rates.
“Different combinations of frame rates and resolutions each have their own chemistry,” Lee observed. “The original plan had been to shoot Billy Lynn at 2K 60fps. After you pass 100 frames per second, the strobing goes away. But once I saw the (120fps) image, I realised we were going to have to rethink how to make the film.”
This new higher frame rate opened a Pandora’s box of challenges which had to be tackled by the production team. Beyond the technical issues, the hyper-realism of 120fps revealed every artificiality of the filmmaking process.
“The aesthetic is different when you see all that detail. You just see everything. It was really intimidating,” Lee confessed. “Things we have done for 100 years in movies, we couldn’t do. And that’s scary. To get even a simple shot to look good takes a lot of work.”
Ang Lee’s forte has always been directing actors, but even this skill was challenged by the new high frame rate.
“The scary part is you can see the acting. So I had to change my direction to the actors. Usually an actor will have an exterior or interior goal, but for this film we had to be subtler and multidirectional. The performances had to be internalised, and as a director I had to put thoughts in their minds – sometimes conflicting thoughts. Every take I found I had to give them five different directions, just to try to keep the performances fresh and alive.”
This kind of filmmaking doesn’t (yet) lend itself to improvisation or run & gun shooting. The crew did six to eight set ups a day, and camera movement was kept to a minimum. Lee laughs: “Dollying in 3D is like trying to move a refrigerator.”
Lee repeatedly said how humbled he was by the technology. Rather than seeing the new tools as an amplification of his own auteurship, he saw in this new palette a vast new set of challenges to overcome and skills to perfect.
He believes that changes in frame rate have subtle, but potentially profound effects on audience perceptions.
“Each frame rate and resolution seems to produce a different mind set. At 120 frames per second, you’re dealing with a sharper audience, and your eyes get more greedy. As we move into higher frame rates, we’ll have a lot to catch up on – we’ll even need changes in the way actors hold themselves and in the writing itself. Moving forward, I would like to rely less on the story arc and more on intuitive experience. I think that’s more reflective and more satisfying.”
A follow-on “technical deep dive” panel discussion added Lee’s long-time editor Tim Squyres, Sony Pictures Entertainment head of production technology Scot Barbour, and the film’s technology supervisor, Ben Gervais.
Gervais said the production went to a lot of commodity hardware in its management of the massive workflow. The team were aware of how experimental the project was: “Lots of times, we’d make an advance and then say, ‘What do we do now?’. I look to other industries to borrow ideas. We used a lot of VFX processes for our dailies.”
Gervais noted that one of the benefites of working in a 120fps source medium was it was easily reduceable to lower frame rates, with 120fps a multiple of 24, 30 and 60fps.
The production employed Real D’s TrueImage virtual shutter software, which allows the creation of virtual shutter of any shape, size and duration and the easy elimination – or augmentation – of motion blur and strobing artifacts. The use of the software has been key in mastering formats at different frame rates.
Ang Lee said that most cinemas around the world will be able to show Billy Lee in 2K 120fps, but its likely that most viewers will see end up seeing the film fraction of its ideal frame rate.
Dollying in 3D is like trying to move a refrigerator
So the inevitable question is will high frame rates be accepted – let alone embraced – by audiences. As James Cameron almost singlehandedly dragged an industry into the 3D world, Ang Lee’s Billy Lynn could be a pivotal point for high frame rates.
A cynic could argue that we’ve already had a long trial with higher frame rates. After all, we've been presented with the option of 24fps at the cinema or 30fps at home for years, and after decades of comparison, the almost universal consensus is that 24fps means quality, means art, means that magical transportation to another world. 30fps, not as much.
It is not impossible that humans need some kind of artifice, some kind of distancing effect, a “once upon a time”, for us to fully embrace the magic a narrative has to offer. Maybe we got lucky with a 24fps cinema industry. Maybe the higher frame rates that Thomas Edison had originally recommended would have made the cinema an intriquing novelty, but not the land of dreams that dominated the 20th century. Maybe it was low frame rates which created the magic of the cinema.
But the truth is, until we see these technologies widely used – and used in a variety of circumstances by a variety of talents, we can’t fully know what they have to offer – and what nuances of storytelling they represent. High frame rates could, we might discover, have nothing to do with movies at all. They may be a doorway to a new artform altogether – the “realies”.
Until the newest formats – and the technologies to produce and exhibit them – get into the hands of the young geniuses in their parents’ basements, we won’t know their full potential.
Ang Lee’s Billy Lynn could open the door to a new format of hyper-realist filmmaking that the industry would do well to give a good trial before we cast it aside – 3D style - for not delivering hoped for economic returns.
To quote Lee himself: “It’s humbling. We have to step up our game.”