Giving Hobbits a voice: A history of dialogue editing in Peter Jackson's Middle Earth
On The Lord of the Rings, the dialogue track was 90-95% ADR
Ray Beentjes started his career as a sound mixer and sound editor for science and natural history documentaries for Television New Zealand. He moved to Wellington to pursue a freelance career as a film and television location sound mixer and effects editor. Gradually, Ray moved entirely into post production, specialising in dialogue editing.
In this article, Beentjes looks at how modern tools are changing the landscape of dialogue mixing in modern film and television production.
I’ve been very fortunate to work with a brilliant team of friends and colleagues in a variety of roles, as a dialogue editor, supervising sound editor, and sound designer. In New Zealand, we have been lucky in that two of our main filmic forces in the past 15 years have been Peter Jackson and Fran Walsh. They have re-invested so much into the local New Zealand industry in projects and also massive capital injections to specific film infrastructure (Park Road Post Production and Weta Digital for example) that it’s hard to imagine what it would be like if they had not been around. Certainly the world-class film sound mixing theatres and facilities have been a major bonus for us in pushing some interesting new workflows in large-scale feature film soundtrack development.
Recently, I (and my colleagues) have been moving to decrease the amount of work that was more traditionally done in a dialogue pre-dubbing theatre. The changes in these workflows have become necessary due to time pressures or indeed different methods of building the dialogue tracks in films. There has been quite a recent movement to integrate as much “real” sound into film and television soundtracks as possible.
The lost dialogue of Middle Earth
Back in the days when we were working on the Lord of the Rings movies, the ability to clean up and almost “forensically restore” damaged location sound to allow its use in the soundtrack simply was not there. We had certain tools such as equalisation and various real-time broadband noise reduction devices, and while these were very valid techniques (and still the cornerstones of current dialogue treatment techniques), they did not hold up when we were faced with noises generated on set in studios that were using green-screen and CGI techniques for picture. Sound was simply not able to cope with the polluting sounds caused by fans and enormous banks of lights and dimmers. In Lord of the Rings, the dialogue track was comprised of roughly 90-95% re-recorded dialogue (or ADR) out of necessity.
Now ADR is both a blessing and a curse. For some actors, the ability to re-visit your work can lead to some wonderful things happening. They can solidify their performances in scenes that were rather fractured in the shooting process, re-visit accents that perhaps didn’t seem consistent, or simply provide the directors with a “new slant” on the performance angle that might not have thought of when the scenes were shot.
However for many actors, the ADR process presents something of a “brick-wall” to the creative process. Sync can be hard to achieve in difficult scenes, and often actors are left with a feeling that the vitality of their original performance is lost when they have to conform to ADR’s sometimes overly technical processes.
A new age of dialogue editing
Over the past decade dialogue editing has been changing dramatically. At last we’ve been delivered a strong quiver of clever tools that can actually save a lot of the dialogue that used to hit the floor. And directors are noticing this. After you have spent months or even years carefully forming a story in the picture editing room using the original sound from location or studio, it’s often a major disappointment to be told that some (or lots) of this sound cannot be used when it’s what you are in love with!
Buzzes, squealing lights, fans, smoke, and snow machines wreak havoc with the original sound. As I mentioned before, when we were doing the Rings films most of these challenges were simply insurmountable. ADR was the only tool we had to deliver the dialogue track that fulfilled all the technical requirements of motion picture production. Scenes in the Lord of the Rings films that had parts of green-screen sets, or were shot totally interior against them, needed air movement via huge fans to move the actor’s hair and costumes to simulate the wind you would expect in the outdoors. Often the interior dialogue scenes were almost impossible to hear at all over the rushing propellers and fans. And indeed anyone who has ever visited New Zealand will be able to vouch for the winds we experience on a daily basis that are so far from a Californian summer.
Now obviously, it’s impossible to fix something that is completely broken, and ADR will always be needed. Such situations as rain and storm scenes will always present major challenges to production sound recording. But also as dialogue editors, we are given the task of trying to simply save “something” so that the actor can actually hear themselves when looping their lines on the ADR stage. This calls for some serious forensic work and whilst the results won’t be of broadcast quality, they are a lot better than simply white noise and wind!
Audio restoration with new tools
I worked with most of the available tools that started to appear in the mid-2000s. The mission objective and hope in thedirectors’ minds was always that we could pull some magic out of the bag and save scenes that were simply not working with the recorded ADR.
The early efforts of the noise reduction software developers were amazing, but frequently, our efforts were rejected in the final mix process of the film’s soundtrack. Digital artifacts and strange changes to the original sound were simply not natural enough to sell to the audience. Whilst we might have spent weeks getting something to happen, in the cold, hard light of day and with the quality control and objective responses of the re-recording mixers (after all, this is one of their primary skills if they are good), our stuff hit the floor and in went the bad-sounding production sound, as it felt more natural and representative of the best way forward!
We simply did not have the tools to “get in there” and manipulate sound with the same detail the CGI artists were able to achieve with their images. But then a real revolution started to occur with the spectrographic display of sound in various high-end restoration packages, such as iZotope RX. We could see what we could hear, we could learn to read these displays, and once we were comfortable with understanding what the annoying sounds looked like, we could formulate ways of coping with those imperfections. We used techniques borrowed from visual re-touching tools to manipulate those spectrographic images, and we painted out or replaced tiny fragments of annoying sound with other parts borrowed from another part of a sound file or generated from other means.
The Hobbit, an unexpected journey
Recently, the tools have been getting hugely better. I think that with the development of RX 4 and then the subsequent honing and fine-tuning that has been happening with RX 5 Audio Editor, our arsenal of available tools to save original sound is amazing compared with where we were 10 years ago.
But things are changing rapidly. In the spirit of nothing staying the same, the expectations of directors as to being able to use snippets of production sound mixed with ADR and alternate lines sourced from unused sound takes together with the original source lines from the actual performance are keeping us on our toes!
We’ve witnessed huge changes in the way we deal with the dialogue tracks in recent films such as The Hobbit, where the process gets even more complicated. The entire film’s sound takes, whether they were being used or not, were synced up in each scene and in a process of what we called “selects,” we weeded through the pile of location sound and ADR and evaluated what was the strongest or most suitable performance of a line.
Fran Walsh has always had a huge part to play with the dialogue tracks in Peter Jackson’s films. Together with supervising dialogue editor, Jason Canovas, they spent months grinding their way through every scene in the movie selecting what became a huge patchwork quilt of “snippets of gold” that Fran felt worked best for the scenes.
That’s fine, until you try and mix that material in the traditional dialogue pre-dubbing process and find that, all of a sudden, it’s simply not working! Also the selecting process would continue right past the deadlines that pre-dubbing dialogue in the traditional way would require. This meant that the work of the dialogue editors was being flown into the final mix with no pre-dubbing.
We realised that we simply could not make the deadlines unless we changed the way we delivered the materials. Now this is a difficult situation, as you are effectively stepping on the toes of the judgment of your dialogue re-recording mixer. You’ve got to build a really good relationship with your mixer; you rely on them and they rely on you!
Mike Hedges was our dialogue mixer for The Hobbit. He’s always been appreciative of new ways of doing things, but with a healthy amount of quality control as the basis of moving forward. And to our relief, he was receptive to the concept that we would be doing a lot of work in our rooms using tools like iZotope RX, De-noising, De-clicking, Spectral Repair, and notching and filtering out complex background noises.
Quite a tall ask, really! Martin Kwok and I formed a team of back room mafia as we blitzed into the challenges we faced with trying to make all these disparate elements into a cohesive dialogue track. Mike Hedges quickly told us if we’d gone too wild with over-processing, and we worked out a unique approach that worked very well for The Hobbit.
The team at iZotope provided us with beta versions of their absolute latest software. I fully believe that if we’d not had those newest tools to use that we’d perhaps not have gotten to the finish line with the traditional methods that we had previously relied on.
For me, the arrival and integration of the standalone RX application with ProTools was the key I needed to save so much original sound. It’s a really deep application, deceptively simple on the outside, but with the power to get right into your audio. RX 5 Audio Editor has helped with this in its ability to speed up repetitive processes with Module Chains that used to take a lot of painstaking work in RX 3 and RX 4. I have now spent literally years deep inside RX and cannot imagine how I could work without it!
In some recent productions I have been involved with — such as Ash vs. Evil Dead — I have been pre-dubbing, cleaning, and leveling all my production dialogue in my cutting room (which is calibrated and properly acoustically treated) and supplying pre-dubbed stems to the final mix along with fitted, leveled, and tidied ADR cues.
I do believe that now more than ever the dialogue editors and mixers must embrace these newer techniques to enhance the end result. But of course it comes with the proviso of an excellent line of communication between all parties. I am anticipating, and participating in, the forward development of iZotope’s tools: They respond to user feedback in a way that is refreshing and I am looking forward to their future laboratory creations with relish!