Visual effects nominees, in the mix from the get-go, are eroding cinema’s limits

The five nominees for the 2023 Oscar for Visual Effects Achievement weren’t just sitting at their computers waiting for the post-production process to begin. Each was closely involved before, during and after production, with boots on the ground and capes in the air.

Paul (Felix Kammerer) faces off against a mysterious foe before Oscar-nominated VFX supervisor Frank Petzold works his magic in “All Quiet on the Western Front.”

(Frank Petzold/Netflix)

“No news from the Western Front”

To make the most organic version of “All Quiet on the Western Front” possible, visual effects supervisor Frank Petzold went into the trenches. He was on set to experience the sights, sounds, feelings and smells of those brutal battles.

“All the snow, the freezing, standing knee high in the water; we were there too. You are part of the filming process rather than being offered a record at the beginning of post-production. You throw ideas around. You put your heart and soul into every effect shot.

“I like to improvise. We did 3D scans of the entire battlefield and everything. But on set: ‘You do dialogues; let me grab some cameras, grab some stunt guys and… blow some stuff up with the (practical effects) guys.

What most viewers will probably get from the movie is the visceral madness of the fight scenes.

“I was shooting things with six different cameras: lots of soldiers running – different camera speeds, just collecting elements. On another project, you’d say, “Let’s simulate this on the computer.” I didn’t want that. We added fog, the planes we didn’t have. We had a tank that could go a few feet. At some point you have to pull out the hot glue gun and do things.

Sigourney Weaver’s performance is captured and then used to generate a face animation puppet from which the fully rendered character is created.

“Avatar: The Way of Water”

The sequel to James Cameron’s ‘Avatar’ isn’t so much a combination of concept, production design, cinematography, sound, costumes, acting and visual effects as a new creature with the DNA of each.

“We’re involved pretty early on because we’re helping with character design, we’re helping build the world,” said Joe Letteri, visual effects supervisor. Aside from fantastical environments and action, the film’s ultimate success is determined by whether viewers can forget they’re looking at ten-foot-tall, blue aliens and see the actors’ performances instead.

“This new facial system is key,” says Letteri. “It allows us to look at what the face (inside) is doing, so we better understand what the actors are doing. We built a neural network that can analyze performance. That was a real breakthrough. When you move one muscle on your face, the muscles on the other side of your face move; they are all connected (below).”

They scanned the actors through a series of facial expressions and speech exercises. Cameron sat with actors and went through their entire performances with eight cameras pointed at them. The resulting data — “tens of thousands of frames that we fed into the system,” Letteri says — powered the engine that translated human performers into Na’vi characters.

“There’s a shot of Kiri when she’s caught in the rain in the woods — it’s Sigourney Weaver playing a 14-year-old — it shows how this new system can express an actor’s performance through whatever character they want, even if it’s not how they look now. We live and die in the close-ups.”

An overhead view of Gotham City at night with Batman visible on a clock tower. The city is not real; it's a visual effect.

“The Batter”

Visual effects supervisor Dan Lemmon says Matt Reeves and company wanted to make “The Batman” “not so much a superhero movie as a gritty, noir detective story, grounded in reality.”

To make an unreal place – Gotham City – and improbable abilities – such as Batman sliding to safety after jumping from a skyscraper – viable, the team used LED volumes – soundstages with huge video screens. They shot the actors against virtual Gotham cityscapes that were projected onto the screens, rather than green screens. For example, instead of the regular film lighting, the actors were illuminated by the virtual sunset and the city atmosphere.

Lemmon says, “You can see the sunset reflecting off Batman’s cowl, Selina Kyle’s outfit. Sunsets have many different colors; you can see it in their costumes – oranges and blues reflect off everything, on the puddles on the floor.

An LED volume also helped Batman escape from police headquarters.

“Matt’s assignment in the order of the wings was, ‘I want people to think that Robert Pattinson actually jumped off a building and landed without a parachute.’

“We watched action sports videos, YouTube Red Bull stuff. He wanted to emulate that style of photography – they would have mounted cameras on their bodies. He felt it felt more like a real action video than a contrived piece of cinema,” says Lemmon.

“We built a wind tunnel of LED panels. We hung both the professional wingsuit performer and Rob Pattinson on safety ropes to hang them in the wind tunnel and blew a lot of air through the suit. That gave it more of a sense of realism.”

The finished shot adds water to the scene.

“Black Panther: Wakanda Forever”

Visual effects supervisor Geoffrey Baumann says the most challenging part of “Black Panther: Wakanda Forever” was the underwater world of Talokan. However, “Most frightening part was to make Namor (the ruler of Talokan, played by Tenoch Huerta Mejía) into a character people would like. Namor had been reinvented so many times, but overall he was just kind of an asshole…”

There was also the fact that, like in the comics, Namor’s flying power depended on little wings attached to his ankles.

“How do you not make that crazy?” asks Baumann.

The production departments of “Wakanda” have conducted many underwater tests. They found that a character’s grass skirt played floating peekaboo in the water. They learned that hair and clothing often misbehaved, so some performers wore skull caps and their beautiful locks were added afterwards, while some wore partial costumes, Baumann’s team added the top layers digitally. Finally, they made most of the underwater footage dry for wet (so basically out of the water), often reshooting scenes they had already captured.

“The challenge was to be able to overlap seamlessly,” he says. The effort was aided by what the camera crew and actors learned from shooting wet initially.

For Namor’s moves, they studied athletes including triple jumpers and all-time great running back Barry Sanders. But when it came to those winged ankles…

“The first step was, we animated,” Baumann says with a laugh, “this mini-CG helicopters on a CG character’s ankle to see how they would move.

Somehow it all worked out – Namor’s flight became majestic, and Huerta Mejía’s take on the character became downright… loveable.

“Top Gun: Maverick”

“Top Gun: Maverick” flies so high in large part because it seamlessly creates the illusion that it’s really the characters performing those insane feats in mid-air.

“We got in where things were just too dangerous to get to practically, because so much was practical, so much was real,” says Ryan Tudhope, visual effects supervisor.

Tudhope’s team added or removed jets as needed, changed the skins on some planes to transform them into others, and created environments, such as expanding the enemy base.

Then there’s the climactic battle, featuring a first-movie-era F-14 jet against new “fifth-generation” fighters. Much has been said about the camera rigs that the filmmakers developed with the Navy to fit into the cockpit of the current F/A-18, the aircraft most frequently featured in the film.

The cockpit of the F-14 is smaller. Those cameras wouldn’t fit.

To keep the film’s visual language and keep using those cameras, “it didn’t make sense to do it any other way than to have (the actors) sit in an F/A-18 … and then digitally to look like (an F-14) Kater.

“There are times when you look over the top of Maverick at the cockpit. That was an F/A-18, but we completely replaced it so you have the Tomcat cockpit. Even the little tracking screen runs at eight frames per second or something.

“That puts visual effects in a supporting role, and that’s where we wanted them in this movie.”