This is the technology behind the Carolina Panthers’ giant AR cat

Last weekend was the first Sunday of the 2021 NFL season, and while it was full of thrilling highlights, perhaps one towered over them: the Carolina Panthers’ viral video of a giant virtual panther jumping around Bank of America Stadium.

The video is ridiculous. A large virtual cat leaps into the stadium, runs across the field, tears up a New York Jets flag (the opponents of the Panthers Sunday) and leaps away with a menacing roar. It has proved hugely popular, with 5.6 million views on Twitter.

Although the Panthers’ tweet calls the mascot a “mixed reality panther,” it actually seems closer to augmented reality applied on a really large scale, according to a conversation I had with Jon Slusser and CJ Davis of The famous group, the company behind the virtual panther.

Basically, what’s going on is that an animation of the virtual panther, which was made in Unreal Engine, is displayed in a live feed of the real world. That means cameramen have to monitor and follow the panther’s animations in real time as it moves around the stadium, as cameramen would do with a real live animal. To give the panther virtual objects to climb on and work with, the stadium is also modeled virtually, but invisible.

This technology isn’t baked into an app, meaning you don’t have to point your phone’s camera at the stadium to view the panther from a different angle if you’re attending a game. The animations are intended to be broadcast live. In Sunday’s case, the video was broadcast live on the big screens in the stadium and the video was posted on social media shortly after. (Fun fact: In the video, you can see that the stadium screens have black cat-eye screens. They were applied to avoid a mirror effect, a PR representative from The Famous Group said.)

Although only one animation of the panther was shown on Sunday, Slusser and Davis said other animations for the panther are being worked on. A possible example? Appearing down during a crucial third to “hover over the opposing team and growl at them,” Davis said.

The illusion in the viral video is clever, but not perfect. The cat definitely looks like a big CGI creature. And towards the end of the video, when the panther roars from the top of the Jumbotron, you can see that its legs don’t exactly match the architecture it’s standing on.

But the giant virtual panther is a fun take on a possible use of AR to add to the experience of watching a major sporting event. That’s already part of The Famous Group’s specialty – check out this video of a giant mixed reality raven made for the Baltimore Ravens, for example:

Or this video of how it used mixed reality to spotlight top NFL players ahead of Super Bowl 2020:

While we’ve been hearing about the promises of augmented reality and mixed reality for a long time – Apple CEO Tim Cook has been praising it for years – it has eluded mass mainstream adoption. And The Famous Group isn’t the only company making giant AR glasses; Riot Games has created its own dazzling mixed reality showcases for League of Legends events, such as bringing a giant dragon to the Opening Ceremony of the 2017 World Cup Final.

Personally, I hope the technology means that one day we’ll see AR Kaiju mascot fights.