Can Ian McKellen, Mixed Reality, and 52 Cameras Reinvent Theater?

You sit in a circle at the Shed, the cultural center in Manhattan’s futuristic Hudson Yards, waiting for the show to begin. Through your enhanced glasses, you see four empty chairs facing you, just out of reach. You watch strangers look out for the actors to arrive. As they do, one at a time, you feel unsettled – each locks eyes with you, specifically. “Don’t panic,” the esteemed British actor Ian McKellen assures you, as the actors take their seats. Except the actors are not there, really – McKellen, along with co-stars Golda Rosheuvel, Arinzé Kene and Rosie Sheehy, appears in An Ark, a new play at the Shed, in video form, a nearly opaque specter overlaid on the candy-apple red carpeting and crisp white walls of the theater and the outlines of your 180 or so fellow audience members. The experimental new play, written almost entirely in the second person by Simon Stephens (whose most recent show, the Andrew Scott-starring Vanya, wowed audiences at the Lucille Lortel theater last year), is one of the first so-called “mixed reality” shows staged in New York, blending physical experience with digital elements. Over 47 minutes, the actors address you, the viewer, directly. Their gaze remains trained on you. Don’t panic, they repeatedly assure. (Though due to some technical malfunctions at the preview I attended, there was some panicking.) And don’t confuse it with VR (virtual reality), the oft-maligned virtual headset technology of Mark Zuckerberg’s metaverse or Apple store demos. The distinction between mixed reality and VR is “very important to me”, said Todd Eckert, the show’s producer, in an interview at the Shed a few weeks before previews. The former combines elements of the physical and digital; the latter is full-on immersion into the digital – “elective isolation”, as Eckert puts it, akin to riding a packed subway while only looking at one’s phone screen. “I don’t see the point of exacerbating that,” he said. “The reason that this is an experience without a screen, that we’re using this technology that not a lot of other people are using, is because you see each other and you see the room. Your experience is one of being connected – that’s the whole point of the story.” Mixed reality is already well integrated in our tech-saturated lives – you experience it watching the computer generated first-down markers on any US football game broadcast, or via the translucent dashboards on new-model cars. But it has rarely been used in a theater context, in part because volumetric capture, the process of filming real subjects in three dimensions over time, struggles to record subtle detail. Eckert and the company he founded in 2016, Tin Drum, have been working to change that. In 2019, Tin Drum produced The Life, featuring a hologram of Eckert’s partner, the renowned performance artist Marina Abramović, in physical space. (Growing pains – the Guardian panned the experience as “a pointless perversion that hurts your eyes”.) In 2023, he produced a virtual, 3D concert featuring the composer/computer-pop pioneer/actor Ryuichi Sakamoto, who died before its release. (A “magical experience”, the Guardian raved.) An Ark attempts to refine the technology even further, presenting, for the first time, four actors at once – filmed by 52 cameras in one take at a studio in the French alpine town of Grenoble, but seated just before the viewer, as if for a private show. The concept for An Ark arose from conversations between Stephens and Eckert, longtime friends, wondering: “What can we do that isn’t possible in theater?” Eckert recalled. “The technology is fundamental to being able to take a thing and give it to the public in a certain way,” he added. “But I never want people to think of this as an expression of technology as opposed to an expression of humanity.” The first line of the show’s program bluntly states: “An Ark is not a work of AI.” Stephens began work on the show, which spans the arc of four separate lives from birth until death, in 2020, when questions of mortality, technology and the vitality of human connection seemed especially pressing. The sweeping, open-hearted themes attracted director Sarah Frankcom, a dramaturge with a murderers’ row of live-theater credits in the UK, though she was skeptical of the technical elements. “I don’t understand anything about technology. I’m not really that interested in it,” she admitted in a recent interview at the Shed, a week into previews. “My work has been very actor-centered, very writer-centered, very much about openness, connection, the way that live experience can momentarily hold you in a space and allow you to feel and see something differently.” But she found “freedom as a theater artist” in the limitations of the volumetric video system, run by company 4Dviews, which, like a standard performance, filmed one whole show at a time. Frankcom cast, directed and rehearsed as if it was a standard play. “It was a completely identical experience,” she said, “it’s just that we had our performance ready before we went to create the environment that it was going to be presented in.” Staging is minimal – the actors walk on and off “stage”, but primarily remain in their chairs. The notable distinction is the steady eye contact, maintained throughout the show, no matter where or when you look for it – a “breakthrough moment in terms of what the technology can do and making a new form of theater”, said Frankcom. “It allows you to be present with the actor on your own terms, and that’s very different to watching film, it’s very different to watching theater. You have a direct and pure relationship, and you do feel seen.” The effect is uncanny, sometimes intentionally (meeting Sheehy’s intense gaze, I felt a little too seen) and, perhaps, unintentionally (due to the projections’ hazy edges, it appeared as if their feet melted into the floor like Salvador Dalí’s clocks). Eckert conceded to some limitations, particularly regarding the final take’s resolution, in stark contrast to the pinprick hyper-reality of Apple’s Vision Pro headset. But the quality of the illusion, he maintained, was second to its connective potential. “VR takes away even any potential for human interaction,” he said. “Every time I’ve watched [An Ark], I’ve seen people that are in a completely different chunk of life than I am. And the fact that they were responding actually gave me hope for humanity. Which sounds overblown, but it’s true. That’s what I want out of this. I want something that really leans into the connection. And for me, resolution has very little to do with that.” “When it finishes, and people take their devices off, it feels to me like it’s a room full of people that have been on a journey and been through something together,” Frankcom added. “I’ve had people talk to me in a way that I’ve never had them talk to me at the end of the theater show.” Technical limitations aside, both expressed enthusiasm for the modality’s potential – for making intimate theater performances more accessible, at a time of soaring Broadway prices (I, for one, could never afford to sit front row at a McKellen performance), and for preserving the work of great actors, like McKellen, with more vitality and immediacy than standard filming. “There’s something very simple and very pure about what we’ve made that feels like it’s the first letter of an alphabet,” said Frankcom. “And to not know what the rest is, I think is probably good.” “Technology is forever being heralded as this thing that’s going to solve all of these problems,” said Eckert. “And I don’t know that it has. But forever, longer than any of us will be alive, people will sit down and feel that Ian, Golda, Arinzé and Rosie are staring them right in the eyes. That’s a result to me.” Whether the technology is ultimately alienating or inviting – whether mixed reality crosses through the uncanny valley or remains somewhere in between is, like any artistic work, up to the viewer. But the importance of live theater, at least to the mixed-reality makers, stays unquestioned. “We’re in a moment in time where being present feels really, really necessary as a balm to the kind of craziness that’s out of our control,” said Frankcom. “Theater, I think, has a really strong place to bring people together and go: ‘You’re here. We lived, we’re alive.’”