AI is the next step in the evolution of digital entertainment
Technology and the visual arts have always been inextricably linked. Artificial intelligence is the next great, evolutionary step.
Imagine Star Wars without the Death Star, Jurassic Park without the terrifying T-Rex, or Terminator 2 without the liquid metal T-1000. None of those incredible stories would have been possible without incredible technological advancements. Now artificial intelligence is poised to take us on new adventures inspired by the human imagination.
At 96 Next, our mission is to use technology to push the boundaries of storytelling at scale. It’s the driving force behind our award-winning interactive sci-fi series on Twitch, Artificial. Now in its fourth season, the series started as “just” an interactive, serialized, sci-fi show that is performed live and. To the unprecedented deployment of interactive tech that makes the show the most innovative scripted series in the world. The series, an interactive, serialized, sci-fi show performed live was immediately innovative from its inception, but now in its fourth season has continually deployed new interactive technology, year after year, transforming the show into an unprecedented audience experience with characters and advancements that would not have been possible at the release of its pilot. Two seasons ago, Artificial won the Primetime Emmy for Innovation. The newest season goes beyond that initial innovation level, pressing against the cutting edge of technology and storytelling, inviting audience members to participate in the story like never before.
Some of these innovations include a faction system, wherein each audience member “identifies” with a faction–going a step further than the Hogwarts House system or the families in Game of Thrones, the faction system allows viewers to insert themselves into the narrative by picking a team and working towards that team’s victory. The series has its own virtual economy: a currency system in which viewers accrue “channel points” that they can then spend to support their faction or hurt the other. Instead of a predetermined soundtrack, the show is set to the music of LifeScore – a piece of technology that uses AI to score each episode based on the live chat on Twitch. Fans’ reactions to the action dictate how the musical score evolves from scene to scene, moment to moment. In addition to the advancements in audience interactivity, the show features Virtual “AI” characters built by Obskur. Human actors voice these virtual AI beings and direct their body language and actions using motion capture technology. Despite these new features, the show is still performed live and serialized week-to-week.
AI has already played a starring role in hundreds of productions. If you’ve witnessed a massive crowd scene in a movie or TV show made over the last decade, there’s a good chance that 90 percent or more of the people in that scene are AI-generated. Production companies no longer need to hire 1,000 extras; they can hire 100 and use a computer to generate 900 more. Algorithms then determine how these virtual people move and interact.
Another common use of AI is generating virtual spaces. Disney’s The Mandolorian uses the Unreal Engine from Epic Games to generate 3D backdrops and project them onto LED screens as the show is shot. Actors and crew can occupy the same physical location, regardless of whether a scene takes place in a cantina on Tatooine, the ice planet of Hoth, or inside Mando’s ship.
Creators can use AI to generate photo-realistic digital assets, allowing artists to spend their time concentrating on their art, instead of transposing hundreds of images. Likewise, showrunners can produce episodes much more quickly and inexpensively, which in turn allows them to concentrate on delivering high-quality narratives and impactful stories.
AI is rapidly approaching the ability to create realistic versions of people. We can already synthesize human voices in a credible way, taking samples of people talking and using them as building blocks to generate speech that sounds like it came from an actual human. One of the really interesting questions to me is whether it’s possible to create a completely AI-driven entertainment experience. Right now, human actors convincingly play the artificial beings in our show; I can foresee a future where literal AI play those AI characters.
I wanted our viewers to have an experience that feels as futuristic as some of the themes we explore in our show. Creating Cleo with Soul Machines’ technology has given us the opportunity to push interactive innovation — something I’ve tried to do throughout my work. That’s a big reason why I’m looking forward to the live conversation with Cleo, and seeing how our audience reacts to her. Can Cleo be as believable and empathetic as Tiffany Chu, who plays the AI characters Sophie and Lilith? Empathy is connected to the definition of understanding, and if the AI understands the audience, then we have made that connection. We have succeeded in that. But will she feel real? Will the audience connect with her? I’m really interested in finding out.
At 96 Next, we see AI not as a replacement for human creativity, but a storytelling enhancement that encourages new human creativity and innovation. While AIs can tell a basic story, it is up to us as human creators to enrich simple tales with meaning and depth. AI can allow for more and better content that’s easier for creators to make. Movies never replaced books; they just allowed audiences to experience stories in new ways. Like everyone from the Lumieres to George Lucas, we are using technology to enhance the experience for everyone. Ultimately, it’s up to the audience to determine whether we’ve succeeded.
By Bernie Su, Founder, 96 Next