The Thoughtworks Arts AI Residency has been awarded to Karen Palmer, a London-based digital filmmaker and storyteller, to further develop her interactive film, RIOT.

RIOT, an emotionally responsive, live-action film with 3D sound uses AI technology, machine learning, and facial recognition to navigate through a dangerous riot in progress.

The film installation experience reveals to the viewer, in real time, how they might react in the face of imminent danger. It provides feedback into the complex cultural issues surrounding split second decisions in times of actual crisis, by placing viewers at the center of a provocative story for which there are a number of potential live time scenarios, each triggered by their authentic reaction.

A webcam or camera is used to monitor each viewer’s facial characteristics as they stand in front of it, and the video narrative responds accordingly. For example, if the viewer becomes agitated, the character in the film will respond defensively or impatiently. The same is true for a number of other emotional responses, like calmness or fear.

One of the unique aspects of the film is that RIOT’s software is built from the ground up. It was designed by RIOT’s technical partner, the Computer Science and Engineering Department at Brunel University London, led by Dr Professor Hongying Meng, a renowned figure in AI and facial recognition technology.

Other partners include the V&A Museum in London, The Perception Institute NY, and the National Theatre of London Immersive Storytelling Studio.

Palmer’s speaking engagements on RIOT include TEDx Australia at the Sydney Opera House, the Google Cultural Institute Paris, Games for Change Festival NY, Mutek Festival Canada and CPH:DOX.

The RIOT prototype was recently honoured as part of the Digital Dozen Break Throughs in Storytelling 2016, and has been exhibited at the Museum of Modern Art Peru, Future of Storytelling Festival NY and Festival of the Mind Sheffield. The project went on to be featured by global broadcasters including CBS, Guardian, Fast Company, NBC and the New York Times.