Ellen Pearlman’s new “emotionally intelligent artificially intelligent” brainwave opera AIBO had its world premiere on February 22nd at the Estonian Academy of Music’s state of the art Black Box Theater in Tallinn.

The opera investigates Artificial Intelligence and its effects on both fascism and love. In the performance viewers take part in an interaction between Eva, played by actor Sniedze Strauta, and AIBO, a ‘sicko’ AI. Eva is wearing an Emotiv EEG brainwave headset connected to a bodysuit of light, designed by smart textiles designer Dila Demir.

The spoken word libretto between the AIBO and Eva explores the troubled history of their love affair. The audience watches Eva’s EEG brainwave emotional states manifest as colors on her bodysuit. For example, yellow lights indicate excitement, purple refers to interest, green for meditation and red for frustration.

Emotionally themed databanks of videos and music, also driven by Eva’s brainwaves, are projected onto hanging screens and played throughout the theater.

Eva’s spoken word libretto is converted to text, which is both projected onto a screen and uploaded to the Google Cloud. There, AIBO’s response is generated via a custom built GPT-2 AI, created by Pearlman and programmed by Thoughtworks developers Jonathan Heng, Wasawat Somno and Andrew Zhou, with help from technical director Hans Gunter Lock.

AIBO’s response is projected as text on the same screen as Eva’s text, so the audience can read along. It is also sonified into artificial speech so the audience hears it. AIBO’s response is analyzed for its emotional sentiment, and categorized as positive, negative or neutral. Positive sentiments display as a green light, negative as red, and neutral as yellow. This analysis is of a synthetic being’s synthetic emotions, as opposed to the very real and visceral emotions displayed live time on Eva’s body.

Further, Pearlman explains that “AIBO attempts to emulate Eva’s projected visual memories to make them it’s own. It wants to become human, not ‘sicko’, but can only create broken video simulations. This allows viewers a glimpse into fractured memories.”

Strauta, performing the character of Eva, continually interacts with audience members. This sets up a live-time feedback loop between Eva’s speech, Strauta’s emotions as analyzed by the brain-computer interface and represented as colors on the body suit, the audience, and the visual and sonic environment.

The opera is based on the relationship between Adolf Hitler and Eva Von Braun. Eva’s libretto is taken from events in Von Braun’s biography.

Pearlman uses the opera to explore two questions: “Can an AI be fascist? And can an AI have epigenetic, or inherited traumatic memory? These questions raise issues pertaining to a potential future where humans and machines may one day merge consciousness.”

How the project came together

In 2019, Pearlman received the European Union’s prestigious Vertigo Starts Residency Laureate Award to develop AIBO. The focus was to create a comprehensible and riveting short performance based on the integration of artificial intelligence and biometrics.

From the summer 2019 through the winter of 2020, Pearlman worked with the Human-Computer Interaction Lab at Tallinn University under the aegis of GoProSocial, led by Aleksandar Väljamäe. She also collaborated with the Estonian Academy of Art and the Estonian Academy of Music and Theater.

Technical director Hans Gunter Lock worked with Pearlman developing, among a number of tools, a MaxMSP speech-to-text feedback display as a mode of connection between the performer and the AIBO character in the Google Cloud.

Rehearsals and specialized training were conducted with Strauta, including training in contact improvisation. Pearlman then worked with Heng, based in Singapore, Sonmo, based in Thailand and Zhou based in Texas to develop and test the GPT-2 database that formed the backbone of the AIBO character.

The post-performance photograph shows some of the team along with collaborators and audience members. From left to right: Sniedze Strauta, Mauri Kaipainen, Ellen Pearlman, Gerhard Lock, Hans Gunter Lock, Julie Yaunches, Rūta Kruliauskaitė, Andrew McWilliams, Pia Tikka, Tavet Jansen and Dila Demir.

Work designed by Ellen Pearlman with the support of the VERTIGO project as part of the STARTS program of the European Commission, based on the technological elements from the GOPROSOCIAL.