Waves of technology-driven change now regularly disrupt industry, culture and society. The challenge of navigating this turbulence grows increasingly complex, requiring deep examination of overlapping trajectories in search of hidden insights.

Research of this kind necessitates synthesis from varied disciplines, viewpoints and areas of expertise. However as the speed of change accelerates, new forms of collaboration are required — forms capable of producing original, far-reaching perspectives from the cutting edge of cultural and technological transformation.

One of the most uniquely impactful approaches is through art and technology partnerships. Art provides a powerful lens for engaging with technology because of its high degree of freedom in making large conceptual leaps across domains, delivering surprising outcomes.

Technologists engage with artists in the creation of new systems, algorithms and prototypes, opening new horizons. This symbiotic relationship becomes a breeding ground for unique projects which investigate, reimagine and reshape issues across business, technology and culture.

How can arts practice function as technology research? What kinds of insights does art derive? How can art help us build a picture of the risks and opportunities of emerging technology landscapes?

This article will explore these questions and more. It is the first in a series examining the symbiosis of art and technology, and will demonstrate why art and technology make such a powerful combination. The next article in the series builds on this, explaining how Thoughtworks has designed programs to incubate artists dealing with issues like cyborgism and transhumanism, bias in machine intelligence and movement in robotics.

This Has Been Done Before

Technological innovation by artists is nothing new. Artists frequently engage with engineering, medicine, science and countless other disciplines, playing leading roles developing the progress foundational to society today. It’s useful to take a moment to examine past examples, in order to see the broader trends at work and better understand what is happening around us now.

At the height of the European Renaissance period in the 15th Century, artists like Antonio Pollaiuolo, Leonardo Da Vinci and Michelangelo developed a ground-breaking mastery of the human form.

These artists performed rigorous analytical study, working with physicians and even dissecting corpses themselves. Their investigations surpassed much of the knowledge of anatomy taught in universities at the time, and merged anatomical with architectural concepts — such as plan, section, elevation, and perspective — producing a new vocabulary for scientific illustration.

The value of these contributions is hard to overstate. Achievements like these leap across domains, producing new foundations which are then built on anew for the betterment of all society.

Detecting the Signals Today

At Thoughtworks, we seek out and collaborate with artists engaging with technology and investigating issues of crucial importance today. Our employees support artists’ projects as partners, and this hands-on engagement enables a sharper focus on the issues raised.

In 2016, Thoughtworks collaborated with the Cyborg Foundation to develop new prototypical human ‘senses.’ Artists Neil Harbisson and Moon Ribas believe human psychology can be ‘reshaped’ via newly designed sensory organs — permanently integrated electronic sensors in the body.

This means that web data feeds can become first-tier physical senses, seamlessly integrated into our experience just like sight, touch and smell.

In this view, each of us can create new and customizable forms of human psychological and sensory experiences, different from our natural ones. The entire human condition as we know it is merely a starting point for technological customization.

Cyborg Foundation artists have experimented with these ideas, creating and integrating new prosthetics, and working with Thoughtworks on the development of new proof-of-concept wearables.

Neil Harbisson created a ‘time sense’, which can be felt as heat sensations around the circumference of his head. Moon Ribas created a ‘seismic sense’, allowing her to feel earthquakes as vibrations on her legs.

After their time with Thoughtworks, the Cyborg Foundation founded the Trans-Species Society to connect with others interested in designing new sensory organs and new forms of human experience. Given the current rate of technological and medical progress, and projecting forwards the standard human social pressures to improve and get ahead, this potential future may not be so far-fetched.

For Thoughtworks and our clients, projects like this inform our discussions and help us imagine new directions. Audiences and project collaborators are often exposed to new ideas, generating discussion and debate. Hands-on experience provides a nuanced view, highlighting both risks and opportunities for the future.

From Signals to Risks and Opportunities

In 2017, filmmaker Karen Palmer joined Thoughtworks to explore emotion detection through Artificial Intelligence.

Her project RIOT is a film in which viewers’ emotional expressions drive the story forward in different directions. Audiences experience a riot in progress, and depending on their perceived level of calm, anger or fear, find themselves either being arrested, assaulted or set free.

Thoughtworks developers worked with Karen to create and open source a new facial expression recognition system named EmoPy. This system estimates human emotion by assessing images of human facial expressions, via machine learning.

While developing the project, we explored the features, implications and emerging technological possibilities in AI. We drew out risks and opportunities inherent to automated decision-making systems, in the present as well as in the near future.

For example, RIOT reveals a risk that the technology will be relied upon for intelligence it does not possess. Expression recognition systems cannot infer human intent, nor genuine emotional responses. Instead, they infer expression perception by other humans. The opportunity here is to educate and advocate for appropriate aims in applications of the technology, so that stakeholders are better empowered to act on the intelligence generated.

Further, inherent human bias embedded in the datasets behind neural networks are often reproduced and amplified in operation. A risk exists that an overfocus by implementers on ‘accuracy’ scoring might mask the human subjectivity throughout both training and testing of such systems.

Unmitigated, this risk could encourage false confidence in inaccurate decisions.

By including affected communities in the development process, a more granular understanding of the system’s capabilities and characteristics can be encouraged. This can not only alter the decisions made by the neural net itself, but can also improve decision-making by humans downstream, because of their improved awareness of what the system’s outputs really mean.

Synthesis, Sharing and Dissemination

Ideas like these and more are exchanged, debated and written up by collaborators on art projects at Thoughtworks. For example, while working on RIOT, developer Stephanie Weber explored issues relating AI to society in her essay on how AI is transforming the criminal justice system.

Additionally, team lead Angelica Perez compared the technology behind various facial expression recognition techniques. We also wrote up research on AI and bias in preparation for our public call for AI artists, created in collaboration with the MacArthur Foundation funded AI Now Institute at New York University.

Each art project we work on is multi-layered and complex, revealing a web of potential risks and opportunities. The dissemination of these ideas enriches the dialog between our employees, partners, friends and clients.

Art & Technology Programs

To put our exploration of these complex issues into practice, we created three programs under the banner of Thoughtworks Arts. This includes the Thoughtworks Arts Residency, Art-A-Hack and the Hardware Hack Lab.

The next article in this series explores the structure of these programs, how they each attract different communities, and how they reinforce each other. It explains how the artists we collaborate with produce valuable critiques and new vantage points in art, social practice and technology. It then illustrates further examples of how these projects engage our employees and produce valuable insights that we bring to our clients.