Blog

Why We Are Investigating the Biases of Artificial Intelligence

Andrew McWilliams Ellen Pearlman
Posted by the Residency Directors
Monday, 15 May 2017

ThoughtWorks Arts has an open call out now for artists and technologists investigating the implications of Artificial Intelligence for society today.

Image by Roger Marks
Image by Roger Marks

AI has incredible potential to automate all kinds of decision-making, and revolutionize industries. However, because of the way in which AI systems are trained, they can also automate and amplify human biases in ways their designers do not intend.

Numerous examples have recently emerged of automated bias in action, suggesting that AI has already permeated into the fabric of our daily lives, and that we are now subject to new forms of pervasive, systemic bias. As our previous resident Heather Dewey-Hagborg pointed out, when decisions are made by computers and not humans, they have a false air of objectivity surrounding them.

Many facets

If you sift through a selection of recent articles on AI, you’ll find a dizzying mix of topics. You’ll see self-driving cars, Amazon Alexa, industry-specific application analyses, startups and investment news, labor market and looming job loss, often accompanied by pictures of the Terminator or other science fiction movie characters.

You’ll also find articles predicting either dystopic futures, or the technical ‘singularity’. The singularity is the moment when AI-based systems, locked in runaway self-improvement cycles, cause an intelligence explosion in which computers far surpass the intelligence of human beings.

These are important topics, but the actual problems of codification of systemic bias is happening right now all around us. It is also not well understood. This is why our fall 2017 residency cycle will be aimed to tackle this subject.

Automated bias in action

In 2016, Julia Angwin et. al., published an analysis of the risk scores assigned to more than 7,000 people arrested in Broward County, Florida. In many US States, scores like this are used by judges during prison sentencing.

The study found that black defendants who did not re-offend over a two-year period were almost twice as likely to be misclassified as high risk than white people. Conversely, white defendants were mistakenly labeled low risk almost twice as often as black re-offenders.

This was not an effect that the designers of the system intended, but more likely a product of the way the system was trained. Further, it is not clear where accountability lies in AI systems, and it is not possible for the algorithms to provide a rationale for their decisions.

AI Now

In 2016, Meredith Whittaker and Kate Crawford founded a new independent research initiative called AI Now. This initiative is intended as a contribution to illuminate blind spots in AI research, and investigate the current implications on industry, society, and individuals.

Meredith contributed to the framing of our open call, and we intend to keep the residency connected to the AI Now initiative as it progresses.

In 2016, AI Now organized a White House symposium to spark cross-disciplinary dialog. All of the presentation recordings are archived online. Following the symposium, AI Now published this report which analyses and makes recommendations across four subject areas: social inequality, labor, healthcare and ethical responsibility.

The role of the artist

We have left the call open-ended, to allow artists to surprise us with their intentions to address these complex and difficult issues. As this is certainly an international issue, we have opened up this call to the broader global community so we can receive diverse perspectives.

We look forward to receiving applications and see a lot of potential for a project in this area of inquiry. Applications are due by the deadline of June 8.