Discriminatory AI Systems: “Revealing Yesterday’s Status Quo”

LGBT people are often discriminated against by AI. Both the technology itself and the conditions in which it was created are to blame.

Masculine stereotypes are often part of our imagination: the humanoid robot REEM B at the development center in Barcelona Photo: Yves Jelly/akg-images

Taz: How can artificial intelligence be alien, Ms. Moraes dos Santos Bros?

Sarah Moraes dos Santos Bros: The term arose out of frustration with the overly narrow concept of artificial intelligence: the idea that there is a certain truth and that artificial intelligence produces some kind of universally valid knowledge. But we must understand that issues of representation or gender are contradictory and changing. In the spirit of Queer AI, we want to look at how technology affects the body, that is, how it affects certain people differently.

Do you have an example?

One software company had an algorithm that rated applications from women lower because too many men worked at that company. Regarding queer identities, there was a case at Uber, where facial recognition software was used that was unable to properly identify trans people and thus excluded them from employment.

In the science anthology Queer AI, which I co-edited, there is a sentence: “AI is viewed as superintelligence and equated with a white male rational subject.” Where else can this be observed?

If you introduce AI into image search, the search results will be very consistent: Images show a white robot on a blue background. This creates a stereotype in which the masculine is re-associated with coldness and rationality. Or a brain is drawn with a circle. The AI ​​visuals have a very humanistic sci-fi aesthetic. Our idea of ​​artificial intelligence will become more realistic if, for example, we depict a humanoid robot that already exists. At the same time, there is artificial intelligence that is being marketed in a gendered way, for example in dating apps.

In what way?

These apps are coded differently and relate to ideas about how people of a particular gender should behave. Dating app Grindr, used primarily by gay men, operates primarily via GPS and location-based services — unlike apps that attract a more heterosexual audience. From this, certain ideas about sexual behavior can be deduced, such as what is most important regarding intense desire.

It is striking that artificial intelligence voices such as Alexa or Siri are female voices. What's the problem with that?

It is based on an idea related to the role of servants who provide a type of service. In 2020, there was an installation at the University of Merseburg with the imaginary language assistant “Meow Meow”. The audience can interact with it directly. However, it was uttered live by an actress – invisible to the audience. Miao Miao was not submissive, but rather rude and cheeky. However, some male participants tried to flirt with her and get close to her. Even when it comes to AI and stuff, we fall into gendered ideas. I have a friend whose bike is in the “she” form and an acquaintance whose laptop is in the “he” form. Of course, the problem is not gender, but the roles we associate with it. In the spirit of queer AI, it was also important for me to consider the contexts in which AI participates in reinterpreting these deterministic structures.

What did you come across?

There are great examples of this, especially in art. For example, artist Jake Elwes created a type of deep drag show. Many drag queens are specifically designed with their own artificial intelligence. Artificial intelligence has created a performance in which bodies transform into each other and identities constantly change. The AI ​​was not able to encode body movements or the most bizarre shapes: it then produced sequences in which bodies sometimes tilted into something monstrous or fantastical. In this regard, the term Queer AI is also a call to make AI more flexible and open.

But how will that work? Artificial intelligence is always based on clear data.

There are already many projects with queer chatbots, especially in the field of art. The training datasets there consist of queer poetry and feminist theory. Artist Sarah Seaston created a multi-faceted AI tool so that Google results no longer produced stereotypes, but instead offered hopeful and poetic suggestions for questions about queerness. Principles can be drawn from such projects, for example, that data from white men should be applied to white men only. Or this allows algorithms to produce inconsistencies. Many technology companies are funding more experimental forms of engagement with AI, for example through fellowships or artist residencies. Artistic experience can contribute to the development of real technologies, and this should not be underestimated.

What still needs to happen in terms of alien AI?

I advocate looking not only at the technologies themselves, but also at the conditions under which they are created. The clicking, or data processing, occurs because people are paid less than usual in Germany and work under precarious conditions. I wonder if those people sitting on the technology front could have a bigger say. The problem with queer AI is that AI usually shows yesterday's status quo because that depends on what data needed to be collected first. We need to question these cognitive constructions more closely, as this would be a big first step towards queer AI.