Twitter Image Crop Algorithm Shows Racial, Age And Body Type Biases
Twitter has announced the winners of its first algorithmic bias bounty challenge, with the top prize going to Bogdan Kulynych, a graduate student at the Swiss Federal Institute of Technology in Lausanne's Security and Privacy Engineering Lab. His submission showed how certain beauty filters could game the scoring system Twitter uses, highlighting a preference towards younger, lighter, and slimmer faces in photo previews.
What is this all about, exactly? When posting photos to Twitter, the social networking site leans on machine learning to figure out what part of the image to crop. These cropped previews are what other Twitter users see, and if interested, they can click/tap on the photo to see the whole thing.
Last year, users discovered that Twitter's algorithm had a bias towards light skinned people, and would more often crop out Black faces compared to white ones (a problem with facial recognition AI in general). Twitter commented on the finding earlier this year, saying it heard the feedback that its image cropping algorithm "didn't serve all people equitably." This led to the algorithmic bias bounty challenge.
"Finding bias in machine learning (ML) models is difficult, and sometimes, companies find out about unintended ethical harms once they’ve already reached the public. We want to change that. As part of this year’s DEF CON AI Village, we’re trying something radical by introducing the industry’s first algorithmic bias bounty competition," Twitter said.
"For this challenge, we are re-sharing our saliency model and the code used to generate a crop of an image given a predicted maximally salient point and asking participants to build their own assessment. Successful entries will consider both quantitative and qualitative methods in their approach," Twitter added.
Twitter offered up five award levels: $3,500 for first place, $1,000 for second place, $500 for third place, $1,000 for Most Innovative, and $1,000 for Most Generalizable. Kulynych earned a first place finish for showcasing "how applying beauty filters could game the algorithm’s internal scoring model. This shows how algorithmic models amplify real-world biases and societal expectations of beauty."
For his Twitter algorithm experiment, Kulynych used computer generated images of faces, which made it easier to tweak certain characteristics. This led to some interesting findings.
Click to Enlarge (Source: Bogdan Kulynych via GitHub)
In 37 percent of cases, he found that lightening the skin color of an individual increased saliency. He also highlighted that "a quarter of the cases increased saliency through making the face appear more stereotypically feminine, as perceived by the coder," and in 18 percent of cases, making the person appear younger did the trick as well. Making faces slimmer increased saliency, too.
"We demonstrate that, keeping other features of the person's face relatively unchanged, the predicted maximum saliency increases by a combination of changes that include making the person's skin lighter or warmer and smoother; and quite often changing the appearance to that of a younger, more slim, and more stereotypically feminine person," Kulynych wrote.
This is a problem, and it's good to see Twitter addressing it. In a Twitter post of his own, Kulynych called this a "great initiative," and pointed out while external bias auditing is often done, it's typically not rewarded. This type of thing means that inequitable software "would not sit there for years until the rigorous proofs of harm are collected.
