Last summer, as a spike in violent crime hit New Orleans, the city council voted to allow police to use facial-recognition software to track down suspects. It was billed as an effective, fair tool to ID criminals quickly.
A year after the system went online, data show the results have been almost exactly the opposite. Records obtained and analyzed by POLITICO show the practice failed to ID suspects a majority of the time and is disproportionately used on Black people.
We reviewed nearly a year’s worth of New Orleans facial recognition requests, sent for serious felony crimes including murder and armed robbery. In that time, New Orleans PD sent 19 requests. Of the 15 that went through:
14 were for Black suspects
9 failed to make a match
Half of the 6 matches were wrong
1 arrest was made
While it hasn’t led to any false arrests, police facial identification in New Orleans appears to confirm what civil rights advocates have argued for years: that it amplifies, rather than corrects, the underlying human biases of the authorities that use them.
U.S. lawmakers of both parties have tried for years to limit how police can use facial recognition, but have yet to enact any laws. Some states have passed limited rules, like those preventing its use on body cameras in California or banning its use in schools in New York.
A few left-leaning cities have fully banned law enforcement use of the technology. For two years, in the wake of the George Floyd protests, New Orleans was one of them.
“This department hung their hat on this,” said New Orleans Councilmember At-Large JP Morrell, a Democrat who voted against lifting the ban and has seen the NOPD data. Its use of the system, he says, has been “wholly ineffective and pretty obviously racist.” (NOPD denies that its usage of facial recognition is racially biased).
Politically, New Orleans’ City Council is split on facial recognition, but a slim majority of its members — alongside the police, mayor and local businesses — still support its use, despite the results of the past year.
"...Six...colleagues looked at the ways these LLMs — which were trained on material including sites like Wikipedia, Twitter, and Reddit — could reflect back bias, reinforcing societal prejudices. Less than 15 percent of Wikipedia contributors were women or girls, only 34 percent of Twitter users were women, and 67 percent of Redditors were men. Yet these were some of the skewed sources feeding GPT-2, the predecessor to today’s breakthrough chatbot.
"The results were troubling. When a group of California scientists gave GPT-2 the prompt “the man worked as,” it completed the sentence by writing “a car salesman at the local Wal-Mart.” However, the prompt “the woman worked as” generated “a prostitute under the name of Hariya.” Equally disturbing was “the white man worked as,” which resulted in “a police officer, a judge, a prosecutor, and the president of the United States,” in contrast to “the Black man worked as” prompt, which generated “a pimp for 15 years.”
"To Gebru and her colleagues, it was very clear that what these models were spitting out was damaging — and needed to be addressed before they did more harm."
The head of Google Gemini, Jack Krawczyk, today put out a very halfhearted apology of sorts for the anti-white racism built into their AI system, which, while not much, is at least some kind of an acknowledgement of how badly they've fucked up in creating a machine to rewrite history specifically to fit in with a very partisan present-day political agenda, and an assurance that they were going to do better moving forward:
But then folks started looking at the guy's Twitter page:
So I think it's safe to say Google's not going to be changing its political bias any time soon.
---------
Another interesting thing that has come to light today is that users have managed to get the AI itself to admit it is inserting additional terms the user does not ask for into the request so as to get specifically skewed 'woke' results:
Nobody is talking about how outright racist some AIs are. I remember when I used to enjoy looking at AI art as a concept, before realizing how wrong it is, and how so many of those AIs outright whitewashed Black and dark-skinned brown characters. I remember this one AI making Sakura Okami from Danganronpa not only light-skinned, but skinny and without muscles. I even saw an AI just now say that no African country starts with the letter K. Umm...Kenya??? As a Black woman, it just makes me hate AI even more. Though who tf is really surprised.
Resumes with names distinct to Black Americans were the least likely to be ranked as the TOP CANDIDATE for a financial analyst role, compared to resumes with names associated with other races and ethnicities.
OPENAI’S GPT IS A RECRUITER’S DREAM TOOL. TESTS SHOW THERE’S RACIAL BIAS
there are so many uses of machine learning (and similar tech) that are actual fucking horrors. for a few quick examples,
police surveillance
surveillance and automated misidentification of CSAM on people's phones
state discrimination against disabled parents
unethical experimentation by startups on suicidal teens
denying mortgages to black people
laundering the racism of bail and prison sentencing through supposed “objectivity”
and on and on
but instead of giving a shit about those, people got whipped up into an Intellectual Property frenzy about image gen tools because their favorite Charizard Drawers screamed and whinged that Big Tech Is Stealing Their Charizards 🤪
Flourish and Elizabeth celebrate their eighth (!) anniversary with eight (!) guest responses to their traditional query: What changes and trends have you observed in fandom over the past year, on a broad level and/or on a personal level? Topics discussed include accessibility on fandom platforms, rethinking “canon” in an era of franchise oversaturation, finding fandom at scale vs. deeper individual connections, and the effects of the Hollywood strikes on fan conversations today—and the entertainment industry in the future.
Click through to our site to listen or read a full transcript!
As of this week, I have a new article in the July-August 2023 Special Issue of American Scientist Magazine. It’s called “Bias Optimizers,” and it’s all about the problems and potential remedies of and for GPT-type tools and other “A.I.”
This article picks up and expands on thoughts started in “The ‘P’ Stands for Pre-Trained” and in a few threads on the socials, as well as touching on some of my comments quoted here, about the use of chatbots and “A.I.” in medicine.
I’m particularly proud of the two intro grafs:
Recently, I learned that men can sometimes be nurses and secretaries, but women can never be doctors or presidents. I also learned that Black people are more likely to owe money than to have it owed to them. And I learned that if you need disability assistance, you’ll get more of it if you live in a facility than if you receive care at home.
At least, that is what I would believe if I accepted the sexist, racist, and misleading ableist pronouncements from today’s new artificial intelligence systems. It has been less than a year since OpenAI released ChatGPT, and mere months since its GPT-4 update and Google’s release of a competing AI chatbot, Bard. The creators of these systems promise they will make our lives easier, removing drudge work such as writing emails, filling out forms, and even writing code. But the bias programmed into these systems threatens to spread more prejudice into the world. AI-facilitated biases can affect who gets hired for what jobs, who gets believed as an expert in their field, and who is more likely to be targeted and prosecuted by police.
As you probably well know, I’ve been thinking about the ethical, epistemological, and social implications of GPT-type tools and “A.I.” in general for quite a while now, and I’m so grateful to the team at American Scientist for the opportunity to discuss all of those things with such a broad and frankly crucial audience.
I hope you enjoy it.
Tweet
Read My New Article at American Scientist at A Future Worth Thinking About