By Emma Cummings
We all have biases, whether it’s an innocuous preference, like always gifting chocolate because you like it, or a choice with far reaching consequences, like bringing in male applicants more often for leadership roles. The bare bones definition of bias is ‘prejudice for or against something’. Accepting bias as a fact of life means continually recognizing how our individual unique experiences shape our values and perception of the world. Acknowledging bias also opens us to seeing where we can change, where our experiences overlap with others and where we may be unknowingly ignorant of some key information. At SJSU’s first annual Health Techquity Conference, Dr. Joe Grzywacz presented on bias, pointing out, perhaps counterintuitively, that bias free data is like searching for a unicorn. He says, instead, “Anticipate that biases are everywhere and systematically mitigate them.”
When a team’s bias is cemented into an AI model, there is a lost opportunity for growth and nuance. Unlike in real life, AI does not understand critique or feedback in the same way as we do. When the foundational information that forms a model’s programming is missing information, it cannot simply live life, ask for input, or make an effort to gain new experiences. To butcher a quote from Confucius, AI cannot know that it doesn’t know about what it doesn’t know about.
But how do we even know that bias is present in AI? Other than the obvious argument that if people are biased, our technology will reflect that – a clear example being Text to Image (TTI) models where we can literally see how AI interprets a prompt. Examples of bias in TTI are the racist image recognition scandal with Google Photos (2015), including labeling a black couple as gorillas and more recently, Vision AI has been found to tag a tool as a gun more frequently when the photo includes a person of color (BBC and Algorithm Watch, Kayser-Bril).
In academic spaces there is often a general acknowledgement that there are many unanswered questions to research in the field of AI. An exemplary example of this is a quote from Berendt et. al. stating “What impact exactly pre-trained models have on the downstream tasks, and how such impact should be measured, are open research questions.” The research from this team goes on to describe that generative AI doesn’t do a thorough job of distinguishing between content versus context. This is a key to understanding why inaccurate or stereotypical results are given – there is simply a statistically high association that the model couldn’t parse for nuance (in this instance Text Synth, a precursor to GPT).
So should you care? Humans are not money making machines, and both personally and professionally setting aside bias and being curious about our differences is an opportunity for growth. There are studies showing that eliminating bias and increasing diversity is simply better for business, such as those pioneered or reviewed by Harvard Business School and Harvard Business Review. However, it’s important to note that money doesn’t need to be the sole driver in our conversation about why we should care. In an article by Robin Ely and David Thomas (Getting Serious About Diversity – Enough Already with the Business Case) they say “When the only legitimate conversation about diversity is one that links it to economic gains, we tend to discount the problem of inequality”. In other words, we’re missing what the point is to be diverse, when the focus is only on what is to be gained, and likely missing out on the benefits as well. Ely and Thomas’ article concludes with the poignant statement, “we are disturbed by the implication that there must be economic grounds to justify investing in people from underrepresented groups”.
As AI grows in use and popularity, so have testing and auditing methods for assessing bias. Some examples of research areas are present in Library Science, for example a large research body is already asking how credible AI answers (and therefore the systems) are, if the data is skewed. In financial literacy and financial planning, there is the question of whether or not racial bias is perpetuated in recommendations given to customers based on profiling. A tool called ‘Stable Bias Explorer’ has been developed to assess the images generated for different systems by professions and personal adjective tags. Lastly, make a search on any library database or in Google Scholar, and you will find dozens of studies testing, assessing, and analyzing if AI is biased and what the implications are.
All in all, these are just a few examples of where AI (and maybe us as well) has room for growth. Learning more about this tool will be relevant for information professionals. We’ve listed our sources below for you to follow up with. If you’re a more interactive learner, we’re giving a shoutout to our parent organization and pointing you towards some upcoming ASIS&T workshops on AI. There is an extensive program for the IDEA Institute on AI, which starts August 25th this year – open to both members and nonmembers for single and full day access. You can also follow SJSU ASIST on Substack, or keep an eye on the iSchool events calendar to find more events about AI this fall.
Did we miss anything? Let us know what you’re working on, whether that’s research, an upcoming project, or a resource we should follow up with.
Sources
Tool: Stable Bias Explorer, hosted by Hugging Face
Article: New Tool Allows Users to see Bias in AI Image Generators by Pasala Bandara, PetaPixel (2022)
Article: Google apologizes for Photos app’s racist blunder, BBC (2015)
Article: Google apologizes after its Vision AI produced racist results by Nicolas Kayser-Bril, Algorithm Watch (2020)
Article: Bias, diversity, and challenges to fairness in classification and automated text analysis by Berendt et al. (2023)
Book Chapter: Getting Serious about Diversity: Enough Already with the Business Case by Robin J. Ely and David A. Thomas (2024)
More articles/ sources…
[1 – statistics/ figures] Artificial intelligence algorithm bias in information retrieval systems and its implication for library and information science professionals: A scoping review
[3] The Routledge Handbook of Heritage and Gender, Chapter 25 Confronting Gender Biases in Heritage Catalogues: A Natural Language Processing Approach to Revisiting Descriptive Data
[2] Ethical artificial intelligence (AI): confronting bias and discrimination in the library and information industry.
[3] From Bias to Transparency : Ethical Imperatives in AI-Based Library Cataloging