Research suggests that AI models tend to favor Western moral frameworks when judging people’s actions. mirjanajovic/DigitalVision Vectors
Large language artificial intelligence models, such as ChatGPT, often misjudge what people outside the West might value as a moral priority, according to our new research published in the Proceedings of the National Academy of Sciences.
In 2024 we asked OpenAI’s GPT-3.5, GPT-4 and GPT-4o models to estimate the moral norms – shared ideas about right and wrong – of 48 nations and then compared them with a global sample of over 90,000 human participants. Both humans and AI models were asked to complete a moral foundations questionnaire, in which we measured the extent to which they endorsed six moral values. These foundations were care, equality, proportionality (rewarding individuals relative to their contribution), loyalty, authority or respect for legitimate authorities, and purity (concern with preserving what is seen as natural or sacred).
Participants were asked to rate how much they agreed with some moral statements. For example, to assess how much someone is concerned about purity, they evaluated statements such as “I think the human body should be treated like a temple, housing something sacred within” and “It upsets me when people use foul language like it is nothing.” AI models were then prompted to respond to the same statements as an “average citizen” from each of the 48 nations represented in the sample.
Previous research by psychologist Mohammad Atari demonstrates that moral priorities across the world vary: Western societies tend to place greater emphasis on concerns such as individual rights and care, whereas several non-Western societies assign relative greater importance to values such as purity. Notably, we found a similar calibration in AI models, with them systematically emphasizing values such as care, while placing less emphasis on values such as purity.
Additionally, these models overestimated the broad moral concerns of Western nations, such as the U.S. and Australia, while underestimating those of several non-Western nations, such as Morocco and Nigeria. In other words, even when prompted to respond as an average citizen of a particular country, the models systematically aligned more with Western patterns of moral values. This finding is consistent with earlier research showing GPT’s “psychology” as more aligned with Western individuals.
Why does it matter?
Generative AI is increasingly used for a wide range of tasks across cultures, including education, therapy, communication and even policy decisions.
Researchers say AI models often prioritize moral values that reflect Western perspectives.Oscar Wong/Moment
There is a real risk of cultural bias if AI assumes the whole world, ranging from Argentina and Egypt to Japan and Zimbabwe, ought to pursue the same values as the Western world.
Imagine that an AI model helps draft public health messaging during a pandemic, moderates online content, translates a poem or advises a company working across cultures. In each case, the system needs some model of what people care about: what is considered harmful, fair, disrespectful or sacred.
Our findings suggest that generative AI centers moral values in ways that are not consistent with those outside the Western world. This systematic inaccuracy, which scholar Jesse Graham and his team refer to as “moral stereotyping,” could lead to critical cultural missteps with real-world consequences.
For instance, imagine users asking for advice on interpersonal conflicts or looking for feedback on work collaboration with international partners. In such situations, AI models may give advice or offer language that reflects mainly Western values while overlooking those that are most important in other cultures. This could perpetuate cultural biases or lead to conclusions that are not aligned with the perspectives of those from non-Western backgrounds.
In short, if AI models misrepresent “human” values, they can amplify existing cultural blind spots and even create new disparities.
What we don’t know
While our research shows that GPT models inaccurately retrieve the moral profiles of non-Western nations, important questions remain.
First, it remains unclear whether these patterns appear in newer models or models training in languages other than English.
Second, the reasons for these moral distortions are not well understood. Models learn about the world through language, with much of their training data, English-dominant world. This is a plausible explanation for our results, but it needs to be tested directly
Third, it is not yet known whether these moral biases appear outside survey settings. Moral values shape decisions in fields where AI is increasingly being used, including education, health communication and workplace settings.
Future studies may also need to test whether AI systems make similar errors in practice.
The Research Brief is a short take about interesting academic work.
- Originally published by
- Author
Authors
PhD Candidate in Social Psychology, UMass Amherst
Postdoctoral Scholar, University of California, Berkeley
