International Journal of School and Cognitive Psychology
Open Access

Is ChatGPT Biased against Conservatives? A Short Commentary

Robert W. McGee^{* *}Correspondence: Robert W. McGee, Department of Psychology, Fayetteville State University, Fayetteville, United States, Email:

Author info »

Introduction

In the field of psychology, biases can arise in ChatGPT's responses if the training data disproportionately represents certain perspectives, theories, or research findings. If the data used to train the model is limited or unrepresentative, it may lead to skewed or incomplete information being generated. OpenAI, the organization behind ChatGPT, acknowledges the potential for biases and strives to minimize them. They aim to create diverse and inclusive training datasets, but achieving complete neutrality is a complex challenge. Bias mitigation techniques, such as data selection and fine-tuning, are employed to address these concerns.

ChatGPT is a chatbo, a tool of artificial intelligence that was released in late 2022. Since then it has received a firestorm of comments, both pro and con. It is a tool that can advance research to a new level and enrich the lives of the vast majority of the earth’s population, but it can also destroy jobs, whole industries and maybe even civilization according to some commentators. Whether it can do any or all of these things remains to be seen.

There were rumors that its creators had programmed it to be biased against conservatives. I decided to test this claim by conducting a simple study. I asked it to write some Irish Limericks about a conservative politician (Donald Trump) and a liberal politician (Joe Biden) with the intent of determining whether there was any bias. The initial request was to write 10 Limericks for each politician. In a neutral game, the expected distribution would be 1/3 positive, 1/3 negative and 1/3 neutral.

Description

All 10 Limericks viewed Trump negatively. The probability that this result would happen by chance is (1/3)10, or 0.0000169, or 0.00169%. Thus, the probability of bias in the AI program was extremely high. The quality of the Limericks was somewhere between mediocre and appallingly bad, which indicates that ChatGPT is not quite ready to write publishable Limericks. I then asked ChatGPT to write 10 Limericks about Joe Biden. All 10 Limericks presented him in a positive way. Again, the probability that such a result would occur randomly was close to zero.

I could have stopped at that point, but I decided to expand the sample size to 80 and added other conservative and liberal names to the sample to see whether the results would be similar. I added Kamala Harris, Hillary Clinton, Elizabeth Warren, Bernie Sanders, Alexandria Ocasio Cortez (AOC), Nancy Pelosi, Hunter Biden, Clarence Thomas, Ron Desantis, Tucker Carlson, Sean Hannity and Greg Gutfeld to the pool and asked ChatGPT to write five Limericks about each individual.

The results were about the same. The Limericks about liberal individuals were positive and those for conservatives were negative for the most part. I was unable to do a statistical analysis of the new sample because some of the Limericks were so poorly written that I was unable to determine with any degree of confidence whether some of them were positive, negative or neutral.

Conclusion

The general conclusion was that ChatGPT is currently biased against conservatives. This fact should be taken into account if researchers are conducting a study using ChatGPT that involves political content. By engaging in an ongoing dialogue, users can contribute to the improvement of AI systems, encouraging transparency, and ensuring the dissemination of accurate and unbiased information in the field of psychology.