GenAI, Voting and Manipulation: the New Cambridge Analytica?
In this second episode with Xavier Puig Farré on the AI impact podcast we discuss the impact on thought diversity, opinions and innovation. (episode released next Monday at 5pm CET on Youtube and Spotify)
LLMs might affect in a very pervasive way our opinions and why not being used to push for a political agenda? There are several levels where LLMs can be pushed to do so:
0. System Prompts
Models have hidden instructions that could be used for censorship or in more subtle ways. (e.g. Tiananmen Square censorship or if you ask ChatGPT for images of Donald Trump in different scenarios)
1. Prompt Rewriting
Often systems have post-processing that rewrite or refine user queries. This is common for text-to-image models. If you ask ChatGPT to create an image you can get the actual prompt ChatGPT used for Dall-E and you will see that it is not what you asked. It was rewritten. This is also potentially what led Gemini to create Nazi-era German soldiers as people of color.
2. Post-Training “Alignment”
At this stage, there are two potential ways to bias the model. Creating specialized datasets of “instruction→best answer” pairs to refine how the model follows instructions or performs tasks (instruction tuning: instruction tuning focuses on giving the model explicit examples of how to respond to specific instructions or prompts).
Also using RLHF (Reinforcement Learning from Human Feedback): Having humans compare pairs of model outputs and ranking which output is “better,” then using RL to steer the model toward producing more of the “better” kind of outputs.
3. Pre-Training Data & Filtering
During pre-training, the organization can select or remove massive chunks of internet text and favor one view over another.
These biases are more or less "controlled/volunteer" and pervasive. Censorship is obvious while bias arising from the large corpus of text (internet skew, popular news site coverage, forum content, etc.) is more subtle and less controlled. The WEIRD bias means that the view of the world of WEIRD people is over represented and LLMs tends to align more with this view (WEIRD: Western Educated Industrialized Rich and Democratic)
Check the full video of my channel