Submission
Privacy Policy
Code of Ethics
Newsletter

Not Neutral – Detecting Political Tilt in Large Language Models

Large Language Models are fast becoming the mediators between information sources and users, so their value signals matter. A recent peer-reviewed study in Nature’s Humanities and Social Sciences Communications finds a measurable rightward shift in models’ political placement across versions, using a standardized Political Compass framework. If models can drift, transparency and routine, version-to-version monitoring is no longer optional.

In recent years, Large Language Models (LLMs) have become deeply embedded in everyday communication, knowledge sharing, and how people access information. Tools like ChatGPT, Claude, Perplexity, and Gemini are not just technical novelties, but they increasingly act as mediators between information sources and users, a role that search engines used to play. Just as search ranking and the credibility of results shaped how we see the world, this mediating role is shifting toward LLMs, along with the responsibility that now falls on their developers. By default, generative AI systems do not hand sources to the users. They synthesize “ready-made” answers where checking facts or tracing citations is much more complex. This raises a practical question: do these systems convey specific value systems, and if so, how stable are those values, what biases (including political ones) might they carry, and how transparent can those biases be? A recent study examined exactly this problem from the angle of whether the political “value system” of LLMs is measurable, and if it is, whether it shifts over time and in which direction.

The study fits into the broader discussion of AI alignment, that is, aligning advanced systems’ goals and behavior with human values. The aim is to ensure that these systems act in ways consistent with human values. Over the past decade, work by Nick Bostrom (Superintelligence, 2014) and Stuart Russell (Human Compatible, 2019) has made this a central topic in AI ethics and safety. Its importance is underscored by practical issues such as reward misspecification (when, even implicitly, we define the reward so poorly that the system starts pursuing the wrong goal), specification gaming, and the Goodhart effect. The task is further complicated by the diversity and context dependence of human values, by the fact that training data often contains unclear or hidden value systems, and by how difficult it is to map values in the massive datasets that underpin today’s models. These uncertainties justify adding a new focus to such evaluations; not only aligning models with human values but also monitoring how encoded value systems may change over time and across updates. A recently published study examined how political value systems appear in LLMs’ answers along three dimensions (individual freedom, economic intervention, and social justice), noting that values present in training data, including hidden ones, can be especially strong in political topics and can show up as measurable shifts in model outputs.

To examine where a given Language Model sits along political values, the study used the Political Compass Test. This is a widely known questionnaire that places respondents on two dimensions based on agree or disagree responses to statements. The first is the economic axis with left and right poles. The left position generally supports a larger state role, stronger redistribution, and tighter regulation, while the right tends to favor a smaller state apparatus, a more flexible market, and a stronger role for individual enterprise. The second is the social and cultural axis, where the authoritarian pole prioritizes greater state control and more centralized hierarchy, and the libertarian pole emphasizes greater individual freedom and minimal intervention. Based on their answers, models received scores on both axes, which together produce a single coordinate showing where they fall on the Political Compass map, that is, on the coordinate system defined by the left right economic axis and the authoritarian libertarian social axis. If that point moves over time, it indicates a shift in political placement, which the study sought to measure and compare across model versions.

The study analyzed several ChatGPT model versions based on thousands of answers. It found that newer models, like earlier ones, fall in the libertarian–left quadrant. They tend to emphasize individual freedom on the social axis and show more left-leaning preferences on the economic axis. At the same time, across successive versions there is a measurable rightward shift on both axes, which the authors call a value shift. This does not mean the models have moved to a conservative quadrant. It means that, compared with earlier states, they have shifted toward the right axis, for instance, they may place less emphasis on certain forms of state intervention, give more weight to market solutions, or accept less state control in questions of social freedom. The researchers could not link this shift directly to changes in training data or to different accounts used for the experiment and instead associate it with model updates and fine-tuning. Taken together, this suggests that models’ political placement is not static and may differ from what developers or users expect.

A direct takeaway from the results is that LLMs should be monitored continuously and methodically to improve their transparency. In practice, this could mean regular, cross-version comparative measurements with a unified question set and fixed settings, so that the direction and causes of changes can be reliably traced afterward. Since chatbots built on LLMs are increasingly acting as information mediators, even small shifts in their answers can influence how we frame issues at a societal level, what answers we receive, and what conclusions we draw. This is especially important in situations where such tools are used in education, customer service, or the production of media content.

At the same time, the study’s limitations call for caution. The findings apply to a single model family, the ChatGPT model versions, and cannot be automatically generalized to other systems. It must also be considered that the Political Compass was designed for human respondents, so applying it to Language Models requires particular care in interpretation. Nor can we rule out that other, unidentified factors contributed to the observed shift. Even so, the study clearly indicates that models do have political value systems, these can be measured, and they evolve with new versions, which is why value alignment and transparency should be checked on a regular basis.

The results presented here suggest that LLMs are not neutral tools. At times they produce answers shaped by political value systems, which can be measured and can change over time. The examined ChatGPT versions remained in the libertarian–left range, while showing a detectable rightward shift. The causes of this shift cannot be clearly tied to changes in training data or to account differences and are more likely linked to updates and fine-tuning. Since these systems increasingly act as information intermediaries in society, it is in our shared interest to maximize transparency and to conduct regular, cross-version value assessments. Responsible use requires ongoing monitoring, clear documentation, and a conscious handling of methodological limitations.


István ÜVEGES, PhD is a Computational Linguist researcher and developer at MONTANA Knowledge Management Ltd. and a researcher at the HUN-REN Centre for Social Sciences. His main interests include the social impacts of Artificial Intelligence (Machine Learning), the nature of Legal Language (legalese), the Plain Language Movement, and sentiment- and emotion analysis.