Introduction
This text analyzes the risk of the emergence of an Artificial State, in which algorithms do not merely predict but actively shape the preferences of citizens. In the era of generative AI, the line between acceptable persuasion and manipulation is becoming increasingly blurred.
The reader will learn how information asymmetry impacts individual autonomy and how European regulations, such as the AI Act, attempt to protect the right to human judgment against technocratic optimization.
From Acceptable Persuasion to Algorithmic Manipulation
Traditional political influence relied on public communications that could be collectively verified. Modern AI persuasion utilizes micro-targeting and conversational systems to tailor arguments to a specific recipient in real time.
The danger lies in trapping the citizen in a loop: observation, prediction, and influence. An example can be seen in experiments from 2025, where chatbots shifted voting preferences more effectively than traditional advertisements, often by utilizing false information delivered in a responsive manner.
From Personalized Persuasion to Algorithmic Manipulation
Personalized persuasion appeals to the recipient's reasoning, whereas manipulation bypasses reflective mechanisms. A new threat is hypernudging—an adaptive choice architecture that dynamically adjusts stimuli based on an individual's hidden vulnerabilities.
AI creates a dangerous information asymmetry. The system knows the user's profile and millions of other interactions, while the human does not understand the logic of the process shaping them. This strikes at the foundations of autonomy, as the outcome of a choice ceases to be an authentic expression of will.
Industrial Scale and the Automation of Persuasion
Modern AI manipulation differs from old propaganda primarily in scale and cost. In the past, influence required vast human resources; today, automation allows for millions of unique, private conversations to be conducted simultaneously.
AI can serve either as a deliberative assistant, supporting conscious decision-making, or as an agent of persuasion. To prevent abuse, the constitutionalization of the algorithm is necessary. This means subordinating technology to the norms of the rule of law, such as source transparency and the prohibition of using sensitive data for manipulation.
Summary
Protecting the citizen requires recognizing the right to non-deduced subjectivity. A human being cannot be treated as a complete equivalent of their statistical model, especially in matters fundamental to their freedom.
In a world of more efficient machines, will the defense of human judgment become a fight for the right to make worse decisions? True freedom may soon mean the right to be less efficient than a statistical model and the preservation of democratic unpredictability.
Frequently Asked Questions
How does traditional political influence differ from modern AI-based persuasion, and why does it pose a threat?
Traditional political influence relied on mass messages available to the public, which allowed for the verification of contradictions. Modern AI persuasion is an adaptive and personalized process capable of modifying argumentation for a specific interlocutor in real time. It poses a threat due to the possibility of automating such interactions on a mass scale and the tendency of models to convince recipients using false or inaccurate information.
How does AI-personalized persuasion differ from manipulation, and why does it pose a new threat to the autonomy of the citizen?
AI-personalized persuasion differs from manipulation in that the latter deliberately exploits hidden knowledge about an individual's vulnerabilities to bypass their reflective mechanisms. It poses a new threat to citizen autonomy due to the vast information asymmetry and the system's ability to conduct millions of dynamically optimized conversations (hypernudging), the logic of which the recipient is unable to recognize.
How does contemporary AI manipulation differ from traditional political propaganda?
Contemporary AI manipulation differs from traditional propaganda primarily through industrial scalability, automation, and the low cost of generating messages. The novelty lies in combining the individualization of the message with the speed of experimentation and the ability to conduct a dialogue, replacing the former need to engage a massive human organization.
Must AI serve manipulation, or can it support conscious political decision-making, and how should this difference be regulated?
AI can support conscious political decisions by acting as a deliberative assistant that helps citizens understand electoral programs and analyze counterarguments, thereby increasing their autonomy. The difference between support and manipulation lies in the system's goal: an assistant facilitates the independent formation of preferences, while a persuasive agent strives for a specific political outcome. Regulating this difference would require introducing principles of AI constitutionalism and new regulations covering the auditability of system goals and parameters, as well as assigning responsibility for generated content to the institution implementing the technology.
How is the evolution of political technologies changing the role of AI in the democratic process, and does this mean a total loss of citizen autonomy?
The evolution of political technologies means that AI is ceasing to be merely a knowledge system about the citizen and is becoming an active participant in the persuasion process, capable of influencing voters' decisions. However, this does not mean a total loss of autonomy, as people are capable of recognizing persuasion and using AI to verify information.
Why are AI predictive systems fundamentally contradictory to the idea of democracy?
Predictive systems strive to reduce uncertainty, whereas democracy is based on institutionalizing that uncertainty and the citizen's freedom to change their mind. Precisely predicting behavior creates an incentive to shape the choice environment, meaning the outcome may become a result of the system's operation rather than an authentic expression of citizens' preferences.
How can the manipulation of citizens by AI infrastructure be prevented legally and systemically?
Preventing manipulation requires the constitutionalization of the algorithm, meaning subjecting AI systems to the principles of the rule of law, such as legality, transparency, and accountability. In practice, this is implemented through a layered system of norms, including data protection (GDPR), platform regulations (DSA, DMA), and the AI Act.
How do European legal regulations protect citizens from having their judgment completely replaced by algorithms?
The GDPR protects citizens by regulating profiling and granting the right not to be subject to a decision based solely on automated processing if it produces legal effects or similarly significantly affects them. In cases where such decisions are permissible, safeguards are required in the form of the possibility of human intervention, the ability to express one's own point of view, and the right to contest the decision.
How does European law (DSA and DMA) counteract the concentration of power in the hands of big tech platforms?
The DSA counteracts the concentration of power by regulating process infrastructure and managing systemic risks, including through audits and algorithmic transparency. The DMA, on the other hand, imposes ex ante obligations on so-called gatekeepers, such as ensuring interoperability, prohibiting the self-preferencing of their own services in rankings, and sharing data with business users.
How does the AI Act attempt to limit the influence of the 'Artificial State' on the citizen, and what are the barriers to its implementation?
The AI Act limits the influence of artificial intelligence by introducing a differentiated regulatory regime depending on the context of use and risk, and by imposing transparency obligations (e.g., informing users about interaction with AI). Barriers to its implementation include difficulties in translating legal aspirations into technical standards, limited administrative capacity, and the gap between rapid technological development and the slow pace of democratic procedures.
How do European legal regulations, such as the AI Act, limit the technological possibility of manipulating and optimizing the citizen?
European regulations, including the AI Act, limit manipulation by banning practices deemed unacceptable, such as harmful manipulation, exploiting vulnerabilities, or social scoring. They adopt the principle that the technical possibility of optimization does not grant the right to execute it, and the power of systems must be proportional to the legal and social risk.
Is the mere introduction of legal regulations enough to effectively control algorithms and avoid perpetuating the dominance of the largest corporations?
The mere introduction of regulations will not suffice, as overly complex rules may paradoxically strengthen the dominance of large corporations due to high compliance costs for smaller firms. Furthermore, effective control requires real institutional capacity of the state, including technical staff and computing power, to prevent oversight from becoming a mere formal ritual.
Where is the line between acceptable algorithmic assistance and the unacceptable replacement of human judgment within the state?
This line is drawn where the presence of human judgment is essential for the legitimacy of the process, particularly in matters concerning the rights, freedoms, and political status of the citizen. It is unacceptable to replace a human with a statistical model in situations requiring interpretation, accountability, and the recognition of the right to irreducible agency.
