Introduction
The modern relationship between humans and AI extends beyond simple task automation. The real challenge lies in the subtle transfer of cognitive functions to machines, which may lead to the erosion of our own agency.
In this article, we analyze the risks associated with delegating judgment and the phenomenon of deskilling—the loss of competence resulting from infrequent use. You will learn how to distinguish AI support from the replacement of human thought, and why simply being 'in the loop' of a decision-making process does not guarantee control.
Delegating Judgment Leads to Deskilling
Standard automation concerns execution—such as calculations—where the human remains the architect of the problem and evaluates the result. The delegation of cognitive processes occurs when AI takes over data selection, hypothesis generation, and the prioritization of alternatives.
This shift upward in the cognitive hierarchy turns the human into a consumer of pre-packaged thought. While this is rational and efficient in the short term, it leads to deskilling. The user loses the ability to independently oversee the technology.
A prime example is medicine, where young specialists may fail to develop independent diagnostic judgment by relying too heavily on system suggestions.
Augmentation Over Automation Protects Cognitive Competencies
Augmentation enhances human capabilities by developing their competencies. Automation merely optimizes the outcome, often at the expense of the operator's growth. The difference becomes apparent in atypical situations or when an algorithm fails.
A key threat is automation bias, or the over-reliance on AI recommendations. This leads to the acceptance of erroneous suggestions and the overlooking of critical information that the system failed to highlight.
Effective collaboration requires a rational distribution of epistemic authority. A human must know when the system is superior and when it fails to avoid the trap of uncritically deferring to the machine.
Proactive Friction Prevents Mindless Compliance
Simply slowing down a process or adding warnings is insufficient, as users quickly treat them as formal rituals. What is required is cognitive forcing—designing interfaces that compel a genuine analysis of alternatives.
Independence is threatened even when multiple options are provided if they were all generated by the same model. This creates an illusion of choice and deepens epistemic dependence (Epistemic dependence).
To protect agency, one should employ proactive friction: requiring a preliminary assessment before revealing the AI's result or presenting arguments against the recommendation. This prevents the anchoring effect and the homogenization of thought.
Summary
The greatest risk is not a machine uprising, but the slow disappearance of the human from their own thought process. We must move from a model of human in the loop to human agency in the loop, where the human presence has real causal significance.
Today, autonomy is the ability to manage dependency. In the pursuit of efficiency, we cannot become mere witnesses to decisions that we are no longer capable of questioning. Ultimately, the key is distinguishing between a model's suggestion and our own conviction.
Frequently Asked Questions
What is the difference between ordinary work automation and the delegation of cognitive processes, and what are the associated risks?
Ordinary automation concerns the performance of tasks, whereas delegating cognitive processes involves handing over judgment functions to AI, such as generating hypotheses or prioritizing alternatives. This carries the risk of deskilling—the loss of independent competencies and the ability to oversee technology due to the less frequent application of cognitive operations.
What is the difference between augmentation and automation in the context of human competence, and how does the phenomenon of automation bias affect this?
Augmentation enhances human capabilities by developing the competencies necessary to control the outcome, whereas automation optimizes the result independently of the operator's development. Automation bias is the tendency to over-rely on automated systems, which can lead to accepting incorrect suggestions or overlooking critical information.
Is simply slowing down the decision-making process and adding warnings enough to prevent a human from succumbing to AI suggestions?
No, merely slowing down the process is not a panacea, as users may begin to click through warnings automatically. Cognitive friction only works when it leads to a real analysis of alternatives and makes the information necessary for critical evaluation actually available.
Does having multiple options generated by AI mean that we still maintain independence in decision-making?
Choosing from several AI-generated options can be illusory, as they all originate from a single model and are based on the same assumptions. This leads to an increase in epistemic dependence and the risk of the system replacing the user's independent judgment (so-called preemption).
Why does the mere presence of a human in the decision loop not guarantee real control over the AI system?
The mere presence of a human may amount to formal approval of decisions without real control if they lack the time, information, or competence to challenge the result. Under such conditions, the final click becomes merely a legitimizing ritual, and the human serves as a 'crumple zone,' taking responsibility for the failure of a system over which they had no actual influence.
How can AI systems influence our thinking styles, and do explainability features (XAI) actually increase our agency?
AI systems can homogenize thinking styles by imposing uniform argumentative forms and rhetorical repertoires in created documents. Explainability features (XAI) do not always increase agency, as they may merely persuasively build trust in the system instead of providing real data that allows for an assessment of the result's reliability.
How should AI systems be designed to assist humans in decision-making without stripping them of their ability to think independently and exercise critical judgment?
AI systems should be designed to enhance the quality of human decisions while preserving the capacity for critical control and revision. This requires the application of 'proactive friction' proportional to the risk of the decision, which may include presenting alternatives, indicating uncertainty, or refraining from answering in matters requiring human values. Key is the transition to a 'human agency in the loop' model, where the architecture of cognitive function distribution is explicit and the human retains real causal and epistemic influence over the process.
What is the difference between ordinary task automation and the dangerous delegation of cognitive processes to AI?
Ordinary automation involves entrusting calculations or information retrieval to a machine while maintaining independence in interpreting and evaluating the results. Dangerous delegation of cognitive processes occurs when AI takes over functions essential for recognizing the correctness of a result and shapes the problem structure, hypotheses, and recommendations, leading to the delegation of judgment rather than just tasks.
Why can using AI, despite being rational and efficient, lead to the loss of our skills?
Using AI can lead to so-called deskilling, which is the loss of specific skills due to their less frequent independent application. Immediate efficiency and energy saving in the short term may result in a long-term weakening of the ability to perform operations independently and difficulty in supervising the technology itself.
What is the difference between AI supporting a human versus replacing their cognitive processes, and how does this affect decision quality?
Supporting humans (augmentation) develops their competencies and allows them to maintain control over the outcome, whereas replacing cognitive processes (automation) can weaken expertise by eliminating formative stages of work. This affects decision quality in atypical situations or system errors, where a lack of independent operator competence leads to an over-reliance on AI (automation bias). The ultimate accuracy of decisions depends on whether the human can recognize model errors and properly calibrate their trust in its recommendations.
How can an AI interface be designed to prevent humans from uncritically accepting recommendations?
One can apply so-called cognitive forcing or 'proactive friction,' for example, by requiring an independent user assessment before showing the AI's suggestion, presenting arguments against the recommendation, or revealing system uncertainty. It is crucial to avoid situations where the AI response becomes an anchor and to design the interface so that it forces real processing of alternatives rather than just a formal confirmation of the choice.
Does having several response variants from an AI mean that we still maintain independence in decision-making?
Having several variants provides a sense of choice, yet it may mask the fact that all of them originate from the same model and are based on the same assumptions. Although the user retains the freedom to choose between proposals, they lose contact with solutions that the system did not generate, leading to an increase in epistemic dependence.
Why does the mere presence of a human in the decision loop not guarantee real control, and what are the risks resulting from long-term interaction with AI?
The mere presence of a human in the loop does not guarantee control when they lack time, competence, or information, making their role a mere legitimizing ritual and exposing them to taking full responsibility for system errors (the so-called moral crumple zone). Long-term interaction with AI, on the other hand, risks amplifying human biases and creating feedback loops in which humans and machines mutually reinforce the same cognitive errors.
Do transparent AI explanations and widespread content automation not lead to a standardization of the way we think and argue?
Using similar AI models can lead to the homogenization of rhetorical repertoires, justification styles, and the way arguments are formulated. This phenomenon creates a cognitive dependence with a cultural dimension, affecting how people construct analyses and communications.
How should AI systems be designed so that they do not replace human thinking, but genuinely extend it?
AI systems should be designed to improve the quality of human decisions while preserving the ability for critical control and management of technological dependence. This requires applying the principle of "proactive friction" (e.g., by presenting alternatives or indicating uncertainty), especially in high-risk situations, and transitioning from a model of human presence ("human in the loop") to a model of real user agency ("human agency in the loop").
What might the future relationship between humans and AI look like if it is not to lead to a loss of agency?
It is possible to create configurations in which humans and AI systems reinforce their different strengths and maintain the capacity for mutual correction. This is realized through concepts such as human-centered AI, Centaur, or Cyborg.
