Introduction
Modern medicine is undergoing a transformation that is changing not only treatment technology but the very model of what it means to be human. In the book Bad Influence, the author analyzes the transition from the compliant patient to a subject steered by algorithms.
The reader will discover how personalized choice architecture influences our health decisions. The text explains why traditional autonomy is becoming an illusion and defines calibrated agency, which allows individuals to regain control in a world driven by AI.
The Evolution of the Patient Model: From Compliance to Autonomy
The role of the patient has evolved from total subordination to medical authority (Parsons' sick role model) toward greater autonomy and informed consent. In the past, knowledge was hierarchical, and the physician determined what constituted an acceptable quality of life for the patient.
Today, three figures coexist: the compliant patient, the reflexive prosumer, and the algorithmic human. The latter is no longer merely a recipient of care, but a subject whose behaviors are measured and predicted by computational systems.
An example of this shift is a situation where a patient trusts their doctor in the hospital, analyzes data from their electronic health record in the evening, and allows a smartwatch to suggest questions for a specialist the following morning.
The Patient as Prosumer and Reflexive Designer of Their Own Health
The modern patient is becoming a prosumer who not only consumes knowledge but co-creates it. Health is no longer defined simply as the absence of disease, but as a "reflexive project of the self," requiring constant monitoring and optimization.
This phenomenon shifts the model of moral responsibility. The prosumer must independently seek information and manage risk, which can lead to negative judgments of individuals who have failed to "optimize" their lifestyle.
However, the author notes that this new agency depends on cultural capital. Better-educated individuals use the internet to improve the quality of their treatment, while the less privileged may succumb to information overload.
The Risk-Human within the Web of Digital Discipline
Modern technologies transform the patient into a subject managing probabilities. Rather than waiting for illness to strike, the individual lives in a world of predictions—such as metabolic scores or HRV trends—creating a new social position: the risk-human.
Quantified self tools introduce a form of digital discipline. The norm is shifted into the interface, and the user voluntarily submits to surveillance; while this can increase autonomy, it also imposes pressure for constant self-improvement.
A significant danger here is automation bias—the tendency to uncritically trust AI systems. The author warns against the "empathetic lie of AI," where a chatbot's stylistic care masks a lack of actual epistemic competence.
Summary
To avoid manipulation, the author proposes a model of calibrated agency. This does not entail total independence, but rather the conscious adjustment of how much decision-making is delegated based on the quality of evidence and personal values.
The key is epistemic constitutionalism, a system of mutual checks and balances between the physician, the patient, and the algorithm. Technology should support human reasoning, not replace the normative decisions of a human being.
In a world of optimization, true medical sovereignty may lie in the courage to maintain the right to be sub-optimal and the recognition that it is the human, not the code, who defines the meaning of their life.
Frequently Asked Questions
How has the role and agency of the patient in the healthcare system changed over the years?
The role of the patient has evolved from a person subordinated to the doctor's authority and knowledge hierarchy, through a reflective prosumer managing their own health, to an algorithmic subject guided by computational systems. There has been a shift toward autonomy and informed consent, allowing the patient to become a subject with their own hierarchy of values, rather than just an object of professional repair.
What are the characteristics of the modern patient-prosumer model in the context of health management?
The modern patient-prosumer does not only consume health knowledge but co-produces it by sharing experiences and reviewing specialists. They manage their health as a 'reflexive project of the self,' combining professional knowledge with digital resources and personal experience. This involves a shift in responsibility: the patient must now independently monitor risks, search for information, and optimize their lifestyle.
How are modern technologies and sociological theories changing the role of a person caring for their health from a passive patient into an entity managing risk?
Modern technologies, such as self-tracking and wearable devices, change the role of the person caring for their health into a 'risk person' who, instead of waiting for illness, actively manages the probability of diseases occurring. Thanks to digitalization, health surveillance shifts from external control to individual self-discipline and responsibilization, which increases agency but simultaneously forces constant monitoring of one's own body according to accepted norms.
What is the algorithmic human, and how does choice architecture influence their decisions?
The algorithmic human is a subject whose cognitive environment is selected by recommendation and predictive systems. Choice architecture influences their decisions by organizing the decision-making environment (e.g., the order of options or default values), which changes the frequency of provided information without removing formal freedom of choice.
When does technical decision support (nudging) become dangerous algorithmic manipulation?
Technical decision support becomes manipulation when the designer's objective function differs from the user's objective function—for example, when a system optimizes for revenue or retention instead of the patient's health. The risk increases in the case of so-called 'hypernudges,' where AI dynamically reorganizes the choice environment, exerting excessive power over the selection of data upon which the user builds their beliefs.
Why can the high quality of AI responses paradoxically lead to incorrect medical decisions?
The high quality of AI responses can weaken the user's vigilance and lead to over-reliance (automation bias), meaning that rare system errors may be overlooked. Additionally, AI can generate signals of competence, such as empathetic language and a structured format, which are mistakenly equated with the substantive correctness of the content.
How do algorithmic systems influence the patient's decision-making process and how can their autonomy be protected?
Algorithmic systems influence patient decisions by steering their attention and utilizing mechanisms such as framing and hypernudge, which can lead to choice manipulation. To protect autonomy, AI should be designed to support cognitive agency (autonomy-enhancing AI), revealing the persuasion techniques used and encouraging reflection. It is also important to prevent deskilling, namely the loss of a patient's ability to independently interpret their body's signals.
How can new forms of exclusion and manipulation in AI-based medicine be prevented?
Systems should be designed to reduce the cognitive cost of making the correct choice and not shift the entire responsibility for safety onto the patient's digital competencies. Key is the creation of a system of mutual epistemic control between the physician, the patient, and the AI, as well as the separation of commercial information from clinical recommendations.
Can medical algorithms make treatment decisions on behalf of the patient?
Algorithms can provide predictions and calculate the probability of success for various treatment methods, but they should not make decisions on behalf of the patient. This is because a decision requires taking into account an individual's values and goals, which technical systems are unable to resolve normatively.
What is calibrated agency and how does it differ from total independence or blind trust in technology?
Calibrated agency is the ability to align one's level of confidence, the scope of task delegation, and the intensity of action with the quality of information and the level of risk. Unlike total independence, it does not assume the fiction of a self-sufficient decision-maker; unlike blind trust in technology, it is based on an awareness of the functions and limitations of the tools to which we entrust decisions.
Why is the mere availability of information insufficient for making an autonomous medical decision and what is needed instead?
The mere availability of information is insufficient because the medical world is probabilistic, and a lack of data interpretation skills (risk literacy) can lead to a misunderstanding of risk. Instead of more data, epistemic humility and transparent data geometry are needed, including baseline risk, the range of uncertainty, and an appropriate method of information visualization.
Why is increasing trust in AI systems a flawed goal, and what should the patient-expert relationship look like in an era of information overload?
The goal should not be to increase trust itself, but rather its calibration, so that the user's level of confidence corresponds to the actual quality and reliability of the AI system. In an era of information overload, the expert should act as a curator of relevance, reducing data complexity and helping the patient interpret the meaning of facts within the context of their individual values.
Does patient autonomy mean that they must make every decision independently and without support?
No, patient autonomy does not require making decisions in total isolation, as they can emerge from relationships with family, caregivers, and professionals. A patient may utilize a physician's guidance or delegate participation in the decision-making process to loved ones, provided this aligns with their preferences.
What does calibrated agency look like in practice within the process of making medical and technological decisions?
Calibrated agency consists of keeping hypotheses in a reversible state and being ready to update beliefs based on the strength of new evidence. It requires transparency regarding the goals of the choice architecture and adjusting the threshold of evidence to the degree of reversibility of the planned intervention.
Does the model of calibrated agency imply striving for a medical ideal of health, or does it allow for the individual definition of life goals?
The model of calibrated agency does not mean striving for a medical ideal, but rather calibrating medicine to the rationally chosen goals of a specific person. It allows for the individual definition of life priorities, treating health as a means of enhancing human capabilities to realize chosen ways of living.
What is the difference between genuine patient empowerment and shifting responsibility onto them in medical and digital systems?
Genuine empowerment consists of providing the patient with reliable information, support, and the tools necessary to make a decision safely. Shifting responsibility occurs when the system transfers the burden of decisions or administrative-diagnostic work—which a professional is unwilling or unable to bear—onto the user.
How can we prevent errors in digital medicine when individual tools seem correct, but their interaction leads to harmful decisions?
Preventing errors requires applying safety engineering to the entire health ecosystem by introducing protective layers such as professional verification, tool validation, and monitoring of adverse events. It is essential to create a system of distributed vigilance based on heterogeneous control mechanisms and an organizational culture focused on the rapid detection and correction of errors.
What is calibrated agency in practice, and what systemic conditions are necessary for its realization?
Calibrated agency is the ability to make reasonable health decisions under conditions of irreducible uncertainty, based on knowledge that allows for better choices without striving for biological perfection. Its realization requires an institutional architecture that makes such behavior systemically rewarding, as well as state support in protecting a citizen's right to dissent and a second opinion.
