A second pair of eyes: AI and the work of care

Abstract

AI’s contribution to health should be judged by the quality of care it enables, rather than the sophistication of the model alone. This paper examines three distinct roles: finding patterns in clinical information, reducing the burden of coordination, and accelerating scientific investigation. Clinical studies of retinal screening and mammography, together with protein-structure research and WHO guidance, establish that important capabilities are real but specific. I argue for a chain-of-care approach in which each claimed improvement must survive the transition from data to interpretation, decision, intervention, and follow-up. A simple hypothetical screening calculation illustrates why accuracy can mislead when prevalence and error costs are ignored. AI can extend clinical attention; its humane value depends on whether institutions turn that assistance into accessible, accountable care.

The patient is not a benchmark

A patient rarely experiences medicine as a classification task. They experience a symptom, an appointment they may struggle to obtain, a test, uncertainty, and a decision that affects their life. AI enters this sequence at particular points. It might examine an image, organize notes, or suggest a question. The value of that contribution depends on the rest of the sequence functioning well enough to make it useful.

This is the argument of the paper: AI should be evaluated along a chain of care. A more accurate signal matters, but its benefit can disappear if nobody acts on it, if the person cannot afford follow-up, or if the signal triggers unnecessary intervention. Technical competence is a necessary part of some improvements. It is not a substitute for an effective care system.

The history of medicine supports this systems perspective. Smallpox eradication required much more than a scientific insight; it depended on coordinated public-health work [1]. Contemporary AI follows a comparable institutional logic without being medically equivalent to vaccination. Producing a promising tool and delivering a better population outcome remain different achievements, connected by evidence, resources, and sustained implementation.

A bounded success can be a major success

The 2018 pivotal study of an autonomous system for diabetic-retinopathy detection in primary-care offices illustrates a narrowly defined clinical application [2]. Its importance lies partly in that specificity. A system evaluated for a particular screening purpose does not thereby become a general eye specialist, a complete diagnosis service, or a replacement for every subsequent clinical decision.

The MASAI trial’s 2023 clinical safety analysis likewise evaluated AI-supported mammography reading against standard double reading in a particular screening setting. It reported a 44.3% reduction in screen-reading workload in that analysis [3]. This is a dated trial result, not a forecast for every radiology department or proof of a reduction in cancer mortality. Workload, detection, treatment, and survival are different outcomes.

These studies matter because they move discussion beyond possibility into structured evaluation. They also demonstrate why the phrase “AI in medicine” is too broad for serious judgment. A screening system, a note-drafting assistant, and a molecular model have different users, failure modes, and evidentiary obligations. Enthusiasm becomes more credible when it can name the exact task and the claim being tested.

Why “90% accurate” can tell you very little

Consider an entirely hypothetical screening test, not a description of any device or disease. Imagine 1,000 people, of whom ten have a condition. Suppose a test detects 90% of those cases and correctly clears 90% of people without the condition. It would identify nine true cases, miss one, and falsely flag 99 people among the remaining 990.

The 108 positive results would therefore contain nine true cases, or about 8.3%. Nothing in this arithmetic proves that screening is useless. It shows that interpreting a positive result depends on prevalence, sensitivity, specificity, and what happens next. A system can have impressive-sounding headline performance while generating many false alarms in a low-prevalence population. Different thresholds change which errors occur and who bears their cost.

This example explains why clinical evaluation needs more than a single accuracy score. A false negative can delay needed attention. A false positive can create anxiety, cost, and additional testing. The balance depends on the condition, the available follow-up, and the purpose of the test. Those are clinical and institutional judgments that a generic model leaderboard cannot settle.

From prediction to care

I propose separating five links: obtaining appropriate information, interpreting it, deciding what should happen, carrying out the intervention, and checking the result. AI may contribute to more than one link. Each transition introduces a new question. Was the right patient identified? Was information missing? Did a clinician understand the uncertainty? Was the follow-up actually completed? A correct intermediate output does not answer these questions automatically.

An illustrative clinic could use an assistant to assemble a patient’s recent records before an appointment. That may help the clinician spend less time navigating documents. But if the summary merges two people, omits a medication, or presents an old diagnosis as current, the convenience creates a new hazard. The system needs traceable source material and a workflow that makes checking practical, not merely theoretically possible.

The same reasoning applies to patient communication. An explanation can be grammatically excellent yet poorly suited to a person’s language, literacy, or circumstances. A useful system should allow clarification and correction without making the person feel they have failed a conversation with a machine. Clear communication is part of the care process, not decoration added after the technical work is finished.

The quiet opportunity in administration

Discussion often concentrates on dramatic diagnostic performance. A less glamorous opportunity lies in helping people coordinate care: preparing records, organizing referrals, drafting routine communications, and identifying missing information for human review. These are application possibilities, not claims that all such systems have demonstrated better health outcomes. Their potential comes from reducing friction around expertise that already exists.

A hospital does not need a computer to impersonate a brilliant physician in every conversation. It may first need the correct referral to arrive with the correct records at the correct time. This is less cinematic than a robot announcing a diagnosis. It may also be more useful. A technology’s social importance should not depend on whether its task makes a good movie trailer.

Administrative assistance still deserves careful evaluation. Automatically creating a convincing but inaccurate record can spread error through later decisions. Time saved by one clinician may become correction work for another. An evaluation should therefore follow information downstream, measure omissions and rework, and ask whether staff gained usable attention rather than simply receiving a higher volume of tasks.

AI as an instrument of discovery

AlphaFold’s 2021 research demonstrated a major advance in predicting protein structures, evaluated through the CASP14 setting [4]. A predicted structure can help researchers form hypotheses and choose experiments. It is not itself a demonstration that a proposed medicine is safe, effective, manufacturable, or clinically useful. The distance from a scientific prediction to an approved intervention remains substantial.

A useful analogy is a better map for an expedition. The map can suggest routes that would otherwise be difficult to identify, but it does not make the terrain disappear. Experiments, biological context, clinical studies, and production still matter. AI can change which questions become economical to ask without eliminating the need to test the answers against the world.

This is a broader account of acceleration. Scientific progress can improve when researchers reject weak candidates sooner, compare more plausible hypotheses, or gain access to useful representations. More generated hypotheses are not automatically more knowledge. The important measure is the rate at which sound, reproducible understanding accumulates, including evidence that a promising idea was wrong.

Trust must survive a change of setting

A system’s performance can change when the population, equipment, procedures, or documentation differ from those in its evaluation. That is a reason to test locally and monitor deployment, not an argument that useful generalization is impossible. The institution needs to know what the system was designed for, which inputs it can handle, and when it should abstain or return the case to a person.

WHO’s guidance on large multimodal models addresses governance, safety, privacy, and accountability across health applications [5]. Its relevance here is that convincing language is not a safety mechanism. A clinical tool needs evidence and oversight proportionate to its intended use. A document describing principles is a starting point; the institution must translate those principles into operational decisions and responsibilities.

For example, a clinician who is nominally responsible but lacks the time or information to inspect a result cannot provide meaningful oversight. Responsibility requires an actionable view of uncertainty, access to supporting evidence, and authority to disagree. Human review becomes ceremonial when the organization rewards speed while withholding the conditions needed for careful judgment.

Care has a distribution problem as well as a prediction problem

An effective tool can widen disparities if it is deployed where resources are already plentiful while underserved settings lack equipment, connectivity, or follow-up capacity. It can also reduce disparities if it helps extend a validated service to people who previously could not reach it. Which outcome occurs depends on deployment arrangements, affordability, staffing, and the design of the referral pathway.

This prevents a simplistic choice between technological optimism and caution. The cost of unsafe deployment matters, and so does the cost of withholding a useful capability. Evaluation should compare realistic alternatives, including the existing service and its failures. “A human would do it” is not a complete benchmark if the actual alternative is that no qualified person is available in time.

A practical research agenda

Future studies should connect technical measures to patient-relevant outcomes, examine error distribution across groups, and account for the work shifted between staff and patients. They should also study what happens after recommendations: whether follow-up occurs, whether disagreements are resolved, and whether performance changes over time. These questions turn a tool evaluation into an evaluation of care.

The argument has limits. It is a synthesis of selected examples and a proposed framework, not a systematic review or a claim that one deployment model fits every health system. Some applications may appropriately operate with substantial autonomy after relevant evaluation and authorization. Others may remain advisory. The level of autonomy should follow the use case and evidence rather than the appeal of a label.

The promise is considerable: more useful attention, better-informed research, and less effort lost between people who need care and people able to provide it. But the patient remains the purpose of the system. A second pair of eyes is valuable when it helps someone see more clearly and act more responsibly. It is a poor substitute for an institution that has stopped looking.