AI-Assist Uplift and Human-in-the-Loop Pathology: Does Help Actually Help?

The diagnostic question

Computational pathology has produced a wave of algorithms that can detect tumor, grade lesions, and quantify biomarkers from whole-slide images. But a cleared or published algorithm is not the same as a clinical benefit. The question that matters for deployment is narrower and more practical: when a pathologist works with an AI tool, does the diagnosis get better, faster, or more reproducible than when the pathologist works alone?

This is the concept of assist uplift — the measurable difference in human performance attributable to the AI aid, rather than the AI's standalone accuracy. It is a distinct and more demanding claim. An algorithm can be highly accurate in isolation yet add nothing in practice (if pathologists ignore it, or if it flags only easy cases), or it can even harm performance (if it induces automation bias on the hard cases). The appropriate study design to answer this is the multi-reader, multi-case (MRMC) study, in which several pathologists each read a shared case set both with and without AI assistance, allowing the uplift to be isolated while accounting for reader and case variability.

Below, three MRMC-style studies — spanning prostate cancer detection, breast nodal micrometastasis, and HER2 scoring — illustrate what assist uplift looks like across different diagnostic tasks.

What the method is and how it works

In a human-in-the-loop workflow, the pathologist remains the decision-maker; the AI serves as a second reader, a triage/flagging tool, or a quantification aid. The three tasks here use convolutional neural-network–based image analysis trained on annotated whole-slide images:

• Prostate detection assist: an algorithm (Paige Prostate) that flags regions suspicious for carcinoma on core-biopsy slides, intended to reduce missed cancers [1].
• Nodal micrometastasis assist: LYNA (Lymph Node Assistant), which highlights candidate metastatic foci in breast sentinel lymph nodes, a task where small deposits are easily overlooked and review is time-consuming [2].
• HER2 IHC assist: an automated scoring algorithm for HER2 immunohistochemistry in breast cancer that classifies staining into the 0 / 1+ / 2+ / 3+ categories and can present its call to the pathologist [3].

In each case, "assist" means the pathologist sees the AI output and renders a final human judgment — the AI does not autonomously sign out.

The evidence

Prostate cancer detection (Raciti 2020). In an MRMC study of 304 single-core prostate biopsy slides, pathologists read cases both unassisted and with AI assistance. With assistance, cancer detection sensitivity rose from 74% to 90%, while specificity was maintained — meaning more true cancers were caught without a cost in false positives [1]. This is the signature of a useful triage/flagging aid: it rescues the small, easily missed foci that drive diagnostic misses in prostate biopsy practice. The study was conducted with commercial involvement (Paige), a conflict of interest readers should weigh [1].

Breast nodal micrometastasis (Steiner 2018). Using the LYNA algorithm in an MRMC design, pathologists reviewing sentinel lymph node slides improved micrometastasis detection sensitivity from 83% to 91% with AI assistance [2]. Just as importantly for workflow, average review time per image was roughly halved when the algorithm pre-highlighted candidate foci, and the subjective difficulty of the task decreased [2]. Micrometastasis detection is a classic needle-in-a-haystack problem, and this pairing — better sensitivity and faster reads — is the most attractive form of uplift because it improves both quality and efficiency. This study also carried a commercial conflict of interest (Google) [2].

HER2 IHC scoring (Krishnamurthy 2024). HER2 scoring is a reproducibility problem rather than a detection problem: the most consequential and least reproducible distinction is 0 versus 1+, which has gained importance with the emergence of therapies directed at HER2-low disease. In this study, the fully automated algorithm achieved a standalone accuracy of 92.1% [3]. More relevant to the assist question, when pathologists used the AI, inter-reader agreement on the 0-vs-1+ call improved from 69.8% to 87.4% [3]. Here the uplift is in concordance and reproducibility — getting different pathologists to agree on the same category — rather than in catching missed tumor [3].

What converges across the three. Despite three different cancers, three different tasks, and three different failure modes, the direction of effect is consistent: AI assistance improved the human reader's performance on the metric that mattered for that task — sensitivity for prostate [1], sensitivity plus speed for nodal micrometastasis [2], and inter-observer agreement for HER2 [3]. This convergence is the core of the "assist uplift" argument: the benefit is not a one-off for a single assay but a reproducible pattern across detection and quantification tasks.

Where it stands

The evidence suggests that well-designed AI assistance can measurably improve pathologist performance, and that the nature of the improvement is task-specific — detection sensitivity, speed, or reproducibility — rather than a single universal metric. That nuance is deployment-critical: a tool's value proposition must be matched to the diagnostic bottleneck it is meant to relieve.

Several important limitations temper enthusiasm:

• Reader studies are not outcome studies. All three reports measure diagnostic performance on curated case sets, not downstream patient outcomes, length of stay, or treatment accuracy [1,2,3]. Improved sensitivity on a reader study is a surrogate; it does not by itself prove patient benefit.
• Conflict of interest. Two of the three pivotal studies were conducted with sponsor involvement [1,2]. Industry-sponsored reader studies are not disqualifying, but independent replication strengthens confidence, and these findings should be read with that caveat.
• Enriched and curated cohorts. Case sets assembled for reader studies may not reflect the prevalence, specimen quality, or staining variability of routine practice. Generalization to a new laboratory's scanners, stains, and case mix is an open empirical question for each site.
• Automation bias and the "hard case" problem. Uplift is typically demonstrated as an average across readers and cases. The failure mode that worries deployment teams — a confident-but-wrong AI output steering a pathologist off a correct call — is not fully captured by average sensitivity/agreement gains and warrants task-specific monitoring.
• Regulatory status varies and is evolving. Readers should not assume that a published reader study equals clearance for a given indication, scanner, and workflow in their jurisdiction. The responsible claim for each assay is: demonstrated assist uplift in a reader study, which is a necessary but not sufficient condition for routine clinical use. Confirm the current regulatory status locally before deployment.

On therapy linkage: improved and more reproducible HER2 0-vs-1+ categorization matters because that distinction governs eligibility for an entire class of HER2-directed agents in breast cancer [3]. The role of computational scoring is to make the categorization more consistent — it does not direct any individual patient's treatment, which remains a clinical decision integrating many factors.

The open questions for the field are whether these reader-study gains translate into measurable patient benefit, whether uplift persists under real-world variability and over time, and how to guard against overreliance. For now, the convergent MRMC evidence supports a specific, bounded conclusion: when the tool is matched to the task, AI assistance can help the human do better — and the human, not the algorithm, remains accountable for the diagnosis.

References

1. Raciti P, et al. Novel artificial intelligence system increases the detection of prostate cancer in whole slide images of core needle biopsies. Mod Pathol. 2020. PMID 32393768.
2. Steiner DF, et al. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am J Surg Pathol. 2018. PMID 30312179.
3. Krishnamurthy S, et al. Fully automated artificial intelligence–based HER2 immunohistochemistry scoring. JCO Precis Oncol. 2024. PMID 39393036; DOI 10.1200/PO.24.00353.