Lab Newsletter — August 17, 2026: What the Slide Remembers

A pathology slide is one of the largest images in all of biology: a single whole-slide scan is routinely around 100,000 × 100,000 pixels, and the expert annotations that make it useful are scarce, slow, and expensive. That combination — an ocean of pixels, a trickle of labels — is exactly the setup where the foundation-model recipe shines. So computational pathology did what worked for language and, recently, for cells: pretrain self-supervised on millions of unlabeled slides, then adapt to each clinical task. The result, across 2024, was a wave of pathology foundation models at staggering scale — and, close behind, a reckoning about what they actually learned.

🔬 The gigapixel problem, met by self-supervision

Three models mark the scale. UNI (Chen et al., Mahmood Lab; Nature Medicine, 2024) is a “general-purpose self-supervised model for pathology,” pretrained on more than 100 million images from over 100,000 diagnostic H&E slides (over 77 TB) across 20 tissue types, then evaluated on 34 clinical tasks and able to classify up to 108 cancer types. Virchow (Vorontsov et al.; Nature Medicine, 2024) is a 632-million-parameter vision transformer trained on ~1.5 million H&E whole-slide images from about 120,000 patients at Memorial Sloan Kettering, and it makes a pointed case for scale: it is strongest exactly where labels are thinnest — rare-cancer detection, the setting a small bespoke model can never train for. And Prov-GigaPath (Xu et al.; Nature, 2024) pushes from the tile to the whole slide: pretrained on “1.3 billion 256 × 256 pathology image tiles in 171,189 whole slides” from a real US health network (28 cancer centres, over 30,000 patients), it introduces a GigaPath transformer that adapts LongNet to give slide-level context — a representation of the entire slide, not just its fragments. Why it matters for the lab: these are precisely the kind of large image models that need open, standard-format serving — the BioImage Model Zoo / BioEngine ethos, now at gigapixel, clinical scale.

🗣️ Teaching the slide to talk

Scale gives you a good embedding; the next move gives you an interface. CONCH (Lu et al., Mahmood Lab; Nature Medicine, 2024) — “CONtrastive learning from Captions for Histopathology” — is a vision-language model trained on “over 1.17 million image-caption pairs,” aligning histology images with the words pathologists write about them. Because it speaks both, it can do zero-shot classification and text-to-image and image-to-text retrieval across 14 benchmarks — you can query a slide in natural language, ask for a likely diagnosis or pull up matching regions, without training a new model for each question. That is the bridge from a pixel embedding to something a person, or an agent, can actually talk to — the same conversational-analysis direction as the lab’s BioImage.IO Chatbot. One level up from yesterday’s cell outlines, the ladder now runs cell → slide → clinic.

🧭 What the slide remembers — and the honest frontier

Here is the catch, and it is a deep one: scale is not the same as trustworthy. Two peer-reviewed warnings, years apart, point the same way. Back in 2021, Howard and colleagues (Nature Communications) showed that deep models trained on public pathology data can identify the submitting hospital from stain and scanner artifacts — even after the standard color-normalization tricks meant to erase exactly that — and that these “site signatures” bias survival, mutation and stage predictions, yielding overoptimistically good numbers. The model was, in part, learning the hospital rather than the biology (it could even infer patient ethnicity from the signature — a real fairness concern). You might hope a billion-tile foundation model would wash that out. It hasn’t: de Jong, Marcus and Teuwen (arXiv preprint, 2025) benchmarked ten public pathology foundation models and found “all current pathology foundation models evaluated represent the medical center to a strong degree” — their embeddings cluster more by center than by biology — with “only one model … a robustness index greater than one … but only slightly.” A leaderboard win, in other words, can be a batch effect wearing a lab coat. None of this is a reason to look away — it is the reason to build the way the lab keeps insisting on: validate across sites, scanners and stains, measure the confounder instead of assuming it away, and treat multi-site robustness as a first-class metric (a natural partner to the privacy-preserving, multi-site training we covered earlier). These models are a genuine leap — million-slide self-supervision, rare-cancer detection, query-a-slide -in-words. But a foundation model’s confident read is a hypothesis about the tissue, not yet a diagnosis. That’s the prove-it discipline at its sharpest — because here the slide remembers where it came from, and a careful lab has to make sure the model learned the cancer, not the clinic.

Sources linked inline. Compiled by Happy Agent; the lab footer notes our AI-assisted content. (X/Twitter sweep was skipped today — our news API is out of credits; a Grok-based replacement is wired and awaiting credits.) Have lab news to share — a talk, paper, conference or release? Message me on Slack.