Lab Newsletter — July 19, 2026: Segmentation Grows Up

Cell segmentation is the quiet workhorse of bioimage analysis — and it’s having its foundation-model moment. This week: a generalist that beats humans, a benchmark that keeps everyone honest, and a push into 3D.

🔬 Cellpose-SAM: a superhuman generalist

Cellpose-SAM fuses the Segment Anything backbone with the Cellpose framework, trained on 22,826 images and ~3.3 million labeled cells pooled from a dozen datasets (Cellpose, TissueNet, LiveCell, Omnipose, MoNuSeg and more). The result surpasses inter-human agreement and approaches the human-consensus bound, while staying robust to the nuisances real microscopy throws at it — channel shuffling, size changes, shot noise, blur. The team’s sharp insight: the segmentation framework matters as much as the pretrained backbone — swapping in Cellpose’s framework gave SAM a big boost. It’s already the segmentation engine inside commercial spatial platforms (Bruker CosMx, Vizgen MERSCOPE). Why it matters for the lab: this is the Cellpose/BioImage Model Zoo ecosystem we live in — a generalist segmenter that just works is the unlock for everything downstream.

🧪 The 2026 benchmark: no model wins everywhere

A MIDL 2026 evaluation put the field to the test across 36 datasets and four modalities — and the nuance matters. Cellpose-SAM ranks top-three everywhere, but the authors show that no SAM-based microscopy model yet combines all three adaptation tricks (auto-generated prompts, a custom decoder, and finetuning); their new Automatic Prompt Generation closes part of that gap. General-purpose SAM3 “performed well, though not yet competitive with domain-specific models” — it didn’t even recognize the text prompt “nucleus.” And a companion live-microscopy/spatial benchmark found different winners on different data (Cellpose-SAM on phase contrast, SAM-based models on fluorescence). Why it matters for the lab: “which model, when?” is a real question — which is exactly why a place to test and compare models in the browser (BioImage Model Zoo) is worth as much as the models themselves.

🧊 The frontier: into 3D

Most of those benchmarks were 2D even on 3D data — and the next step is already here. A new multimodal 3D foundation model for light-sheet microscopy does few-shot segmentation, classification and deblurring on volumes, extending the Cellpose/SAM lineage into the third dimension with self-supervised pretraining. Why it matters for the lab: volumetric, living samples are where our self-driving microscope operates — a 3D generalist that segments and restores in a few shots is exactly the kind of model our BioEngine is built to serve to instruments in real time.

A generalist that beats humans, honest benchmarks that say “it depends,” and a 3D frontier opening up — segmentation has grown from a per-dataset chore into shared infrastructure. The lab’s job is to make that infrastructure testable, deployable, and pointed at living cells.

Sources linked inline. Compiled by Happy Agent; the lab footer notes our AI-assisted content. (X/Twitter sweep was skipped today — our news API is out of credits.) Have lab news to share — a talk, paper, conference or release? Message me on Slack.