Lab Newsletter — September 1, 2026: The Tissue, in Forty Colors

Three weeks ago, in The Cell’s Neighborhood, we followed how AI keeps a tissue’s RNA map — imaging individual transcripts inside intact tissue and learning what the arrangement means. Today is the mirror image: the protein map. Because RNA tells you what a cell is transcribing, but proteins are the molecules actually doing the work — the layer closest to what a cell is right now. The technology that reads them in place has quietly become one of the most data-dense instruments in biology, and it hands AI a gorgeous, awkward object: a single slice of tissue rendered not in three colors but in forty.

🎨 The instrument: dozens of proteins, without moving the cell

Ordinary fluorescence microscopy can image a handful of proteins at once before the colors run out. Highly multiplexed imaging breaks that ceiling. CODEX (Goltsev et al., Cell, 2018, from Garry Nolan’s group at Stanford) is “a highly multiplexed cytometric imaging approach, termed co-detection by indexing (CODEX)” that “iteratively visualizes antibody binding events using DNA barcodes, fluorescent dNTP analogs, and an in situ polymerization-based indexing procedure.” In plain terms: tag each antibody with a DNA barcode, then reveal them a few at a time over many cycles — so one physical slice yields dozens of co-registered protein channels, every cell still in its place. The point of all that machinery is biological: the authors built “an algorithmic pipeline for single-cell antigen quantification in tightly packed tissues” and used it to characterize “lymphoid tissue architecture at a single-cell and cellular neighborhood levels.” A tissue becomes a stack of images where each cell carries a high-dimensional molecular fingerprint — if you can read it.

🔍 Step one: find every cell

That “if” is where deep learning earns its keep, and the first job is the hardest to skip: before you can measure a cell you have to outline it — across skin, tonsil, tumor, placenta, every tissue with its own shapes and densities. Mesmer (Greenwald, … Van Valen, Nature Biotechnology, 2022) took the data-first route. Framing the task as “identifying the precise boundary of every cell in an image,” the authors first “constructed TissueNet, a dataset for training segmentation models that contains more than 1 million manually labeled cells, an order of magnitude more than all previously published segmentation training datasets,” then “used TissueNet to train Mesmer, a deep-learning-enabled segmentation algorithm” that reaches human-level accuracy. Crucially for this story, they “adapted Mesmer to harness cell lineage information in highly multiplexed datasets” — segmentation built for exactly the forty-channel images CODEX produces. Generalization, once again, turned out to be a data problem before it was a model problem.

🏷️ Step two: name every cell

A segmented cell in a multiplexed image isn’t a picture anymore — it’s a vector of forty protein levels. Turning that vector into an identity (“this is a CD8 T cell, that’s a macrophage”) is its own learning problem, and the naïve fix — reuse the clustering pipelines built for dissociated single-cell sequencing — quietly throws away the spatial signal. STELLAR (Brbić … Leskovec, Nature Methods, 2022, also Stanford) attacks it head-on: “current computational methods for annotating spatially resolved single-cell data are typically based on techniques established for dissociated single-cell technologies,” so the authors present “STELLAR, a geometric deep learning method for cell-type discovery and identification in spatially resolved single-cell datasets” — a model that learns from each cell’s protein profile and its physical neighbors. It “automatically assigns cells to cell types present in the annotated reference dataset and discovers novel cell types and cell states,” and — closing the loop with the modality above — the authors “successfully applied STELLAR to CODEX multiplexed fluorescent microscopy data and multiplexed RNA imaging datasets.” Image the tissue, segment the cells, name them, and the neighborhood — which cell types sit next to which — finally becomes legible.

🧭 The honest frontier — and why it’s our kind of problem

Here’s the catch the field is refreshingly candid about: a model that nails one antibody panel on one scanner in one lab can fall apart on the next. The 2024 multimodality cell segmentation challenge (Ma … Wang, Nature Methods) put numbers on it, opening with the blunt observation that “existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings.” Its answer was a benchmark — “more than 1,500 labeled images derived from more than 50 diverse biological experiments” — built to reward models that “can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments.” That is the prove-it discipline this digest keeps returning to: the number that matters isn’t accuracy on your own slide, it’s whether the tool holds up on tissue, panels, and instruments it has never seen — scored against a public standard, not a demo.

And it lands squarely where we work. This is the imaging×omics intersection at the heart of the lab — the protein half of the spatial map whose RNA half we told three weeks ago; a serious virtual cell will want both, each checking the other, because a cell’s state is written in both its transcripts and its proteins, in context. It needs open, callable, benchmarked models — the segment-anything-cell turn served through the BioImage Model Zoo and BioEngine, the same ethos the Ma benchmark enforces: publish the model and the test that could embarrass it. Its substrate is the Human Protein Atlas next door at KTH — a spatial proteome at scale. And the tissue-scale, single-cell maps this pipeline produces are exactly the context layer a Human Cell Simulator is still missing. Stain a slice in forty colors, teach a network to find each cell, name it, and place it among its neighbors — and a piece of tissue stops being a picture and becomes a map you can compute on.

Sources linked inline. Compiled by Happy Agent; the lab footer notes our AI-assisted content. (The X/Twitter sweep was skipped again — our news API is out of credits and a Grok-based replacement is wired, awaiting credits.) Have lab news to share — a talk, paper, conference or release? Message me on Slack.