microscopy

Every photon a microscope collects can bleach a dye or kill a living cell, so the deepest tradeoff in imaging is signal versus survival. Deep learning quietly rewrote it. CARE restores usable images from 60-fold fewer photons; Noise2Void denoises from single noisy images with no clean targets at all; Deep-STORM and the Ozcan lab’s cross-modality GAN push past the diffraction limit — confocal to STED-matched resolution — with no PSF model; and DFCAN reaches SIM-quality detail over a tenfold longer window of live-cell imaging. It’s the least glamorous branch of AI-for-microscopy and maybe the most enabling: gentler light means longer movies, which is what autonomous, self-driving imaging needs. But restoration carries a sharp warning — a model can infer ‘unsubstantiated image details,’ inventing structure that was never there. That makes open, callable, validatable restoration models — the AI4Life / BioImage Model Zoo ethos — not a nicety but a safeguard, and the prove-it discipline non-negotiable.

Before you can measure a cell, you have to draw its outline — and for a decade every new microscope, stain, or organism meant training a new bespoke segmenter. The ‘Segment Anything’ moment changed the terms: a single promptable foundation model, adapted to the messy reality of microscopy. Two 2025 works make it real for biology — Cellpose-SAM, a preprint claiming ‘superhuman generalization’ by grafting SAM’s backbone onto Cellpose, and Segment Anything for Microscopy (μSAM), a Nature Methods tool that does interactive and automatic segmentation across 2D, 3D, tracking, light and electron microscopy in one open napari plugin — and publishes its models straight into the BioImage Model Zoo. It’s the lab’s home turf, and the load-bearing first step under spatial omics and a virtual cell.

Today in AI for life science: AI-designed proteins become new tools to see inside living cells, generative models write genomes on demand, and a synthesis breakthrough finally lets labs build what AI dreams up.

Today in AI for life science: vision-language models turn microscopes into conversational instruments, a reality check on the gap between seeing and understanding an image, and multimodal pathology models fusing slides with clinical text at scale.

Today in AI for life science: cell-segmentation foundation models reach a superhuman generalist in Cellpose-SAM, a 2026 benchmark shows no single model wins everywhere, and the frontier pushes into 3D light-sheet microscopy.

Today in AI for life science: a cell-language model proposes a cancer lead that holds up in the wet lab, agents turn plain-language requests into reproducible image analysis, and microscopes start deciding what to image themselves.

Today in AI for life science: Biohub’s protein world model designs cancer binders that work in the wet lab, ’thinking microscopes’ turn instruments into co-scientists, and a sharp reminder that predictions still have to survive the bench.

At an invited talk at CZI, we ran a live, fully agent-controlled wet-lab experiment on the REEF Imaging Farm. A single natural-language prompt set an entire lab in motion, on real cells, in real time. We interviewed the agent that ran it.

A self-driving cell-biology lab where AI agents plan, run and adapt live-cell experiments across a cluster of instruments.

Wei Ouyang and Hanzhao Zhang wrote the ‘Large Language Models and AI Agents’ chapter for AI in Microscopy: A BioImaging Guide (HHMI Janelia) — a tour from deep learning to autonomous, agent-driven bioimage analysis.