emilio andere

Microscopy lets scientists literally see the biological world. We think giving agents the same ability will be an important source of progress in science. Today we introduce TissueMicroscopyBench, our first release in a new microscopy series covering tissues, cells, and in vitro systems. The benchmark comprises 49 evaluations built from four public imaging datasets. Agents must locate structures, trace their boundaries, reason through three-dimensional images, and follow changes over time. Some of the work: - Colorectal cancer sections stained with H&E and multiplexed immunofluorescence. Tasks include outlining mucosa, fat, and nerves, and locating evidence of tumor invasion. - Liver metastasis biopsies taken before and after treatment. Agents compare tumor compartments, PD-L1 staining, and T-cell-rich regions across the images. - Light-sheet recordings of living organoids. Overlapping structures and densely packed nuclei require agents to distinguish individual objects and draw their boundaries. - A regenerating leg recorded in three dimensions over time. Tasks examine the injury site, local cell divisions, tissue retraction, and the sequence of regeneration. Compared with our previous benchmarks, these evals are especially visually striking. The rollouts are dense, interleaving image manipulation (including hand-rolled segmentation and CV in Python) with reasoning. Some slides contain billions of pixels. Agents have to decide where to look, examine the relevant regions at sufficient resolution, and return precise outlines, tracks, or volumes. These are scored against expert annotations. Every task was also solved independently by a human as a check on task quality. We tested 22 models in 30 model–harness configurations, with three attempts per task. Claude Fable 5.2 (LSVP) with Pi had the highest pass rate at 34.0%, passing 50 of 147 attempts. Timeouts and invalid answers counted as failures. Twenty-three of the 49 tasks were never solved by any configuration. We found that agents can be shockingly bad at localization, often finding the general region of interest but drawing boundaries that included surrounding tissue or random patches of empty space. Small structures and changes across time were particularly difficult. We think visual perception and spatial reasoning will become core capabilities for scientific agents. It’s also just a super fun space to think creatively about evals (h/t Ian Diks). Excited to release more in this series and watch agents improve here. - Blog: blog.latch.bio/p/tissue-micr… - GitHub: github.com/latchbio/tissuemi… - Leaderboard: benchmarks.bio/tissuemicrosc…

Rare diseases collectively affect an estimated 300 million people worldwide. For patients and families, reaching a diagnosis can take years of referrals and testing. One place AI agents could help is in interpreting each new sequencing result or functional assay alongside the patient’s symptoms, family history, and earlier findings. We introduce RareDiseaseBench, a benchmark of 35 diagnostic workups reconstructed from published rare disease cases. Each agent starts with the patient’s presentation and part of the medical record. Over one or more visits, it must decide which tests to request, explain why, analyze the files it receives, and propose a diagnosis and management plan. Five cases from the benchmark: - A 13-year-old boy has growth restriction, hearing loss, and severe chilblains. An MN1 variant explains several developmental findings. Long-read sequencing reveals a translocation disrupting SAMHD1 that accounts for his inflammatory symptoms. - A 21-year-old man has heterotaxy with right atrial isomerism. A variant just beyond the standard ZIC3 transcript falls within an intron of an alternative transcript, causing 57 extra bases to be spliced into the RNA. - A boy develops progressive scalp hair loss while his eyebrows and body hair are preserved. A scalp biopsy shows miniaturized follicles, and sequencing reveals a 357-kb duplication spanning TAB3 and neighboring genes. - A 16-year-old boy with typical growth has a language disorder, learning disability, and autism. He carries two inherited variants in RNU2-2, a small nuclear RNA gene outside the coverage of his exome sequencing. - A boy with delayed motor milestones and enlarged calves has a creatine kinase level of 10,100 U/L. At a second visit, an outside lab reports a DMD inversion. Targeted long reads show duplicated segments containing exons 1 and 2 joined head to head. Each configuration made three attempts on each of the 35 cases. Gemini 4 Argon with Pi passed 46 of 105 attempts (43.8%). Claude Opus 5.5 with Claude Code and GPT-6 Astra with Pi each passed 35 of 105 (33.3%). A pass required the diagnosis and management plan to meet every criterion in the case-specific rubric. Timeouts counted as failures. Among the 14 models that passed at least one case, 2,097 runs failed. Data acquisition and diagnostic workup was the most frequent primary failure category (36%). In 15% of all analyzed failures, the agent failed to pursue the type of data the case required. In another 12%, it omitted a necessary non-sequencing investigation, such as a tissue stain, imaging study, or laboratory test. Paper: latch.bio/raredisease Leaderboard: benchmarks.bio/raredisease Blog: blog.latch.bio/p/raredisease… We think rare disease diagnosis is one of the areas where AI systems have the most potential to make rapid progress and directly benefit patients over the next few years. Incredible work from @chriswzou and team.