RudolfV 2: A State-of-the-Art Open-Weight Pathology Foundation Model

🤗 Link to foundation models on HuggingFace

📜 Download PDF of preprint

The field of pathology is being transformed by large-scale slide digitization and AI-based image analysis, powered by the rise of pathology foundation models. Two and a half years ago we introduced RudolfV, our first pathology foundation model. The field has moved quickly since then, but one pattern has held: the most capable models are large, expensive to run, or released with closed weights, while openly available models have trailed in accuracy and robustness.

Today, we're excited to announce RudolfV 2, an open-weight model that closes this gap. Across a broad set of public benchmarks, it is the best-performing openly available pathology foundation model, approaching both the performance and robustness of our closed flagship, Atlas 2.

RudolfV 2 was trained on more than 300,000 whole slide images from Charité – Universitätsmedizin Berlin and LMU Munich, is built on a ViT-g architecture with approximately 1.1 billion parameters, and includes a dedicated post-training stage that to our knowledge is a first for pathology foundation models. We further distilled RudolfV 2 into two lightweight versions, RudolfV 2-B (ViT-B, 86M parameters) and RudolfV 2-S (ViT-S, 22M parameters), which preserve much of its performance at a fraction of the compute. We are releasing the weights of all three models for academic research.

In this overview, we discuss results from our evaluation across four established public frameworks that cover performance, robustness, and efficiency.

Setting a New Standard Among Open-Weight Models

To ensure reproducibility and comparability, we evaluated RudolfV 2 against leading publicly available foundation models on HEST, eva, PathoROB, and Plismbench, along with additional benchmarks for microsatellite instability and cancer-type classification. Our distilled models were also compared against other models of similar size.

On average, RudolfV 2 outperforms every other open foundation model we tested, including UNI2-H, H-Optimus-0, Midnight-12k, Virchow2, and Phikon-v2. It leads the open field on the morphology tasks, by 1.1 percentage points over Midnight-12k, and on the molecular tasks, by 0.2 points over UNI2-H. 

[Paper Figure 1] Average benchmark performance for each model, combining the morphology tasks from eva and the molecular tasks from HEST. The dot above each bar shows the model's processing speed (images per second  on an L4 GPU). RudolfV 2 has the highest average of any open-weight model, and the distilled RudolfV 2-B and 2-S achieve similar prediction performance to competing models while running considerably faster.

RudolfV 2 also performs on par with the closed-weights model Pluto-4G, and among all the models we tested it is surpassed only by our own flagship, Atlas 2.

Robustness Across Labs and Scanners

Robustness matters because the same tissue can look different depending on the scanner, stain, and lab that processed it. A model that performs well in one setting can falter when those conditions change. That's a real problem for research: a finding for one cohort may not be replicable on slides from another lab, simply because the model was reacting to scanner or stain differences rather than true biological signal.

We tested robustness on two benchmarks that cover medical center variation (PathoROB) and scanning and staining variation (Plismbench), the two most common sources of technical variation in multi-site data. RudolfV 2 is the most robust open model to date, ahead of Virchow2 by 8.3 points and close to Atlas 2. On the Plismbench leaderboard metric it edges ahead of the flagship.

This carries through to the smaller models: In our evaluation, RudolfV 2-B and RudolfV 2-S were markedly more robust than other open models of a similar size.

[Paper Figure 2A] Each model's average performance plotted against its average robustness, where robustness reflects consistency across scanners, stains, and medical centers. Among open-weight models, the RudolfV 2 family sits at the top right, combining high accuracy with strong robustness.

Efficiency: Faster Inference Without Sacrificing Performance

Better performance and robustness usually come at a computational cost. In pathology, where a single whole slide image is split into thousands of tiles, that cost can compound quickly. Distillation is one way to cut costs. Using a similar approach as the one we applied to Atlas 2, we distilled RudolfV 2 into two smaller versions:

• RudolfV 2-B: ~13x smaller in parameter size, 2.5x faster
• RudolfV 2-S: ~50x smaller in parameter size, 6.5x faster

Each is the most capable open model of its size released so far, matching the performance of much larger models at a fraction of the compute cost. 

[Paper Figure 2B] Each model's average performance plotted against its throughput (images per second on an L4 GPU). The RudolfV 2 models offer the best performance at each speed.

Getting Started with RudolfV 2

RudolfV 2, RudolfV 2-B, and RudolfV 2-S have been released for academic researchers to inspect, build upon and run for non-commercial use cases.

By releasing all three RudolfV 2 models openly, we hope to lower the barrier to reproducible work in computational pathology and give the community a stronger open foundation to build on. Weights and usage instructions are available on Hugging Face. For detailed methodology and full results, the complete paper is available for download here.

For commercial use cases, Atlas 2 is available. It was trained on 5.5 million whole slide images from Charité – Universitätsmedizin Berlin, LMU Munich, and Mayo Clinic, with roughly 2 billion parameters and clinical-grade regulatory documentation for integration into medical devices.