MuM: Multi-View Masked Image Modeling for 3D Vision

Figure 1. MuM extends masked autoencoders to arbitrarily many views, learning features tailored for 3D vision that outperform DINOv3 and CroCo v2 on feedforward reconstruction, dense matching, and relative pose estimation.

Abstract

Self-supervised learning on images seeks to extract meaningful visual representations from unlabeled data. When scaled to large datasets, this paradigm has achieved state-of-the-art performance and the resulting trained models such as DINOv3 have seen widespread adoption. However, most prior efforts are optimized for semantic understanding rather than geometric reasoning. One important exception is Cross-View Completion, CroCo, which is a form of masked autoencoding (MAE) tailored for 3D understanding. In this work, we continue on the path proposed by CroCo and focus on learning features tailored for 3D vision. In a nutshell, we extend MAE to arbitrarily many views of the same scene. By uniformly masking all views and employing a lightweight decoder with inter-frame attention, our approach is inherently simpler and more scalable than CroCo. We evaluate the resulting model, MuM, extensively on downstream tasks including feedforward reconstruction, dense image matching and relative pose estimation, finding that it outperforms the state-of-the-art visual encoders DINOv3 and CroCo v2. Code is available at github.com/davnords/mum.

Results

Figure 2. Training dynamics.

The MuM objective is trivially scalable

BibTeX

@inproceedings{nordstrom2026mum,
  title={MuM: Multi-View Masked Image Modeling for 3D Vision}, 
  author={David Nordström and Johan Edstedt and Fredrik Kahl and Georg Bökman},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}