Perspective n-view multibody structure-and-motion through model selection

Abstract

Multi-body structure-and-motion (MSaM) is the problem to establish the multiple-view geometry of an image sequence of a 3D scene, where the scene consists of multiple rigid objects moving relative to each other. So far, solutions have been proposed for several restricted settings, such as only two views, affine projection, and perspective projection of linearly moving points. We give a solution for sequences of several images, full perspective projection, and general rigid motion. It can deal with the fact that the set of correspondences changes over time, and is robust to outliers. The proposed solution is based on Monte-Carlo sampling and clustering of two-view motions, linking them through the sequence, and model selection to yield the best explanation for the entire sequence.