SM4RT: Learning Structured Motion Geometry for 4D Reconstruction

Most monocular motion-perception systems treat every pixel's movement as an independent displacement, ignoring that real objects move as rigid bodies. SM4RT challenges this by encoding scene dynamics as a small set of shared "motion bases," each a temporal sequence of 6D twists in SE(3), so points belonging to the same object inherit a common rigid trajectory. A parallel encoder-decoder recovers 3D geometry and world-coordinate motion from a single RGB video in one pass. Potential applications include robotics perception, autonomous driving scene understanding, AR/VR content creation, and any pipeline needing physically consistent 4D reconstructions from ordinary video. Authors: Shing Ho J. Lin, Wenzhao Zheng, Dong Zhuo, Yuqi Wu, Jie Zhou, Jiwen Lu Paper: https://arxiv.org/abs/2607.22534v1