We introduce Lifting By Gaussians (LBG), a novel approach for open-world instance segmentation of 3D Gaussian Splatted Radiance Fields (3DGS). Recently, 3DGS Fields have emerged as a highly efficient and explicit alternative to Neural Field-based methods for high-quality Novel View Synthesis. Our 3D instance segmentation method directly lifts 2D segmentation masks from SAM (alternately FastSAM, etc.), together with features from CLIP and DINOv2, directly fusing them onto 3DGS (or similar Gaussian radiance fields such as 2DGS). Unlike previous approaches, LBG requires no per-scene training, allowing it to operate seamlessly on any existing 3DGS reconstruction. Our approach is not only an order of magnitude faster and simpler than existing approaches; it is also highly modular, enabling 3D semantic segmentation of existing 3DGS fields without requiring a specific parametrization of the 3D Gaussians. Furthermore, our technique achieves superior semantic segmentation for 2D semantic novel view synthesis and 3D asset extraction results while maintaining flexibility and efficiency. We further introduce a novel approach to evaluate individually segmented 3D assets from 3D radiance field segmentation methods.
我们提出了 Lifting By Gaussians (LBG)，一种用于 3D 高斯 Splatting 辐射场（3DGS）开放世界实例分割的新方法。近年来，3DGS 辐射场作为一种高效且显式的替代方案，已成为基于神经场的方法进行高质量新视角合成的有力竞争者。我们的 3D 实例分割方法直接从 SAM（或其他如 FastSAM 等）中提取 2D 分割掩模，并结合来自 CLIP 和 DINOv2 的特征，直接将它们融合到 3DGS（或类似的高斯辐射场如 2DGS）中。与之前的方法不同，LBG 不需要针对每个场景进行训练，从而能够无缝地在任何现有的 3DGS 重建上运行。我们的方法不仅比现有方法速度快、实现更简单，而且具有高度模块化的特点，使得在无需对 3D 高斯进行特定参数化的情况下，能够实现现有 3DGS 辐射场的 3D 语义分割。此外，我们的技术在 2D 语义新视角合成和 3D 资产提取结果中实现了优越的语义分割效果，同时保持了灵活性和高效性。我们还提出了一种新的方法，用于评估从 3D 辐射场分割方法中单独分割出的 3D 资产。