BiMAConv: Bimodal Adaptive Convolution for Multispectral Point Cloud Segmentation

Multispectral point cloud segmentation, leveraging both spatial and spectral information to classify individual points, is crucial for applications such as remote sensing, autonomous driving, and urban planning. However, existing methods primarily focus on spatial information and merge it with spectral data without fully considering their differences, limiting the effective use of spectral information. In this letter, we introduce a novel approach, bimodal adaptive convolution (BiMAConv), which fully exploits information from different modalities, based on the divide-and-conquer philosophy. Specifically, BiMAConv leverages the spectral features provided by the spectral information divergence (SID) and the weight information provided by the modal-weight block (MW-Block) module. The SID highlights slight differences in spectral information, providing detailed differential feature information. The MW-Block module utilizes an attention mechanism to combine generated features with the original point cloud, thereby generating weights to maintain learning balance sharply. In addition, we reconstruct a large-scale urban point cloud dataset GRSS_DFC_2018_3D based on dataset GRSS_DFC_2018 to advance the field of multispectral remote sensing point cloud, with a greater number of categories, more precise annotations, and registered multispectral channels. BiMAConv is fundamentally plug-and-play and supports different shared-multilayer perceptron (MLP) methods with almost no architectural changes. Extensive experiments on GRSS_DFC_2018_3D and Toronto-3D benchmarks demonstrate that our method significantly boosts the performance of popular detectors.