Dictionary Expansion for Incompletely Overlapped Multi- and Hyperspectral Image Fusion

High-spatial-resolution multispectral (HM) and low-spatial-resolution hyperspectral (LH) image fusion over the same scene has been intensively studied. However, in practical application scenarios, the HM image usually covers a larger area than the LH image; thus, the same scene-oriented fusion framework can only be used to a limited range of the overlapping area. To solve this problem, a dictionary expansion-based incompletely overlapped HM and LH fusion method is proposed here, which tries to expand the dictionary learned in the overlapping area to the other nonoverlapping area and then to reconstruct an entire high-spatial-resolution hyperspectral (HH) image over the area covered by the whole HM image. Specifically, the above incomplete fusion problem is divided into one fusion problem in the overlapping area and one reconstruction problem over the nonoverlapping area separately, where the former is formulated as a traditional spectral dictionary learning-based fusion process and the latter is formulated as a new dictionary expansion-based reconstruction process only using the HM image in the framework of sparse and low-rank representation. To ensure the dictionary learned in overlapping areas could be precisely expanded to nonoverlapping areas, a strategy for extracting universal spectra is proposed by using the spectral similarity between overlapping and nonoverlapping areas. The expanded spectral dictionary is calculated using the spectral information from the LH image and the co-constraints applied to the representation error in both overlapping and nonoverlapping areas, and the corresponding sparse coefficient matrix is calculated using the spatial information from the HM image and a spectral similarity constraint between them. Experimental results on simulated and real datasets with different coverage areas indicate that our proposed method gives better performance in fusing incompletely overlapped multispectral and hyperspectral images, compared with other related or tangentially related state-of-the-art methods.