Two streams ResNet-50 network for infrared and visible image fusion
摘要
In this work, we put forward a unique and first attempt of infrared and visible image fusion using frequency-domain features induced with a two-stream ResNet-50 network. In the proposed scheme, both the visual and the thermal images are geometrically transformed by NSCT to get a shift-invariant, multi-direction, and multi-scale decomposition output. Two streams of parallel ResNet-50 networks: one for the low-frequency and another for the high-frequency components of the NSCT are used here. A weighted combination strategy is proposed here to fuse the information from both visual and thermal image features output using the spatial inter-dependency among the pixels and precisely retain the correlative details from both the source images. The proposed fusion strategy propagates lesser artifacts with rich edge details into the fused image. Different experiments were carried out on the “TNO" benchmark database to estimate the efficacy of the proposed algorithm. The empirical results of the proposed scheme are verified both qualitatively and quantitatively and found to be providing higher accuracy against the existing ten state-of-the-art (SOTA) techniques. It is also observed that the proposed algorithm surpasses the competitive SOTA techniques in terms of different considered quantitative evaluation measures with at least a gain of 0.07% and the highest gain of 99.42%. Further, the proposed technique gives promising results in terms of mean with a standard deviation value of