XoFTR: Cross-modal Feature Matching Transformer

Date

2024-06-16

Author

Tuzcuoğlu, Önder
Köksal, Aybora
Sofu, Buğra
Kalkan, Sinan
Alatan, Abdullah Aydın

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

19
views

0
downloads

Cite This

We introduce XoFTR a cross-modal cross-view method for local feature matching between thermal infrared (TIR) and visible images. Unlike visible images TIR images are less susceptible to adverse lighting and weather conditions but present difficulties in matching due to significant texture and intensity differences. Current hand-crafted and learning-based methods for visible-TIR matching fall short in handling viewpoint scale and texture diversities. To address this XoFTR incorporates masked image modeling pre-training and fine-tuning with pseudo-thermal image augmentation to handle the modality differences. Additionally we introduce a refined matching pipeline that adjusts for scale discrepancies and enhances match reliability through sub-pixel level refinement. To validate our approach we collect a comprehensive visible-thermal dataset and show that our method outperforms existing methods on many benchmarks. Code and dataset at https://github.com/OnderT/XoFTR.

URI

https://openaccess.thecvf.com/content/CVPR2024W/IMW/html/Tuzcuoglu_XoFTR_Cross-modal_Feature_Matching_Transformer_CVPRW_2024_paper.html
https://hdl.handle.net/11511/111742

Collections

Department of Electrical and Electronics Engineering, Conference / Seminar