Transformer-Based Encoders Significantly Improve Remote Sensing Image Captioning: A Systematic Encoder Ablation Study

OpenAlex · English

Remote sensing image captioning (RSIC) requires joint understanding of complex visual scenes and the generation of coherent natural-language descriptions. In this study, we investigate the influence of different visual representation models on caption generation performance inside a unified transformer-based captioning framework. Seven encoder backbones representing three major paradigms—convolutional neural networks, multimodal contrastive-pretrained models, and hierarchical vision transformers—are evaluated under an identical training protocol and a fixed decoder configuration. Experiments are conducted on three widely used benchmark datasets: RSICD, UCM Captions, and Sydney Captions. The results reveal a consistent performance trend across datasets. Conventional CNN encoders provide strong baseline performance, while multimodal pretraining improves semantic alignment in complex aerial scenes. Hierarchical transformer-based encoders achieve the best overall results, primarily due to their capability to capture multi-scale spatial structures and long-range dependencies in remote sensing imagery. In particular, the Swin Transformer consistently achieves the highest CIDEr scores across all datasets, reaching 2.814 on RSICD compared to 2.598 obtained by the ResNet50 baseline, representing an absolute improvement of$\text{+ 0. 2 1 6}$CIDEr. These outcomes indicate that architectural design has a greater impact on captioning performance than simply increasing model depth within the same model family. Overall, this study provides a systematic empirical reference for selecting encoder architectures in remote sensing image captioning research.