Deep Clustering of Remote Sensing Scenes through Heterogeneous Transfer Learning

ABSTRACT Satellite and aerial imagery is collected at a dizzying rate, but how to best distill this information is often unknown. Classifying images is the most popular approach but requires specifying groups a priori. This article introduces a method for fully unsupervised whole-image clustering, specifically designed for massive datasets of remote-sensing scenes with no labels. Our approach is tailored to the unique challenges of this domain and offers several key advantages: (1) We fine-tune a pretrained deep neural network (DINOv2) on a labelled source satellite imagery dataset, enabling broad applicability across diverse remote-sensing imagery with varying ground sample distances (GSDs), resolutions and image sizes. This fine-tuned model can effectively extract meaningful feature vectors from unseen remote-sensing datasets with minimal further adaptation, making it both scalable and robust for large-scale remote-sensing tasks. (2) We reduce the dimensionality of these deep features through manifold projection, mapping them into a low-dimensional Euclidean space to streamline downstream processing and enhance computational efficiency. (3) These low-dimensional features are then clustered using a computationally scalable Bayesian nonparametric (BNP) technique, which automatically infers both the number of clusters and their membership. Our extensive evaluation on the challenging SATellite ImageNet (SATIN) Land Use task, a benchmark that includes 10 overhead imagery datasets, highlights the robustness of our method in the remote-sensing domain, outperforming state-of-the-art zero-shot classification techniques on multiple datasets. Additionally, we provide a proof-of-concept demonstration on a leading computer vision benchmark, which showcases the competitiveness of our unsupervised approach against even supervised techniques. Our approach is highly relevant for future geospatial analysis since it can be further fine-tuned to specific downstream tasks, including those that incorporate data modalities beyond the visible spectrum, without the need for significant modifications.