HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Data
Stella Girtsou, Konstantinos Alexis, Giorgos Giannopoulos, Charalambos Kontoes
Why It Matters
What makes this one worth your time
This research addresses the urgent need for timely data in disaster management, potentially improving response strategies and outcomes in climate-related emergencies.
HighFM leverages high-frequency satellite data for improved disaster monitoring and response.
Summary
The paper introduces HighFM, a foundation model designed to learn representations from high-frequency Earth Observation data, specifically utilizing SEVIRI imagery to enhance real-time monitoring of climate-related disasters.
Key contributions
- Development of HighFM, a foundation model for high temporal resolution multispectral Earth Observation data.
- Benchmarking of SEVIRI pretrained Vision Transformers against traditional baselines and recent geospatial foundation models.
- Demonstration of consistent performance gains in cloud masking and active fire detection tasks.
Notable insights
- The adaptation of the SatMAE framework for high temporal resolution data is a novel approach that could influence future models in Earth Observation.
- Incorporating fine-grained temporal encodings to capture short-term variability is a significant enhancement for real-time applications.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2604.04306v2 Announce Type: replace-cross Abstract: The increasing frequency and severity of climate related disasters have intensified the need for real time monitoring, early warning, and informed decision-making. Earth Observation (EO), powered by satellite data and Machine Learning (ML), offers powerful tools to meet these challenges. Foundation Models (FMs) have revolutionized EO ML by enabling general-purpose pretraining on large scale remote sensing datasets. However most existing models rely on high-resolution satellite imagery with low revisit rates limiting their suitability for fast-evolving phenomena and time critical emergency response. In this work, we present HighFM, a first cut approach towards a FM for high temporal resolution, multispectral EO data. Leveraging over 2 TB of SEVIRI imagery from the Meteosat Second Generation (MSG) platform, we adapt the SatMAE masked autoencoding framework to learn robust spatiotemporal representations. To support real time monitoring, we enhance the original architecture with fine grained temporal encodings to capture short term variability. The pretrained models are then finetuned on cloud masking and active fire detection tasks. We benchmark our SEVIRI pretrained Vision Transformers against traditional baselines and recent geospatial FMs, demonstrating consistent gains across both balanced accuracy and IoU metrics. Our results highlight the potential of temporally dense geostationary data for real-time EO, offering a scalable path toward foundation models for disaster detection and tracking.