Sylvain Lobry, Université Paris Cité, France

Foundation models for remote sensing: which one(s)?

Recently, the first developments of foundation models for remote sensing imagery have appeared. These models, trained on large quantities of varied data and adaptable to many specific tasks, have the advantage of addressing a frequent problem in remote sensing: the difficulty of obtaining supervision data. Through a number of case studies at various geographical scales, we will highlight the different issues that arise when formulating models using remotely sensed images. We compare these issues with the first developments of foundation models, to identify new research directions.

Bio

Sylvain Lobry is an assistant professor in Computer Science since 2020. He is a researcher at the SIP team of the LIPADE laboratory and teaches at UFR de Mathématiques et Informatique in Université Paris Cité. He obtained his PhD in image processing from Télécom Paris in 2017. His research interests are in the areas of methodological developments in image processing with applications in particular to remote sensing imagery. This includes high-resolution optical images processing using deep learning techniques and change detection, classification and regularization on multi-temporal series of SAR images using Markov Random Fields models. During his PhD, he was working on the SWOT mission, dedicated to the study of the world’s oceans and its terrestrial surface waters. Since 2019, he also works on the interactions between remote sensing data and natural language. In particular, he introduced the task of Visual Question Answering for Remote Sensing (RSVQA).

Claudia Paris, University of Twente, the Netherlands

Bridging the Gap: Combining Satellite Images and Street-level Pictures to generate Comprehensive Reference Data for Land-Cover Mapping

Satellite data have been extensively used for mapping the Earth's surface. Given the increasing frequency of extreme weather events and climate change, the production of continuously updated land-cover maps is extremely relevant for tracking ongoing environmental changes. However, as we advance in both the availability of satellite data and computational capabilities, there is an opportunity—and a need—to move beyond traditional land-cover mapping, enabling real-time environmental monitoring and providing more detailed semantic information that can capture landscape complexity. For example, in agricultural monitoring, assessing the impact of extreme weather conditions on crop productivity is not limited to the identification of crop types, but also requires the ability to map crop phenological stages, since crops are more vulnerable to frost damage during and after flowering.
Achieving this requires a shift in field data collection: moving from conventional in-situ annotations to the use of street-level pictures. When combined with satellite data, street-level pictures offer the possibility to create dynamic, multi-view reference data, enriching the semantic context of information collected in the field to better understand environmental processes. However, the combination of satellite data and street-level pictures remains underexplored due to the challenges posed by their multimodal nature. Differences in resolution, scale, and georeferencing between ground and satellite images, make it challenging to align them for model training. Recent advances in Artificial Intelligence (AI), such as foundation models, visual-language models, and Agentic Reasoning AI, present new opportunities for combining these complementary data sources.
This talk will present the challenges and advantages of exploring the synergy between satellite images and street-level pictures to pave the way for enhanced environmental monitoring and more effective responses to climate challenges. It will also discuss operational strategies for integrating these data sources to create comprehensive reference datasets for land-cover mapping.

Bio

Claudia Paris is a Senior Assistant Professor (UD1) in the Faculty of Geoinformation and Earth Observation Sciences (ITC) at the University of Twente, Enschede, the Netherlands. She received the “Laurea” (B.S.), the “Laurea Specialistica” (M.S.) (summa cum laude) degrees in Telecommunication Engineering and the Ph.D. in Information and Communication Technology from the University of Trento, Italy, in 2010, 2012, 2016, respectively. She accomplished the Honors Master Program in Research within the Master’s Degree in Telecommunication Engineering in 2012. Claudia Paris' research encompasses image processing, signal processing, pattern recognition, machine learning, and deep learning, specifically applied to remote sensing image analysis. She focuses on designing innovative and automated workflows for the analysis and classification of large-scale Earth Observation (EO) data for various applications (e.g., forest/agricultural mapping and monitoring) by leveraging high-performance computing (HPC) and cloud computing platforms (Google Earth Engine). Her main research interests focus on the classification and fusion of multisource remote sensing data, multitemporal image analysis, domain adaptation methods, and land cover map updates. She has been conducting research on these topics in the framework of national and international projects. She is a member of the scientific and programme committee of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) and the SPIE International Symposium on Remote Conferences, respectively, and is also a referee for several international journals. Dr. Paris was twice the recipient of the prestigious Symposium Prize Paper Award (exceptional paper in terms of content and impact on the Geoscience & Remote Sensing Society) at the 2016 IEEE IGARSS (Beijing, China, 2016) and at the 2017 IEEE IGARSS (Fort Worth, TX, USA, 2017). She also won the IEEE Geoscience and Remote Sensing Society 2022 Letters Prize Paper Award (exceptional paper in terms of content and impact on the GRS-Society).