Satellite image deep learning

In this episode I caught up with Samuel Bancroft to learn about segmenting field boundaries using Segment Anything, aka SAM. SAM is a foundational model for vision released by Meta, which is capable of zero shot segmentation. However there are many open questions about how to make use of SAM with remote sensing imagery.

In this conversation, Samuel describes how he used SAM to perform segmentation of field boundaries using Sentinel 2 imagery over the UK. His best results were obtained not by fine tuning SAM, but by carefully pre-processing a time series of images into HSV colour space, and using SAM without any modifications. This is a surprising result, and using this kind of approach significantly reduces the amount of work necessary to develop useful remote sensing applications utilising SAM. You can view the recording of this conversation on YouTube here

- Samuel on LinkedIn

- https://github.com/Spiruel/UKFields

Bio: Sam Bancroft is a final year PhD student at the University of Leeds. He is assessing future food production using satellite data and machine learning. This involves exploring new self- and semi- supervised deep learning approaches that help in producing more reliable and scalable crop type maps for major crops worldwide. He is a keen supporter in democratising access to models and datasets in Earth Observation and machine learning.

In this episode I caught up with Yotam Azriel to learn about interpretable deep learning. Deep learning models are often criticised for being "black box" due to their complex architectures and large number of parameters. Model interpretability is crucial as it enables stakeholders to make informed decisions based on insights into how predictions were made. I think this is an important topic and I learned a lot about the sophisticated techniques and engineering required to develop a platform for model interpretability. You can also view the video of this recording on YouTube.

* tensorleap.ai

* Yotam on Linkedin

Bio: Yotam is an expert in machine and deep learning, with ten years of experience in these fields. He has been involved in massive military and government development projects, as well as with startups. Yotam developed and led AI projects from research to production and he also acts as a professional consultant to companies developing AI. His expertise includes image and video recognition, NLP, algo-trading, and signal analysis. Yotam is an autodidact with strong leadership qualities and great communication skills.

In this episode I caught up with Daniele Rege Cambrin, to learn about Earthquake detection with Sentinel-1 (SAR) images. Daniele has a key role in organising a new competition on this task, SMAC: Seismic Monitoring and Analysis Challenge. The topics covered include the logistics of organising this competition, and the lessons Daniele learned from organising a previous one. You can also view the recording of this discussion on YouTube.

- Daniele on LinkedIn

- Competition website

Bio: Daniele Rege Cambrin is currently pursuing his Ph.D. and his research interests lie in deep learning. He is particularly interested in finding efficient and scalable solutions in areas such as remote sensing, computer vision, and natural language processing. Additionally, he has a keen interest in game development, and worked on two machine-learning competitions related to change detection.

In this episode Robin catches up with Inon Sharony to learn about the fascinating world of machine learning with SAR imagery. The unique attributes of SAR imagery, such as its intensity, phase, and polarisation, provide rich information for deep learning models to learn features from. The discussion covers the innovative applications ASTERRA is developing, and the nuances of machine learning with SAR imagery. This video of this episode is available on YouTube

* https://asterra.io/

* https://www.linkedin.com/in/inonsharony/

Bio: Inon Sharony is the Head of AI at ASTERRA, where he is responsible for pushing boundaries in the field of deep learning for earth observation. Sharony brings a decade of experience leading development of cutting-edge AI technology that meets real-world business and product needs. His previous roles include Algorithm Group Manager at Rail Vision Ltd and R&D Group Lead & Head of Automotive Intelligence at L4B Software. He was PhD trained in Chemical Physics at Tel Aviv University and combines his extensive academic background in Physics and his hands-on experience with machine learning to develop strategic AI solutions for ASTERRA.

In this episode, Robin catches up up with Alistair Francis and Mikolaj Czerkawski to learn about Major TOM, which is a significant new public dataset of Sentinel 2 imagery. Noteworthy for its immense size at 45 TB, Major TOM also introduces a set of standards for dataset filtering and integration with other datasets. Their aim in releasing this dataset is to foster a community-centred ecosystem of datasets, open to bias evaluation and adaptable to new domains and sensors. The potential of Major TOM to spur innovation in our field is truly exciting. Note you can also view the video of this recording on YouTube here. The video also includes a demonstration of accessing the dataset and a walkthrough of the associated Jupyter notebooks.

* Dataset on HuggingFace

* Paper

Alistair Francis is a Research Fellow at the European Space Agency’s Φ-lab in Frascati, Italy. Having studied for his PhD at the Mullard Space Science Laboratory, UCL, his research is focused on image analysis problems in remote sensing, using a variety of supervised, self-supervised and unsupervised approaches to tackle problems such as cloud masking, crater detection and land use mapping. Through this work, he has been involved in the creation of several public datasets for both Earth Observation and planetary science.

Mikolaj Czerkawski is a Research Fellow at the European Space Agency’s Φ-lab in Frascati, Italy. He received the B.Eng. degree in electronic and electrical engineering in 2019 from the University of Strathclyde in Glasgow, United Kingdom, and the Ph.D. degree in 2023 at the same university, specialising in applications of computer vision to Earth observation data. His research interests include image synthesis, generative models, and use cases involving restoration tasks of satellite imagery. Furthermore, he is a keen supporter and contributor to open-access and open-source models and datasets in the domain of AI and Earth observation.

In this video Robin catches up with Konstantin Klemmer to discus SatClip, which is a new global & general purpose location encoder trained on Sentinel 2 imagery. The conversation covered the training of encoders such as CLIP, and discussed the implications for downstream applications. Note you can also view the video of this recording on YouTube here

* Konstantin on LinkedIn

* SatCLIP

Bio: Konstantin is a postdoctoral researcher at Microsoft Research New England. His research interests lie broadly within geospatial machine learning and bridging adjacent domains like remote sensing or spatial statistics. Konstantin has a PhD from the University of Warwick and NYU, a Master's from Imperial College London and an undergraduate degree from the University of Freiburg, Germany.

In this episode Robin catches up with James Gallagher to learn about the latest AI innovations reshaping image annotation. The conversation covered significant new models such as Segment Anything, GroundingDINO and RemoteCLIP, and discussed how these models can be linked together to enable new annotation capabilities. Note you can also view the video of this recording on YouTube here

* James on LinkedIn

* Autodistill on Github

* Roboflow

Bio: James is a technical marketer at Roboflow, and has written over 100 guides on computer vision, covering areas from CLIP to dataset distillation and model evaluation. He also maintains several open source software packages at Roboflow, including Autodistill, a framework for auto-labelling images. In his free time, James has a unique hobby; he maintains a website that catalogues pianos available for public use in airports around the globe at airportpianos.org

A large fraction of acquired satellite images contain 2D projections of Earth. However, for many downstream applications, 3D understanding is beneficial or necessary. In recent years, deep learning has enabled a number of solutions for learning 3D representations from 2D satellite images.

This episode delivers an overview of some of the prominent works in this area. Mikolaj hosts 3 guests: Dawa Derksen, Roger Marí, and Yujiao Shi, providing a summary of each guest’s contributions on the topic as well as a panel discussion. Note you can also view the video of this recording on YouTube here

Dawa Derksen - Origins of Shadow-NeRF

Dawa pursued a post-doctoral research fellowship at the European Space Agency from 2020-2022, and is currently working at the Centre National d’Etudes Spatiales (French Space Agency) where he is involved in the field of 3D Implicit Representation Learning applied to Remote Sensing.

* 🖥️ Shadow-NeRF

Roger Marí - EO-NeRF

Roger is a post-doc researcher from Barcelona specialised in 3D vision tasks. He is currently working at the Centre Borelli, ENS Paris-Saclay, in France, where his research topic is the application of neural rendering methods to satellite image collections. He is the author of Sat-NeRF and EO-NeRF, some of the first models in the literature to provide quantitatively convincing results in terms of surface reconstruction.

* 🖥️ https://rogermm14.github.io/

* 🖥️ EO-NeRF

Yujiao Shi - Connecting Satellite Image with StreetView

Yujiao is a research fellow at the Australian National University. She obtained her PhD degree at the same institute. Her research interests include satellite image-based localisation, cross-view synthesis, 3D vision-related tasks, and self-supervised learning.

* 🖥️ https://shiyujiao.github.io/

* 📖 Geometry-Guided Street-View Panorama Synthesis from Satellite Imagery

Host & Production: Mikolaj Czerkawski

https://mikonvergence.github.io

In this episode Robin catches up with Jake Wilkins to learn about Deep learning in Google Earth Engine. Jake has been building commercial Earth Engine applications for the past three years and in this conversation he describes the pros and cons of several approaches to using deep learning models with Earth Engine. Note you can also view the video of this recording on YouTube here

* Jake on LinkedIn

* https://earthengine.google.com/

Bio: Jake is a Software Engineer and Data Scientist based in London, UK. He has been building commercial Google Earth Engine applications for the past three years. His significant contributions include the no-code platform, Earth Blox, and the climate monitoring platform STRATA for UNEP (United Nations Environmental Programme). Alongside this, Jake has consistently developed his skills in machine learning, and a notable accomplishment in this field is winning the Earth-i hackathon last year. Jake has a deep passion for addressing the climate crisis and is committed to making Earth Observation more accessible to combat it.

In this episode Robin catches up with Roberto Del Prete to learn about PyRaws. PyRaws is a powerful open source Python package that provides a comprehensive set of tools for working with Sentinel 2 raw imagery. It provides tools for band coregistration, geo-referencing, data visualisation, and image processing. What is particularly exciting is that this software could be deployed onto future satellites, enabling on-board processing using python. Note you can also view the video of this recording on YouTube here

* https://github.com/ESA-PhiLab/PyRawS

* https://www.linkedin.com/in/roberto-del-prete-8175a7147/

Bio: Roberto Del Prete is a PhD candidate focused on expanding the uptake of Deep Learning for enhancing the applications of onboard edge computing. His aim is to improve decision-making in time-critical scenarios by reducing the time lag required to process and deliver useful information to the ground. He is also working on developing autonomous spacecraft navigation systems using onboard instruments like cameras. Through his research he wants to contribute to the advancement of AI technology and its real-world applications, pushing the boundaries of what is possible to accomplish onboard.