Boosting In-Orbit Cloud Detection with AI

Cloudy coverage. (Credit: ESA)

KP Labs supports ESA Φ-lab mission with novel application  

KP Labs advances the next generation of Earth observation (EO) satellites with artificial intelligence and autonomous software. Deep learning models can process Earth’s frequent cloudy coverage onboard these satellites. The team has been developing a novel in-orbit Cloud Detection Application for the European Space Agency’s Φ-sat-2 mission powered by deep learning. With this mission, KP Labs aims to demonstrate how the in-orbit cloud detection capability can be used as a pre-processing service in tandem with a suite of other applications or in a standalone mode for any Earth observation (EO) satellite. KP Labs is one of the seven industry partners supported by ESA Φ-lab to create the breakthrough AI applications on board the 6U CubeSat designed and developed by Open Cosmos Ltd.

Cloudy Skies, Cloudy Data  

Routine cloud coverage of Earth’s atmosphere has been a frequent bottleneck for image processing. Clouds restrict the visibility of the target areas, so a significant percentage of the images captured are rendered useless due to low visibility or low quality. Furthermore, the excess images captured in orbit often overload onboard computers, storage units, and energy sources seeking to compress and process large raw data sets for the downlink broadcast. Robust machine learning (ML) models that accelerate and automate the extraction of useful data are becoming essential for downlink broadcast and analysis, significantly pruning the images covered by clouds with poor visibility of the target area. Likewise, automating a satellite’s onboard database with quality metrics and top images can be a helpful prioritization resource for missions with suites of AI applications. Typically, these applications process cloud coverage independently or repeat the capturing process until a clear image is obtained. Autonomy can support rescheduling, recapturing, and reprocessing critical areas of interest quickly and efficiently.

Detecting top-quality, detailed images of clouds and distinguishing their features raises the urgency for increased satellite autonomy. Cloud coverage is becoming an important research area in climate and atmospheric sciences, especially in understanding and mitigating the impacts of natural disasters, volcanic eruptions, and severe weather. However, the AI-based applications and ML models typically used in the industry require clear, pre-processed image data without cloudy coverage. So training and benchmarking machine learning models on the ground has proved particularly challenging for the industry, especially training models to distinguish between snowy and cloudy areas when managing first-time, in-orbit scenarios.

Cloudy coverage during the Cumbre Vieja volcano eruption, La Palma, Spain. (Credit: ESA, Copernicus Sentinel data [2021]), CC BY-SA 3.0 IGO).

Innovating Image Processing Chains

KP Labs has been developing robust expertise in the latest satellite image processing models to differentiate clouds and natural features on Earth and other celestial objects. As reported by The Quantum Insider, previous research collaborations included studying cloudy coverage with quantum algorithms. Project Red Kyte leverages the deployment, benchmarking, and performance of three deep learning algorithms specialized in space applications for Earth, Mars, and Lunar observations.

The recent research and development at KP Labs in 2021 and 2022 tested the performance of convolutional neural networks, a deep learning model that extracts, applies, and generalizes previous learnings to new or unseen target data. Using big data sets from the United States Landsat-8 and ESA’s Copernicus Sentinel-2 satellites, KP Labs experts showed how deep learning models could be trained with both original and simulated test images, including large samples with snowy patches.

Technical diagram illustrating how deep learning networks were trained. (Credit: KP Labs)

The Head of Artificial Intelligence at KP Labs, Jakub Nalepa, Ph.D., D.Sc., leads the team of seven developing the Cloud Detection Application for the Φ-sat-2Phi-Sat 2 mission. Nalepa is excited about contributing to ESA’s AI Earth observation mission.

“Onboard cloud detection is an essential pre-processing step because it reduces the volume of useless data so that other AI-powered applications can prioritize the best images for analysis based on this output. The pre-processing capability is a type of smart compression where automatic selection adds value to other mission areas,” explained Nalepa.

Nalepa is also an associate professor of computer science at the Silesian University of Technology. He frequently publishes in the scientific literature about deep learning, pattern recognition, and computer vision. A common thread in work is how to use big data with increased machine autonomy and intelligence. The number of satellites increases exponentially, and so does the volume and complexity of data requiring sophisticated processing.

“Nowadays, better sensors and sophisticated instrumentation onboard the satellite capture complex imagery. Still, we must process this raw data more efficiently to extract useful information onboard while keeping the state-of-the-art software compact and lightweight. At KP Labs, we want to bring the machine’s processing brain closer to the eyes.” said Nalepa.

KP Labs has trained deep learning models to work with post-processing algorithms that classify and determine cloud data. This post-processing generates baseline statistics for the mission, such as the percentage of clouded pixels or how the clouds are distributed. Baseline statistics can serve as quality criteria or metrics that can help the suite of onboard AI applications reduce misclassification or misdetection.

As a bridge between academia and industry, Nalepa believes there is a virtuous exchange of ideas between the new generations of students interested in machine autonomy and the new AI capabilities advanced at KP Labs.

“This mission is exciting at the data and execution levels because the extremes of space constrain us in ways that on-ground deployment doesn’t. But, more importantly, I’m excited about the potential impact. If we can observe Earth better, we can understand what affects our home planet,” concluded Nalepa.

KP Labs is on target, progressing with the milestones and technology readiness for Φ-sat-2 Phi-Sat-2’s In-Orbit Cloud Detection Application.

About us

Founded in 2016, KP Labs is a European leader in autonomous space systems by delivering AI computers and software for demanding space missions. KP Labs applies autonomy in space missions to accelerate space exploration. Their product portfolio – Smart Mission Ecosystem was designed with the holistic approach to enable on-board data processing on the payload and satellite level, as well as to make the mission more fault-tolerant and safer. KP Labs was awarded the Research and Development Center status by the Ministry of Science and Higher Education in Poland.

Written by: Monica Hernandez

‍