Innovating Space Applications with Deep Learning Benchmarking

Project Red Kyte advances machine learning for on-board data processing  

Complex space missions must increasingly rely on artificial intelligence and deep learning that can conduct in-orbit data processing and offer real-time awareness and decision-making. However, benchmarking machine learning algorithms for space applications remains challenging because the features and constraints inherent in each of the neural network models vary in relation to the different hardware and power configurations. In partnership with Canada-based Global Spatial Technology Solutions (GSTS) and with the support of the Canadian Space Agency, KP Labs has been tackling this emerging area for autonomous space systems and smart missions with Project Red Kyte.  

Project Red Kyte brought together an interdisciplinary team of scientists and engineers at KP Labs and GSTS to optimize the deployment, benchmarking, and performance of three deep learning algorithms specialized in space applications. By using real-use cases, the experimental results were published in the open-access journal Remote Sensing, showing how state-of-the-art machine learning techniques can be benchmarked, compared, and tested end-to-end before deployment.

Leopard DPU

The Need to Benchmark Deep Learning Models  

The transmission of detailed data from spacecraft to ground stations on Earth remains constrained by the transmission bandwidth and contact duration. Furthermore, the absence of on-board decision-making capabilities introduces significant latency in spacecraft operations and the delivery of valuable data. However, with the advances in scientific instrumentation that can capture more optical data and increase spacecraft autonomy, there is an urgent demand for robust AI-powered hardware and software that can operate in space’s extreme conditions.

To date, limited on-board computational processing power has been a bottleneck for the space community. Therefore, to advance this critical area of research, the Red Kyte Project team selected three deep learning models and conducted a series of experiments to process and send data more efficiently. Each selected neural network model, the Deep Earth, Deep Mars, and Deep Moon, provided a distinctive advantage for classifying, segmenting, and detecting objects.    

The Deep Earth model is a novel convolutional neural network pioneered by KP Labs for classifying hyperspectral imaging (HSI) in real-time. HSI captures and analyzes exquisitely detailed information about the scanned objects at multiple contiguous wavelengths across the visible light spectrum, making it an ideal candidate for advances in machine learning. Additionally, the classification and segmentation of hyperspectral data in orbit reduce the necessary bandwidth, so only the extracted value is broadcast back to Earth, minimizing data overload and maximizing the mission capabilities.  

KP Labs’ Deep Earth model is designed to be energy efficient by extracting valuable information from Earth observation satellites where the raw data is generated. For the experiment, the Deep Earth Model was trained and tested on a satellite image (340×610 pixels) from the University of Pavia, Italy, captured by a Reflective Optics System Imaging Spectrometer sensor. The team used a baseline of 87.2% accuracy for the model’s classification of objects spread across nine categories or classes, including meadows, trees, shrubs, and others.  

The classification and detection of Lunar craters have traditionally been conducted post-mission through human inspection. Therefore, in-orbit crater identification is becoming paramount, especially for the new classes of autonomous spacecraft aiming for long-term lunar exploration. Project Red Kyte leveraged the Deep Moon work to automatically identify the position and size of lunar craters from a digital elevation map (DEM). The DEM generated by NASA and the Japanese Aerospace Agency with the spatial resolution of 118 m per pixel used datasets from lunar campaigns, which were previously categorized by researchers into two catalogues: craters with a diameter between 5 and 20 km and others surpassing 20 km in size. The Red Kyte team adapted the original Deep Moon model and trained it to process the crater data as a two-step system: first, through image segmentation, followed by the pruning of false positives by using a template matching algorithm.

The Deep Mars model used in the Red Kyte project is also based on the creators of the Deep Moon model (Silbert et al.). and classifies objects on the Martian surface, including craters, dunes, slopes, streaks, and edges. The Deep Mars model can be adapted for studying other surfaces, which is particularly helpful for understanding the evolution of planetary landscapes and for autonomous spacecraft landings. The images used in the experiment with a spatial resolution of 30 cm per pixel were previously collected by the High-Resolution Imaging Science Experiment camera on-board NASA’s Mars Reconnaissance Orbiter. In addition, the data set had been manually reviewed and annotated through a crowdsourcing effort.  

All deep learning models in Project Red Kyte were deployed and tested on the elegant breadboard model (EBB) of KP Lab’s data processing unit – Leopard DPU – which is available for CubeSats starting at 6U. The Leopard DPU (EBB) Model is an affordable prototype tailored to jumpstart ground development, benchmarking, and testing. It is available for customers worldwide, including on the popular Satsearch online catalog. It is part of KP Lab’s Smart Mission Ecosystem – the portfolio of hardware, software, and AI-powered algorithms customized for demanding space missions.  

Smart Mission Ecosystem – hardware, software, and AI-powered algorithms customized for demanding space missions

Since 2019 Michał Gumiela has worked as a systems engineer at KP Labs. With a background in electrical and software embedded systems for the industry, Gumiela is part of the Red Kyte Project and paper co-author. Gumiela explained:  

“The traditional path we have seen in the field often starts with a machine learning or software model on a standard computer. But for deep learning on-board spacecraft, crucial steps, techniques, and tools must be compared across devices before deployment. You can develop sophisticated AI and ML models with plenty of servers, electricity, thermal regulation, and ideal conditions on the ground. But in-orbit, you have to manage with limited hardware, power, cosmic radiation, and many factors interfering with your models and data.”

The Red Kyte team benchmarked different scenarios for the three deep learning models in three test modes: offline mode, single-stream mode, and multi-stream mode. The main distinction was related to the set of conditions (volume and timing) delivering the inputs to the models. These test modes on the Leopard EBB allowed the researchers to measure and optimize their performance with a given set of parameters across different hardware and for separate use cases in Martian, Lunar, or Earth observation missions. The experiment used a combination of custom and commercial software platforms to deploy deep learning models with popular AI libraries and frameworks.  

Bringing the Future Closer  

Sophisticated spacecraft will require powerful in-orbit data processing that leverages deep learning models. Project Red Kyte’s experiments demonstrated there are still a variety of technical, scientific, and engineering questions waiting to be studied and resolved. However, a key to advancing AI-dedicated computing software and hardware is an internal and external collaboration with cutting-edge companies worldwide.  

“Very few companies and institutions specialize in deploying AI capabilities for satellites. We are well positioned at KP Labs for this portfolio of services because we have strong expert AI and hardware teams, so we seamlessly combine these two worlds for the constraints in space. Furthermore, we are very excited to partner with GSTS, bringing us closer to real-life applications of the technologies we create at KP Labs for space missions.” – Gumiela says.

GSTS, a leader in AI solutions for the maritime domain, selected and trained Project Red Kyte’s deep learning models and datasets. Additionally, they defined the testing and benchmarking scenarios, including the parameters for the experiments. The international collaboration between KP Labs and GSTS for Project Red Kyte demonstrates how AI innovation enhances the capabilities of space applications.

Founded in 2016, KP Labs is a European leader in autonomous space systems by delivering AI computers and software for demanding space missions. KP Labs applies autonomy in space missions to accelerate space exploration. Their product portfolio – Smart Mission Ecosystem was designed with the holistic approach to enable on-board data processing on the payload and satellite level, as well as to make the mission more fault-tolerant and safer. KP Labs was awarded the Research and Development Center status by the Ministry of Science and Higher Education in Poland. For more information, visit https://kplabs.space/.

GSTS is a leader in Artificial Intelligence solutions for the maritime domain. Our solutions are designed to save lives, energy, and the environment on a global scale through the use of innovative applications based on emerging data sets and analytics. We enable enhanced decision-based operations for civil, commercial and security agencies and industries. For more information, visit https://gsts.ca

GSTS was selected to develop space-based AI capability to support enhanced decision-making for a range of space applications. In effect, this AI capability will enable the “brain” to be near the “eyes” of any space asset. Applications focused on tasks using computer vision include robotics, exploration landers and Earth Observation systems. GSTS acknowledges the support of the Canadian Space Agency (CSA) [20STDPL06].

Written by: Monica Hernandez

‍