Without any doubt, machine learning (ML) is one of the most transformative technological areas these days. Affordable cloud-based compute, complex and more efficient algorithms, plus accessible big data volumes helped to make incredible scientific breakthroughs in various fields and achieve (or even exceed) human parity with computer vision (CV) and natural language processing (NLP).
Quality and variety of data is crucial to the machine learning’s generalisation and reliability of its outcomes. At the same time, there are certain situations where data may be rare or hard to find. E.g., NASA Jet Propulsion Laboratory's Center for Near Earth Object (NEO) Studies has been continuously tracking and cataloguing NEOs in its discovery statistics database since 1998. At the time of writing, on 12th of April 2023, the total number of known NEOs reached 31,758. So, would you consider it to be sufficient to train an ML model for a proper asteroid detection and monitoring?
Well, in October 2017 the University of Hawaii’s Pan-STARRS1 telescope detected quite a strange object that was named `Oumuamua (“a messenger from afar arriving first” in Hawaiian). This asteroid was unlike any other space objects seen by the Earth telescopes in our solar system before. `Oumuamua was 400 m long and had a length-to-width ratio of 10:1 (most NEOs in NASA’s catalogue have a ratio of 3:1). It was travelling at a speed of about 87 km/s and while spinning on its axis every 7 hours, `Oumuamua’s brightness was changing by a factor of 10 - something that was never seen with any other known NEO. So, nearly 20 years of collected asteroid and comet monitoring statistics still left astronomers baffled by this new asteroid’s characteristics. There was even an article published by the Harvard researcher on the possibility of `Oumuamua’s artificial origin. 
We don’t know how many other strange and unknown space objects are lurking somewhere out there. And this may be one of those research areas where automatic generation of synthetic asteroid images with various (even unimaginable) shapes and forms can greatly contribute to and improve accuracy and performance of asteroid detection and monitoring solutions.
Synthetic asteroids
I started to explore the idea of using synthetic images for a potential asteroid hunting in 2021 as a part of the Microsoft Global Hackathon.
Back then, I built an asteroid using Blender 3D open source application. However, as a prototype it was still very raw and lacked any scalability. Because of some other priorities at the time, the project was shelved. 
Great opportunity to revive it came in 2022, when I got a chance to discuss this idea with Pedro Urbina and Shannon Monroe, who not only encouraged me to develop a synthetic asteroid using their platform, but also actively supported throughout the process.
I was introduced to a team of talented artists, got access to onboarding materials and provided with software licenses to some of the best 3D modelling solutions. The learning curve was quite steep and some aspects of procedural design were sometimes challenging. But whenever I needed any assistance, I could always reach out to the in-house experts: Jon Hanzelka and Luke Tiday, who were kind enough to share their time and knowledge, and help with any problematic theoretical or practical aspects.
Importance of asteroid detection
More recent and documented Tunguska event happened on 30th of June, 1908 when an unknown space object exploded near the Podkamennaya Tunguska River in Siberia. Some scientists believe that the event was caused by a 50–100 metre-wide asteroid. Its explosion flattened about 2,000 square km of remote pine forest. If a similar asteroid exploded above London, it would have killed millions by destroying everything within the London Orbital Motorway, M25.
So, asteroid detection and tracking is of concern and on agenda of many big organisations. United Nations endorsed setup of the International Asteroid Warning Network (IAWN) back in 2013 to coordinate global efforts in this space. NASA - National Aeronautics and Space Administration’s Planetary Defense Coordination Office (PDCO) in cooperation with European Space Agency - ESA has conducted several tests since 2017 to check humankind’s operational readiness to detect hazardous asteroids, assess their potential risks and take preventive measures, and so forth.
From 2021’s joint exercise between NASA and ESA, it was found that with the current level of our technological readiness we would be defenseless if a hypothetical asteroid was detected 6 months or less before hitting Earth. That’s why even more efforts are now being put into detection of potentially hazardous asteroids: from existing NASA’s space observatory NEOWISE, to DART programme on the potential asteroid trajectory’s alteration and a planned launch of a more advanced asteroid detector, the NEO Surveyor, in 2028.
Practical implementation
After generating synthetic asteroid data and training a computer vision model, we need to deliver it somehow into an upper space. Here we can combine the magic of synthetic data with the power of two other solutions:
- In 2021 as a collaboration between NASA, Hewlett Packard Enterprise (HPE) and Microsoft, a commercial off-the-shelf supercomputer, Spaceborne Computer-2 (SBC-2), was deployed to the International Space Station (ISS). SBC-2 was designed to deliver Edge AI computing capabilities to ISS and offload more intensive computations to Azure backend on the ground;
- In February 2023 we heard about the release of Microsoft’s open-source KubeAI Application Nucleus (KAN) project. KAN can simplify development and operation of Edge AI solutions with an intuitive no-code interface, something that we can leverage during early stage prototyping.
To demonstrate the solution capability, I’ve trained a new ML model in Azure Custom Vison with a synthetic asteroids dataset, tested it with the asteroid 2004 EW95’s fly-by simulation from the European Southern Observatory and then exported machine learning’s outcomes in a video format.
On this YouTube video you can see the model in action, where it was able to detect and trace an asteroid in a simulated camera’s video stream. Copy of a trained model (in the TensorFlow format) along with the custom inference code can be found on this GitHub repo.
Closing remarks
Big thanks again to Pedro, Shannon, Jon and Luke for all their help and support with making this prototype development possible. It was a massive learning opportunity and a chance to apply new skills to something really interesting.
Artificial intelligence / machine learning is a transformative technology, that requires big and diverse data volumes for a better accuracy and performance. So, synthetic data can help us with a better representation of the real world and push our boundaries of unknown. One day it may also save somebody's life.