Authors: @Balamurugan Balakreshnan @Swati Ghare @radu kopetz @Shruti Harish
Background
Constellation (Nasdaq: CEG) is the nation’s largest producer of carbon-free energy and a leading competitive retail supplier of power and energy products and services for homes and businesses across the United States. In March 2022 Constellation announced a sustainability partnership with Microsoft in which Microsoft will develop software to allow Constellation’s customers to match their energy use with local carbon-free energy sources, 24 hours a day, 365 days a year.
Constellation Clearsight is a wholly owned subsidiary of Constellation that provides safer, more effective inspections of critical infrastructure by using cutting-edge drone, robot, sensor, and software technologies to deliver key insights that reduce the total cost of asset ownership. Their mission is to leverage innovation and technology to improve inspection services for critical infrastructure and help drive progress towards a sustainable, safer, and more reliable future.
Motivation
Visual inspection of electrical system distribution infrastructure has historically been a mostly manual process. These inspections are difficult to perform in certain environments and at desired levels of cost effectiveness and scale. The use of technology to aid the acquisition, storage, and analysis of data can greatly improve inspection safety, efficiency, and quality. Constellation Clearsight (Clearsight) is finding Azure and Azure Machine Learning to be effective platforms upon which to build and enable several key technology solutions.
Prior to using Azure Machine Learning, the ML model development process was handled using a collection of different tools that were not well integrated. Building these ML models and tuning them for optimal performance took significant data scientist time and effort. It was recognized that a collaboration environment to manage datasets, train use case specific models and build test pipelines without writing custom code would enable a much more scalable approach.
Azure Machine Learning provided Clearsight with this unified platform. They made extensive use of Azure ML Data Labeling to label the images and leverage NVIDIA GPU compute that could be scaled as needed to develop their computer vision models. AutoML for Images allowed them to quickly experiment with multiple algorithms and find the model algorithms and hyperparameters that optimized model performance for their different use cases. The tight integration with MLOps allows them to manage the end-to-end ML life cycle of these models.
Approach
Clearsight uses cutting edge drone technology to support visual inspection of electrical infrastructure equipment. The drones obtain high resolution imagery from various offset distances and at multiple angles and perspectives, all while keeping the drone operators in a safe location. Once obtained, the imagery is uploaded to Azure storage. The images are then presented to subject matter experts for image inspection and observation logging via custom-built applications. Image attributes and related attributes identified through inspection activities are also recorded relationally in Azure storage.
Azure Machine Learning end to end workflow
Clearsight is leveraging Azure Machine Learning, including AutoML for Images, to build and deploy computer vision models (i.e., object and defect detectors) to enhance and support the inspection activities. These models are trained on datasets comprised of images and bounding boxes tagged with corresponding classes and labels that define objects and defects of note. Using AutoML for Image capabilities, Clearsight can easily try a variety of algorithms and hyperparameters for training these models (including Yolo, RetinaNet and Faster-RCNN), and subsequently choose the best performing one.
The custom-built inspection applications and various other platform components leverage the computer vision models through deployed endpoints on Azure Kubernetes Service (AKS) and other Azure components. Continual refinement of the models is enabled by the inspection workflow and the associated feedback loop provided by the subject matter experts.
Benefits
New capabilities in Azure Machine Learning that add support for computer vision tasks in AutoML have proven to be beneficial to Clearsight’s use cases. The AutoML solution was evaluated in comparison to the previous process used for model development. They found the end-to-end solution using AutoML to be more efficient and generally easier to use. The resulting models were able to achieve performance similar to those from the legacy system, while driving significant efficiency in the model building process. Some specific features and updates of value to Clearsight include:
- The ability to use both Azure Machine Learning Labeling for new data and to bring in datasets of images that were labeled outside of Azure
- The ability to manually select from a range of state-of-the-art algorithms
- The ability to access not only full models, but also access the scripts needed to automate the process and productionize the models
- The level of control and customization available in support of model training and management via Machine Learning Operations (MLOps)
These feature capabilities in Azure Machine Learning Data Labeling and AutoML for Images have significantly reduced Clearsight’s end to end cycle time required to build and deploy new computer vision models by an estimate of more than 50%.
You can find out more about Constellation Clearsight at www.constellationclearsight.com or by following them on LinkedIn.
Specific contacts at Constellation Clearsight include:
- Teague Maxfield, Lead Solution Delivery Architect (teague.maxfield@constellation.com)
- Alex Harvey, Solution Delivery Architect (alexander.harvey@constellation.com)
- Fan Yang, Principle Tech Innovation Specialist (fan.yang@constellation.com)
About Constellation Clearsight
Constellation Energy Corporation (Nasdaq: CEG) is the nation’s largest producer of clean, carbon-free energy and a leading supplier of energy products and services to millions of homes, institutional customers, the public sector, community aggregations and businesses, including three fourths of Fortune 100 companies. A Fortune 200 company headquartered in Baltimore, our fleet of nuclear, hydro, wind and solar facilities has the generating capacity to power more than 20 million homes, providing 10 percent of all carbon-free energy on the grid in the U.S. Our fleet is helping to accelerate the nation’s transition to clean energy with more than 32,400 megawatts of capacity and annual output that is 90 percent carbon-free. We have a goal to achieve 100 percent carbon-free power generation by 2040 by leveraging innovative technology and enhancing our diverse mix of hydro, wind and solar resources paired with the nation’s largest nuclear fleet. Follow Constellation on Twitter @ConstellationEG.
Thanks:
@vijai Kannan
@Shu Peng
@Mi Sun Park
@Malek el Khazen
@Geoff Noth