Overview of SR-CNN algorithm in Azure Anomaly Detector
Author: Tony Xing (@XingGuodong), AI Platform, C + AI In the last blog “Introducing Azure Anomaly Detector API”, I didn't provide enough details on one of the algorithms. As the algorithm paper was in the publishing process. The paper was accepted by KDD 2019 for oral presentation later, and this blog serves as an overview of the SR-CNN algorithm and for more details user can always read the paper. By the way, we have a 2-minute video here. Problem definition Before we go into details, let us revisit the problem definition of time series anomaly detection. Challenges For any time-series anomaly detection system that is operating in production with a large scale, there are quite a few challenges, especially on the three areas below: 1. Lack of labels - As you can imagine, with signals generated from clients, services, and sensors every second, the huge amount of volume makes it infeasible to manually label the data. 2. Generalization - With real-world data, there are so many different types of time series with different characteristics, which make it hard to generalize and find a silver bullet to solve all the problems. Some examples can be found in the figure below. 3. Efficiency - For any online anomaly detection system, efficiency is one of the key challenges. The system is expected to have low compute cost and low latency for serving. Inspiration In the computer vision domain, there is this concept called “visual saliency detection”. Saliency is what "stands out" in a photo or scene, enabling our eye-brain to quickly focus on the most important regions, as shown in figures below. Fig. Original image Fig. The salient part of the original image When we look at the time series chart, the most dominant and stand-out part is the anomalies. This similarity is where we got the inspiration and it turned out to generate great results. Algorithm Our solution then borrowed Spectral Residual (SR) from the visual saliency detection domain, then apply CNN on the results produced by the SR model As you can see from the algorithm architecture, after SR transformation, the transformed result magnifies the anomalies and the resulting signal is easier to generalize, therefore it provides us a way to training CNN with synthetic data. Spectral Residual The spectral residual algorithm consists of three major steps: Fourier Transform to get the log amplitude spectrum Calculation of spectral residual Inverse Fourier Transform that transforms the sequence back to the spatial domain Benefits SR is unsupervised, efficient, and has good generality. The problem becomes much easier based on the output of the SR model. We can train CNN on the SR output using fully synthetic data with simple synthetic rule Randomly select several points in the saliency map and calculate the injection value to replace the original point. Result We have performed online and offline experimentation, it outperformed state-of-the-arts consistently on open datasets and internal production datasets.
Building a digital guide dog for railway passengers with impaired vision
Catching your train on time can be challenging under the best of circumstances. Trains typically only stop for a few minutes, leaving little room for mistakes. For example, at Munich Main station around 240 express trains and 510 regional trains leave from 28 platforms per day. Some trains can also be quite long, up to 346 meters (1,135 ft) for express ICE trains. It is extremely important to quickly find the correct platform and platform section, and then the door closest to a reserved seat needs to be located. This already challenging adventure becomes even more so, if a vision impairment forces a customer to rely exclusively on auditory or tactile feedback. When traveling autonomously, without assistance, it is common practice to walk along the outside of a train, continuously tapping it with a white cane, to discover opened and closed doors (figure 1). While this works in principle, this practice has limitations, both in terms of speed and reliability. We therefore partnered with DB Systel GmbH, the digital partner for all Deutsche Bahn Group companies, to build the Digital Guide Dog. This is a feasibility study based on an AI-powered smartphone application that uses computer vision, auditory and haptic feedback to guide customers to the correct platform section and train car door. In this blog post, we are sharing some of the details and unique challenges that we experienced while the AI model behind this application.
Building Image Classifiers made easy with Azure Custom Vision
In our previous blog, we outlined that Supervised machine learning (ML) models need labeled data, but majority of the data collected in the raw format lacks labels. So, the first step before building a ML model would be to get the raw data labeled by domain experts. To do so, we outlined how Doccano is an easy tool for collaborative text annotation. However, not all data that gets collected is in text format, many a times we end up with a bunch of images but the end goal is again to build a Supervised ML model using them. Like stated previously, the first step would be to tag these images with specific labels. Image tagging as well as building and even deploying either a multi-class or a multi-label classifier can be done in a few simple steps using Azure Custom Vision.
Speeding up the Development of Natural Language Processing Solutions with Azure Machine Learning
The NLP Recipes Team Natural Language Processing (NLP) systems are used to ease the interactions between computers and humans using natural language. It is used in a variety of scenarios and industries from personal assistants like Cortana, to language translation applications, to call centers responding to specific users’ requests. In recent years, NLP has seen significant growth both in terms of quality and usability. Through new deep learning methods and state-of-the-art (SOTA) Deep Neural Network (DNN) algorithms, businesses are able to adopt Artificial Intelligence solutions to meet their customer’s needs. Unfortunately, finding the correct algorithm to use in different scenarios and languages remains a challenge. To help researchers and data scientists find the best fit for the problem at hand, Microsoft is open-sourcing the Microsoft NLP Recipes repository containing best practices in building and evaluating NLP systems across multiple tasks and languages. Specifically, our goals are to provide information for anyone who wants to: Learn about the newest algorithms and topics in NLP Develop and deploy NLP systems efficiently and with faster development times Bring SOTA algorithms to production with Azure Machine Learning Easing the Process for Data Scientists Several models have emerged over the years within the NLP community pushing towards neural network architectures for language modeling over more traditional approaches such as conditional random fields (CRFs) and Hidden Markov Models (HMMs). Since 2017, “Transformer” based neural network architectures, such as BERT, GPT-2, ELMo, XLNet, and RoBERTa, have developed as a dominant choice within the NLP community. These architectures dominate multi-task benchmarks such as GLUE as well as single task benchmarks (e.g. text classification and named entity recognition) as they allow leveraging pre-trained language models and adapting them to different downstream tasks. In addition, these pre-trained models are available with support for 100+ languages out of the box. The following table includes the current implementations of models within the repository, across different tasks and languages. Category Applications Methods Languages Text Classification Topic Classification BERT, XLNet, RoBERTa, DistilBERT en, hi, ar Named Entity Recognition Wikipedia NER BERT en Entailment MultiNLI Natural Language Inference BERT, XLNet en Question Answering SQuAD BiDAF, BERT, XLNet, DistilBERT en Sentence Similarity STS Benchmark BERT, GenSen en Embeddings Custom Embeddings Training Word2Vec, fastText, GloVe en Annotation Text Annotation Doccano en Model Explainability DNN Layer Explanation DUUDNM (Guan et al.) en The examples and utilities of the Microsoft NLP Recipes repository are focused with the following goals in mind to address these issues: Walk through NLP scenarios: These are common scenarios that are popular within the research community. The repository provides walkthrough examples within each scenario that show how once can get started with custom modeling and sample datasets. Ease use of SOTA algorithms: The utility functions includes easy-to-use wrappers of SOTA algorithms that dominate popular benchmarks like GLUE and SQuAD and provide an easy way to switch between them. With contributions from the community, we expect the latest algorithms to be included as they make their way up in the leaderboard. This gives the users easy access to the latest algorithms and reduces the friction when a new model is added. Global Language Support: Open source technologies like BERT, XLNet, and other transformer-based models, support 100+ languages and allow implementation of all the NLP scenarios across these languages. The datasets are hard to find, though, so we provide example notebooks and sample datasets on non-English languages such as Hindi, Arabic, and Chinese showing the implementation of NLP scenarios on sample datasets. Ease the use of common datasets: The repository has documentation and provides utility functions to use common academic datasets such as the Microsoft Research Paraphrase Corpus and the The Multi-Genre NLI (MultiNLI) Corpus A list of the datasets can be found here. Azure Machine Learning service support: The GitHub repository provides best practices for how to train, test, optimize, and deploy models on Azure using the Azure Machine Learning (Azure ML) service. Azure ML can be used intensively across various notebooks for tasks relating to AI model development, such as: Accessing datastores Scaling up and out on Azure Machine Learning Compute Automated Machine Learningand Hyperparameter tuning for model selection and hyperparameter tuning Tracking experiments and monitoring metrics to enhance the model creation process Distributed Training of models on clusters of many nodes and GPUs Deploying trained models as a web services to Azure Container Instance and Azure Kubernetes Service The NLP repository is meant to be accessible to anyone interested in building NLP solutions easily across a wide range of tasks and languages. Contributions from the community are always welcome to keep up to date with the latest state-of-the-art methods. Learn More Utilize the GitHub repository for your own NLP systems Try out an example of Text Classification using Transformer models Try out an example of Question answering using BiDAF and Azure Machine Learning Learn more about the Azure Machine Learning service Get started with a free trial of Azure Machine Learning service
