Detecting anomalies in oil&gas machines

%3CLINGO-SUB%20id%3D%22lingo-sub-1828199%22%20slang%3D%22en-US%22%3EDetecting%20anomalies%20in%20oil%26amp%3Bgas%20machines%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1828199%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20there%2C%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3ECurrently%2C%20I'm%20developing%20a%20Project%20with%20the%20first%20oil%26amp%3Bgas%20company%20in%20my%20country.%20This%20Project%20is%20having%20full%20attention%20from%20the%20customer%20C-Level%20people.%20We%20are%20so%20interested%20in%20having%20your%20advice%20on%20anomaly%20detection%20using%20ADX%20and%20of%20course%20the%20next%20step%20after%20anomalies%20which%20is%20Predictive%20Maintenance.%3C%2FP%3E%3CP%3EThanks.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1828199%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAMA%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1828275%22%20slang%3D%22en-US%22%3ERe%3A%20Detecting%20anomalies%20in%20oil%26amp%3Bgas%20machines%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1828275%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F219057%22%20target%3D%22_blank%22%3E%40Ricardo%20Gutierrez%3C%2FA%3E%26nbsp%3B%2C%20you%20can%20read%20about%20anomaly%20detection%20and%20forecasting%20in%20ADX%20for%20preventive%20maintenance%20and%20other%20scenarios%20here%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fanomaly-detection%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Ehere.%3C%2FA%3E%3CBR%20%2F%3ESpecific%20time%20series%20functions%20for%20anomaly%20detection%20are%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fkusto%2Fquery%2Fseries-decompose-anomaliesfunction%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Eseries_decompose_anomalies()%3C%2FA%3E%3CBR%20%2F%3Efor%20point%20anomalies%2C%20and%20series_fit_2lines()%20for%20change%20point%20detection.%20These%20functions%20are%20optimized%20to%20process%20thousands%20of%20time%20series%20in%20seconds%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1828334%22%20slang%3D%22en-US%22%3ERe%3A%20Detecting%20anomalies%20in%20oil%26amp%3Bgas%20machines%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1828334%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F308490%22%20target%3D%22_blank%22%3E%40adieldar%3C%2FA%3E%26nbsp%3B%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWe%20are%20uisng%20these%20features%20as%20per%20the%20attached%20file.%20My%20question%20is%20related%20to%20include%20things%20like%20Deep%20Learning%20LSTM%20as%20part%20of%20the%20predicition%20process.%20Do%20you%20have%20experiences%20that%20you%20could%20share%3F.%20Is%20it%20possible%20to%20have%20a%20meeting%20with%20your%20team%20to%20explain%20the%20idea%20just%20to%20se%20your%20opinion%3F.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1828425%22%20slang%3D%22en-US%22%3ERe%3A%20Detecting%20anomalies%20in%20oil%26amp%3Bgas%20machines%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1828425%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F219057%22%20target%3D%22_blank%22%3E%40Ricardo%20Gutierrez%3C%2FA%3E%26nbsp%3B%2C%20you%20can%20do%20training%2Fscoring%20on%20ADX%20using%20our%20inline%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fkusto%2Fquery%2Fpythonplugin%3Fpivots%3Dazuredataexplorer%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Epython()%3C%2FA%3E%20plugin%20that%20can%20run%20Python%20workloads%20in%20distributed%20manner%20on%20secure%20sandbox%20on%20ADX%20compute%20nodes.%20The%20Python%20image%20is%20based%20on%20Anaconda%2C%20but%20extended%20with%20Deep%20Learning%20packages%20like%20Tensorflow%2C%20keras%2C%20Pytorch%20and%20others.%20You%20can%20also%20consume%20and%20score%20ONNX%20models.%20We%20can%20have%20a%20session%20on%20that%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1828429%22%20slang%3D%22en-US%22%3ERe%3A%20Detecting%20anomalies%20in%20oil%26amp%3Bgas%20machines%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1828429%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSTRONG%3EExcellent%26nbsp%3B%3C%2FSTRONG%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F308490%22%20target%3D%22_blank%22%3E%40adieldar%3C%2FA%3E%26nbsp%3B.%20Please%20let%20me%20know%20how%20can%20we%20schedule%20the%20meeting%20so%20I%20can%20explain%20the%20idea%20and%20have%20a%20better%20perspective%20from%20you!.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Hi there,


Currently, I'm developing a Project with the first oil&gas company in my country. This Project is having full attention from the customer C-Level people. We are so interested in having your advice on anomaly detection using ADX and of course the next step after anomalies which is Predictive Maintenance.

Thanks.

 

5 Replies

@Ricardo Gutierrez , you can read about anomaly detection and forecasting in ADX for preventive maintenance and other scenarios here here.
Specific time series functions for anomaly detection are series_decompose_anomalies()
for point anomalies, and series_fit_2lines() for change point detection. These functions are optimized to process thousands of time series in seconds

Hi @adieldar ,

 

 

We are uisng these features as per the attached file. My question is related to include things like Deep Learning LSTM as part of the predicition process. Do you have experiences that you could share?. Is it possible to have a meeting with your team to explain the idea just to se your opinion?.

 

Thanks.

 

@Ricardo Gutierrez , you can do training/scoring on ADX using our inline python() plugin that can run Python workloads in distributed manner on secure sandbox on ADX compute nodes. The Python image is based on Anaconda, but extended with Deep Learning packages like Tensorflow, keras, Pytorch and others. You can also consume and score ONNX models. We can have a session on that

Excellent @adieldar . Please let me know how can we schedule the meeting so I can explain the idea and have a better perspective from you!.

 

Thanks!

@Ricardo Gutierrez, email me adieldar@microsoft.com and we shall set it