spark
5 TopicsAdvanced Time Series Anomaly Detector in Fabric
Anomaly Detector, one of Azure AI services, enables you to monitor and detect anomalies in your time series data. This service is being retired by October 2026, and as part of the migration process the anomaly detection algorithms were open sourced and published by a new Python package and we offer a time series anomaly detection workflow in Microsoft Fabric data platform.2.9KViews2likes0CommentsTrain your Model on Spark/Databricks, score it on ADX
Are you using Spark/Databricks to build Machine Learning models? Do you need to score new data that is streamed into Azure Data Explorer? If this is your scenario please read on! In this blog we show how to train an ML model on Azure Databricks, export it to ADX, and score new samples directly on ADX, in near real time, using inline Python code embedded in KQL query.6.2KViews2likes4Commentsazure synapse analytics /Apache spark logging
We are looking to implement logging using log4j in Azure synapse analytics - Spark pool / pyspark could not find any documentation for application level logging in azure synapse analytics. is there any default log library supported in azure synapse analytics ( log4j or any other) We are not able to upload the log4j.properties files into Synapse workspace as the workspace only accepts the .txt files for configuration. We were able to capture the spark logs using below MSFT docs. https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-storage.md now want to capture only Application logs with some logger library. There are good articles for log4j implementation for Azure Databricks. but we are not able follow same steps as the spark cluster in synapse not allowing to go into ubuntu folder of the cluster. any documentation or samples for synapse analytics will be a great help. Mark Kromer , James Serra - adding you for visibility1.7KViews2likes0Comments