Where to run your Azure Stream Analytics job?
Published May 17 2021 11:27 AM 3,533 Views
Microsoft

Azure Stream Analytics is generally available across Azure ecosystem – global Azure, Azure IoT Edge and Azure Stack Hub. It allows developers to build architectures for stream processing in different scenarios using the same tools and query language. This blog will provide common scenarios recommendation to help to make the best choice for deliver near-real-time analytical intelligence for your organization.

 

Azure Cloud

Azure Stream Analytics on Azure provides large-scale analytics in the cloud. It is designed to analyze and process high volumes of fast streaming data from multiple sources simultaneously. Patterns and relationships can be identified in information extracted from several input sources including devices, sensors, clickstreams, social media feeds, and applications. These patterns can be used to trigger actions and initiate workflows such as creating alerts, feeding information to a reporting tool, or storing transformed data for later use.

The following scenarios are examples of when you can use Azure Stream Analytics:

  • Real-time analytics on Point of Sale for inventory control and anomaly detection.
  • Remote monitoring and predictive maintenance of high value assets.

For more details about Stream Analytics on cloud, please see here.

 

Azure IoT Edge

Azure Stream Analytics on IoT Edge extends the streaming capabilities and analytics from the cloud to the device level. It is designed for scenarios where customers need low latency command control, have limited cloud connectivity or bandwidth, or require regulatory compliance. An Edge job is created in the Azure portal and then deployed as an IoT Edge module without additional code. By processing telemetry streams at the edge, customers can reduce the amount of uploaded data and reduce the time it takes to react to actionable insights.

The following scenarios are examples of when you can use Azure Stream Analytics on IoT Edge:

  • Local real-time analytics and decision making for vessel managements.
  • Analyze and anonymize telemetry streams before sending it to the cloud.

 Visit Stream Analytics Edge to get started.

 

Azure Stream Analytics is built into Azure SQL Edge to provide capabilities to stream, process, and analyze relational and non-relational such as JSON, graph and time-series data. It uses the same constructs and capabilities as Azure Stream Analytics on IoT Edge. Customers can pull and run the container image without interacting with the cloud, which allows customers to set up Stream Analytics and stay fully disconnected to the Internet.

More details are available on the product documentation page.

 

Azure Stack Hub

Azure Stream Analytics is a hybrid service on Azure Stack Hub. It is an IoT Edge module which is configured in Azure but can be run on Azure Stack Hub. Internet connection is needed when creating ASA Edge job and deploying the module. It provides customers with the ability to build truly hybrid architecture for stream processing in their own private and autonomous cloud.

The following scenarios are examples of when you can use Azure Stream Analytics on Azure Stack Hub:

  • Real-time analytics on large amounts of confidential data in a facility with top level security.
  • Process stream data from financial reporting on-premises to meet regulatory requirements.

Learn more at the tutorial page.

 

Here are current features and limitations in different environments:

 

ASA on Cloud

ASA on IoT Edge

ASA on Azure Stack Hub

SQL Edge

Requires Internet Connection

All the time

Yes, during job creation and deployment

Yes, during job creation deployment

no

Maximum SUs

192

6 per container

6 per container

6 per container

Input Adapters

IoT Hub

Event Hub

Azure Storage

Data Lake Gen 2

Edge Hub

Event Hub

IoT Hub

Edge Hub

Event Hub

 

Edge Hub

Kafka

Output Adapters

Event Hub

Azure Functions

Power BI

Azure Synapse Analytics

Cosmos DB

SQL DB

Blob Storage

more...

Edge Hub

SQL Database/DW

Event Hub

Blob Storage

Edge Hub

Event Hub

Blob Storage

Edge Hub

SQL Database

Time Windowing function

Yes

Yes (no late arrival policy)

absence of late arrival policy does not affect latency. This is due to limitation on the Edge Hub. 

Yes (no late arrival policy)

Yes (no late arrival policy)

UDF C#

Yes

Yes

Yes

No

UDF JavaScript

Yes

No

No

No

Machine Learning

Yes

No

No

Yes

Anomaly Detection

Yes

Yes

Yes

Yes

Geospatial Analytics

Yes

Yes

Yes

Yes

Reference Data

SQL

Blob Storage

Static local file

Static local file

No

Operator: PARTITION BY

Yes

No

No

No

Replay from checkpoint

Yes

No

No

No

 

 

Co-Authors
Version history
Last update:
‎Jun 07 2021 08:41 AM
Updated by: