HDInsight on AKS is a modern, reliable, secure, and fully managed Platform as a Service (PaaS) that runs on Azure Kubernetes Service (AKS). HDInsight on AKS allows you to deploy popular Open-Source Analytics workloads like Apache Spark, Apache Flink, and Trino without the overhead of managing and monitoring containers.
This blog explains how you can use Trino with HDInsight on AKS to experience the flavor of running federated queries.
Trino is an open-source distributed SQL query engine and well known for its capability to query data from multiple sources without moving your data. That is cool!! Isn't it?
And of course! you can use Trino for other use cases as well.
- Interactive and ad-hoc querying.
- Batch ETL across heterogenous systems.
- Reporting and Dashboarding.
With Trino on HDinsight on AKS, you can set up your cluster easily and fast with a few clicks. You can also enjoy the advantages of auto scaling, an intuitive UI to configure your cluster, and monitoring tools like Azure monitor and managed Prometheus and Grafana. Moreover, there are some exclusive features, such as a special connector for Power BI, result caching for all connectors, query scan statistics to analyze query performance, and Shared SQL connector.
There is lot more coming on your way with intuitive UI for catalog management, fine-grained access control with Apache Ranger, and an upgraded OSS version. Stay tuned!!
Let's see in action how you can run federated queries on Trino with HDInsight on AKS.
Scenario: Analyze NYC taxi data to understand the average fare and the passenger count at different zones.
Summary of steps for the demo:
- Take NYC taxi data from the official source.
- Land the data in ADLS Gen 2 and expose it as a Hive table in Trino.
- Prepare zone data and land in Azure Database for PostgresSQL.
- Run a federated query on two data sources ADLS Gen2 and Azure Database for PostgresSQL.
To build the demo yourself and experience the power of Trino firsthand, follow the step-by-step guid...
To learn more about HDInsight on AKS -
- Read our documentation - https://aka.ms/hdionaks-docs
- Join our community, share an idea or share your success story - https://aka.ms/hdionakscommunity
- Have a question on how to migrate or want to discuss a use case - https://aka.ms/askhdinsight