HDInsight tools June release has been released! In this release, you can choose SBT as a build tool in addition to Maven when creating Spark projects in IntelliJ. With improvements to Spark job view and job graph in IntelliJ and Eclipse, more job info and statistics are now provided. You can also view job logs including driver stderr, stdout, and directory info easily in Spark job view.
Summary of key updates
Improved Spark job view and job graph in Intel
Improved Spark log view in IntelliJ and Eclipse
SBT build tool support when creating Spark project in IntelliJ
Read about it in the Azure blog.