Use BigDL on HDInsight Spark for Distributed Deep Learning

%3CLINGO-SUB%20id%3D%22lingo-sub-58437%22%20slang%3D%22en-US%22%3EUse%20BigDL%20on%20HDInsight%20Spark%20for%20Distributed%20Deep%20Learning%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-58437%22%20slang%3D%22en-US%22%3E%3CP%3EDeep%20learning%20is%20impacting%20everything%20from%20healthcare%2C%20transportation%2C%20manufacturing%2C%20and%20more.%20Companies%20are%20turning%20to%20deep%20learning%20to%20solve%20hard%20problems%20like%20image%20classification%2C%20speech%20recognition%2C%20object%20recognition%2C%20and%20machine%20translation.%20In%20this%20blog%20post%2C%20Intel%E2%80%99s%20BigDL%20team%20and%20Azure%20HDInsight%20team%20collaborate%20to%20provide%20the%20basic%20steps%20to%20use%20BigDL%20on%20Azure%20HDInsight.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIn%202016%2C%20Intel%20released%20its%20BigDL%20distributed%20Deep%20Learning%20project%20into%20the%20open-source%20community%2C%20BigDL%20Github.%20It%20natively%20integrates%20into%20Spark%2C%20supports%20popular%20neural%20net%20topologies%2C%20and%20achieves%20feature%20parity%20with%20other%20open-source%20deep%20learning%20frameworks.%20BigDL%20also%20provides%20100%2B%20basic%20neural%20networks%20building%20blocks%20allowing%20users%20to%20create%20novel%20topologies%20to%20suit%20their%20unique%20applications.%20Thus%2C%20with%20Intel%E2%80%99s%20BigDL%2C%20the%20users%20are%20able%20to%20leverage%20their%20existing%20Spark%20infrastructure%20to%20enable%20Deep%20Learning%20applications%20without%20having%20to%20invest%20into%20bringing%20up%20separate%20frameworks%20to%20take%20advantage%20of%20neural%20networks%20capabilities.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22a72de289-ce6a-4382-a066-e5a33ab466bc%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F12627i44C836D43B0B88C2%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22a72de289-ce6a-4382-a066-e5a33ab466bc%22%20alt%3D%22a72de289-ce6a-4382-a066-e5a33ab466bc%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ERead%20about%20it%20on%20the%20%3CA%20href%3D%22https%3A%2F%2Fazure.microsoft.com%2Fen-us%2Fblog%2Fuse-bigdl-on-hdinsight-spark-for-distributed-deep-learning%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20blog%3C%2FA%3E.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-58437%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAnalytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EHDInsight%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Community Manager

Deep learning is impacting everything from healthcare, transportation, manufacturing, and more. Companies are turning to deep learning to solve hard problems like image classification, speech recognition, object recognition, and machine translation. In this blog post, Intel’s BigDL team and Azure HDInsight team collaborate to provide the basic steps to use BigDL on Azure HDInsight.

 

In 2016, Intel released its BigDL distributed Deep Learning project into the open-source community, BigDL Github. It natively integrates into Spark, supports popular neural net topologies, and achieves feature parity with other open-source deep learning frameworks. BigDL also provides 100+ basic neural networks building blocks allowing users to create novel topologies to suit their unique applications. Thus, with Intel’s BigDL, the users are able to leverage their existing Spark infrastructure to enable Deep Learning applications without having to invest into bringing up separate frameworks to take advantage of neural networks capabilities.

 

a72de289-ce6a-4382-a066-e5a33ab466bc

 

Read about it on the Azure blog.

0 Replies