Natural language processing (NLP) has attracted tremendous attention recently, due to the success of deep neural networks. In this session, we introduce the Microsoft/NLP open source repository, which contains best practices for building NLP systems at scale, with the help of AzureML. We show examples of how to develop state-of-the-art models, such as BERT, for multiple NLP scenarios and for multiple languages. These models, however, require considerable resources to train. Azure ML facilitates the training, deployment, and management of such models at scale. The content is based on our engagements with customers, partners, researchers, and the open source community.