Introducing Text Analytics for Health
Text Analytics for health is a new feature of Text Analytics that enables developers to process and extract insights from unstructured medical data. Trained on a diverse range of medical data—covering various formats of clinical notes, clinical trials protocols, and more—the health feature is capable of processing a broad range of data types and tasks, without the need for time-intensive, manual development of custom models to extract insights from medical data.Introducing Azure Cognitive Service for Language
Azure Cognitive Services has historically had three distinct NLP services that solve different but related customer problems. These are Text Analytics, LUIS and QnA Maker. As these services have matured and customers now depend on them for business-critical workloads, we wanted to take a step back and evaluate the most effective path forward over the next several years for delivering our roadmap of a world-class, state-of-the-art NLP platform-as-a-service. Each service was initially focused on a set of capabilities supporting distinct customer scenarios. For example, LUIS for custom language models most often supporting bots, Text Analytics for general purpose pre-built language services, and QnA Maker for knowledge-based question / answering. As AI accuracy has improved, the cost of offering more sophisticated models has decreased, and customers have increased their adoption of NLP for business workloads, we are seeing more and more overlapping scenarios where the lines are blurred between the three distinct services. As such, the most effective path forward is a single unified NLP service in Azure Cognitive Services. Today we are pleased to announce the availability of Azure Cognitive Service for Language. It unifies the capabilities in Text Analytics, LUIS, and the legacy QnA Maker service into a single service. The key benefits include: Easier to discover and adopt features. Seamlessness between pre-built and custom-trained NLP. Easier to build NLP capabilities once and reuse them across application scenarios. Access to multilingual state-of-the-art NLP models. Simpler to get started through consistency in APIs, documentation, and samples across all features. More billing predictability. The unified Language Service will not affect any existing applications. All existing capabilities of the three services will continue to be supported until we announce a deprecation timeline of the existing services (which would be no less than one year). However, new features and innovation will start happening only on the unified Language service. For example, question answering and conversational language understanding (CLU) are only available in the unified service (more details on these features later). As such, customers are encouraged to start making plans to leverage the unified service. More details on migration including links to resources are provided below. Here is everything we are announcing today in the unified Language service: Text Analytics is now Language Service: All existing features of Text Analytics are included in the Language Service. Specifically, Sentiment Analysis and Opinion Mining, Named Entity Recognition (NER), Entity Linking, Key Phrase Extraction, Language Detection, Text Analytics for health, and Text Summarization are all part of the Language Service as they exist today. Text Analytics customers don’t need to do any migrations or updates to their in-production or in-development apps. The unified service is backward compatible with all existing Text Analytics features. The key difference is when creating a new resource in the Azure portal UI, you will now see the resource labeled as “Language” rather than “Text Analytics”. Introducing conversational language understanding (preview) - the next generation of LUIS: Language Understanding (LUIS) has been one of our fastest growing Cognitive Services with customers deploying custom language models to production for various scenarios from command-and-control IoT devices and chat bots, to contact center agent assist scenarios. The next phase in the evolution of LUIS is conversational language understanding (CLU) which we are announcing today as a preview feature of the new Language Service. CLU introduces multilingual transformer-based models as the underlying model architecture and results in significant accuracy improvements over LUIS. Also new as part of CLU is the ability to create orchestration projects, which allow you to configure a project to route to multiple customizable language services, like question answering knowledge bases, other CLU projects, and even classic LUIS applications. Visit here to learn more. If you are an existing LUIS customer, we are not requiring you to migrate your application to CLU today. However, as CLU represents the evolution of LUIS, we encourage you to start experimenting with CLU in preview and provide us feedback on your experience. You can import a LUIS JSON application directly into CLU to get started. GA of question answering: In May 2021, we launched the preview of custom question answering. Today we are announcing the General Availability (GA) of question answering as part of the new Language Service. If you are just getting started with building knowledge bases that are query-able with natural language, visit here to get started. If you want to know more about migrating legacy QnA Maker knowledge bases to the Language Service see here. Your existing QnA Maker knowledge bases will continue to work. We are not requiring customers to migrate from QnA Maker at this time. However, question answering represents the evolution of QnA Maker and new features will only be developed for the unified service. As such, we encourage you to plan for a migration from legacy QnA Maker if this applies to you. Introducing custom named entity recognition (preview): Documents include an abundant amount of valuable information. Enterprises rely on pulling out that information to easily filter and search through those documents. Using the standard Text Analytics NER, they could extract known types like person names, geographical locations, datetimes, and organizations. However, lots of information of interest is more specific than the standard types. To unlock these scenarios, we’re happy to announce custom NER as a preview capability of the new Language Service. The capability allows you to build your own custom entity extractors by providing labelled examples of text to train models. Securely upload your data in your own storage accounts and label your data in the language studio. Deploy and query the custom models to obtain entity predictions on new text. Visit here to learn more. Introducing custom text classification (preview): While many pieces of information can exist in any given document, the whole piece of text can belong to one or more categories. Organizing and categorizing documents is key to data reliant enterprises. We’re excited to announce custom text classification, a preview feature under the Language service, where you can create custom classification models with your defined classes. Securely upload your data in your own storage accounts and label your data in the language studio. Choose between single-label classification where you can label and predict one class for every document, or multi-label classification that allows you to assign or predict several classes per document. This service enables automation to incoming pieces of text such as support tickets, customer email complaints, or organizational reports. Visit here to learn more. Language studio: This is the single destination for experimentation, evaluation, and training of Language AI / NLP in Cognitive Services. With the Language studio you can now try any of our capabilities with a few buttons clicks. For example, you can upload medical documents and get back all the entities and relations extracted instantly, and you can easily integrate the API into your solution using the Language SDK. You can take it further by training your own custom NER model and deploy it through the easy-to-use interface. Try it out now yourself here. Several customers are already using Azure Cognitive Service for Language to transform their businesses. Here's what two of them had to say: “We used Azure Cognitive Services and Bot Service to deliver an instantly responsive, personal expert into our customers’ pockets. Providing this constant access to help is key to our customer care strategy.” -Paul Jacobs, Group Head of Operations Transformation, Vodafone “Sellers might have up to 100,000 documents associated with a deal, so the time savings can be absolutely massive. Now that we’ve added Azure Cognitive Service for Language to our tool, customers can potentially compress weeks of work into days.” -Thomas Fredell, Chief Product Officer, Datasite To learn more directly from customers, see the following customer stories: Vodafone transforms its customer care strategy with digital assistant built on Azure Cognitive Services Progressive Insurance levels up its chatbot journey and boosts customer experience with Azure AI Kepro improves healthcare outcomes with fast and accurate insights from Text Analytics for health On behalf of the entire Cognitive Services Language team at Microsoft, we can't wait to see how Azure Cognitive Service for Language benefits your business!Simple Application to summarize data using GPT-3 Openai model
Let's build a Power App to summarize data using GPT-3 openai model What's needed First go to - https://beta.openai.com/ and sign up for an trial account If you organization has an account then register with that account Create a new API key Go to Right top in the above web site and click your name and then click on the API key then create a new key to use please make sure delete the key after completing the tutorial Go to documentation and take a look at completions we are going to use completion api Also need Azure account and power platform license Create a Power App To create a power app first need to create a power flow Flow is invoked by a powerapp trigger Text information will be passed to the flow Power Flow Let's create a power flow On the left menu in power apps click on flows https://make.preview.powerapps.com/ Click on flows Click New Flow Name it as getsummary here is the entire flow First add trigger as Power Apps then Initialize a variable for value assign from Power apps that will take the input value and assign to the variable called prompt Now lets send the data to openai API to use davinci model using GPT-3 First bring HTTP action Then select the action as POST here is the URL https://api.openai.com/v1/engines/davinci-msft/completions Note we need content-type as application/json also need Authorization as Bearer <your_api_key> here is the body { "prompt": @{triggerBody()['Initializevariable_Value']}, "temperature": 0.5, "max_tokens": 100, "top_p": 1, "frequency_penalty": 0.2, "presence_penalty": 0, "stop": [ "\"\"\"" ] } make sure the prompt property is substituted with the value of the variable prompt as shown above Next we need to parse the response from above HTTP output Now we need to provide a sample document to parse the JSON schema { "id": "cmpl-xxxxxxxxxxx", "object": "text_completion", "created": 1640707195, "model": "davinci:2020-05-03", "choices": [ { "text": " really bright. You can see it in the sky at night.\nJupiter is the third brightest thing in the sky, after the Moon and Venus.\n", "index": 0, "logprobs": null, "finish_reason": "stop" } ] } Schema generated from sample { "type": "object", "properties": { "id": { "type": "string" }, "object": { "type": "string" }, "created": { "type": "integer" }, "model": { "type": "string" }, "choices": { "type": "array", "items": { "type": "object", "properties": { "text": { "type": "string" }, "index": { "type": "integer" }, "logprobs": {}, "finish_reason": { "type": "string" } }, "required": [ "text", "index", "logprobs", "finish_reason" ] } } } } initialize a variable called outsummary select the Type as String After parsing we need to loop the array and assign the text to the variable Bring Apply to each action Select Choices as the array property now bring Set variable action Assign the currentitem to the variable outsummary Next add Respond to Power Apps Sent the outsummary as response back to Power Apps Save the flow Do a manual test run by passing sample text If successful then you are set with flow Power Apps Now lets create a Power App This is only a simple app i am creating a canvas app Name the app as: OpenAPITest Note: this process can be applied to any HTTP REST enabled actions needed to be invoked by Power Apps Now we need to create a canvas Bring Text Input Box Add default text as prompt My second grader asked me what this passage means:\n\"\"\"\nJupiter is the fifth planet from the Sun and the largest in the Solar System. It is a gas giant with a mass one-thousandth that of the Sun, but two-and-a-half times that of all the other planets in the Solar System combined. Jupiter is one of the brightest objects visible to the naked eye in the night sky, and has been known to ancient civilizations since before recorded history. It is named after the Roman god Jupiter.[19] When viewed from Earth, Jupiter can be bright enough for its reflected light to cast visible shadows,[20] and is on average the third-brightest natural object in the night sky after the Moon and Venus.\n\"\"\"\nI rephrased it for him, in plain language a second grader can understand:\n\"\"\"\nJupiter is the fifth planet from the Sun. It is a big ball of gas. It is really bright, and you can see it in the sky at night. The ancient people named it after the Roman god Jupiter.\nJupiter is really big. It is bigger than all of the other planets in the Solar System combined. Jupiter is so big that if you could fit all of the other planets inside of Jupiter, you could still see Jupiter shining in the night sky!\nJupiter is Now add a button Call the flow and assign the return value to the variable Set(summarytext,getsummary.Run(TextInput1.Text)) in OnSelect apply the above formulat. getsummary is the name of the flow and we are passing parameters as textinput1.text Now lets add a Text lable as label1 Assign the text property to summarytext.summarytext summarytext is the output property set in the flow summarytext.summarytext Save the canvas app Run the app and test it below should be the output The above flow can be used to access most API's in open AI. So does we can use this for other Cognitive services Samples2022/openaigpt3.md at main · balakreshnan/Samples2022 (github.com)Integrating AI: Best Practices and Resources to Get Started
What are the problems you can solve with AI? How do you experiment and prototype? This article aims to help you decide if and how to integrate AI into your applications, get you started with Azure’s ready to use AI solutions, Cognitive Services and answer your most frequent questions when getting started.Introducing Native Document Support for PII Detection (Public Preview)
This capability can now identify, categorize, and redact sensitive information (PII - Personally Identifiable Information) in unstructured text directly from complex documents, allowing users to ensure data privacy compliance within a streamlined workflow. It effortlessly detects and safeguards crucial information, adhering to the highest standards of data privacy and security.
Summarize text with Text Analytics API
We strive to identify and summarize important ideas from the ever-growing data everyday. Extractive summarization is a feature in Text Analytics that produces a summary by extracting sentences which collectively represent the most important or relevant information within the original content.
