Forum Discussion

Richard_Wray's avatar
Richard_Wray
Copper Contributor
Jun 04, 2024

Azure OpenAI Whisper From Power Automate

Hi all,  

 

I've successfully created a working Whisper Model In Azure OpenAI Service (tested at 

Speech Studio - Whisper Model in Azure OpenAI Service ) to do audio transcriptions.  I've had the exact model working via a Custom Connector to the regular OpenAI version of Whisper but need direct access via Azure to avoid needing to give guest users to my app the API Key.

 

When I try and call it via HTTP in Power Automate I get an 'UnresolveableHostName' error. and the 502 error in the image below

 

HTTP output is 

 

{
    "statusCode": 404,
    "headers": {
        "apim-request-id": "1859e66c-6662-470e-8844-ec0f4053ac86",
        "Strict-Transport-Security": "max-age=31536000; includeSubDomains; preload",
        "X-Content-Type-Options": "nosniff",
        "Date": "Tue, 04 Jun 2024 04:35:01 GMT",
        "Content-Length": "56",
        "Content-Type": "application/json"
    },
    "body": {
        "error": {
            "code": "404",
            "message": "Resource not found"
        }
    }
}

 

I'm wondering if it is my URI

 

https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio/transcriptions?api-version=001

 

Speech Service is 'azureopenaispeechjuneeastus2'

Location / Region is 'eastus2'

Endpoint is 'https://azureopenaispeechjuneeastus2.openai.azure.com/'

Deployment is 'wmcazureopenaiwhisper'

 

Any thoughts would be appreciated.

 

 

 

  • ymuto's avatar
    ymuto
    Icon for Microsoft rankMicrosoft
    Jun 14, 2024

    Hello Richard_Wray 

     

    api-version in your URI is incorrected.

    api-version should be 2024-02-01

     

    Like this.
    https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio/transcriptions?api-version=2024-02-01

     

     

    Thanks,

    Yoshihiko Muto(@ymuto)

    • Richard_Wray's avatar
      Richard_Wray
      Copper Contributor
      Jun 14, 2024

      ymuto 

       

      Thanks.  I actually cracked it in the end (found a blog, that I can't find now 😞  ).  You are correct.  The line should have read 'https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio/transcriptions?api-version=2024-02-01'

       

      Here's the full flow to pass the contents from a Power App Mic control to Azure OpenAI Whisper and then pass the transcribed text back to the Power App.  In this case the flow is called 'AzureTextTranscription1.0'

       

      This was called using the following line in the 'On Stop' setting on the Mic in the Power App.

       

      Set(TempRecordingVar,MicrophoneQ1.Audio);
      Set(TempJason,JSON(MicrophoneQ1.Audio,JSONFormat.IncludeBinaryData));
      Set(StringB64Var,Mid(TempJason,25,Len(TempJason)-25));
      Set(flowResonseQ1, 'AzureTextTranscription1.0'.Run(Q1UserDetails,", ",StringB64Var));

       

       

       

       

       

      Take the feed from Power Apps

       

      Convert the audio content

       

      Compose

      {
      "$content-type": "multipart/form-data",
      "$multipart": [
      {
      "headers": {
      "Content-Disposition": "form-data; name=\"file\"; filename=\".mp3\""
      },
      "body": {
      "$Content-type": "audio/webm",
      "$content":
      }
      }
      ]
      }

       

      HTTP Action (I've removed my key from this image)

       

      Parse the response

       

      {
          "type": "object",
          "properties": {
              "text": {
                  "type": "string"
              }
          }
      }
       
      Respond back to the Power App with the Azure Open AI Whisper results
       

       

      It works great on browsers, but still a few issues on making it work through ther the Power Apps app on my Android phone.  Works fine on the browsers on the phone though.  Weird!!!!

       

      That will be later though.  I think it is to do with how phones store audio.

       

       

       

       

       

Resources