Azure OpenAI Whisper From Power Automate

Copper Contributor

Hi all,  

 

I've successfully created a working Whisper Model In Azure OpenAI Service (tested at 

Speech Studio - Whisper Model in Azure OpenAI Service ) to do audio transcriptions.  I've had the exact model working via a Custom Connector to the regular OpenAI version of Whisper but need direct access via Azure to avoid needing to give guest users to my app the API Key.

 

When I try and call it via HTTP in Power Automate I get an 'UnresolveableHostName' error. and the 502 error in the image below

 

HTTP output is 

 

{
    "statusCode": 404,
    "headers": {
        "apim-request-id": "1859e66c-6662-470e-8844-ec0f4053ac86",
        "Strict-Transport-Security": "max-age=31536000; includeSubDomains; preload",
        "X-Content-Type-Options": "nosniff",
        "Date": "Tue, 04 Jun 2024 04:35:01 GMT",
        "Content-Length": "56",
        "Content-Type": "application/json"
    },
    "body": {
        "error": {
            "code": "404",
            "message": "Resource not found"
        }
    }
}

 

I'm wondering if it is my URI

 

https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio...

 

Speech Service is 'azureopenaispeechjuneeastus2'

Location / Region is 'eastus2'

Endpoint is 'https://azureopenaispeechjuneeastus2.openai.azure.com/'

Deployment is 'wmcazureopenaiwhisper'

 

Any thoughts would be appreciated.

 

 

 

Richard_Wray_0-1717475679434.png

3 Replies

Hello @Richard_Wray 

 

api-version in your URI is incorrected.

api-version should be 2024-02-01

 

Like this.
https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio/transcriptions?api-version=2024-02-01

 

 

Thanks,

Yoshihiko Muto(@ymuto)

@ymuto 

 

Thanks.  I actually cracked it in the end (found a blog, that I can't find now :(  ).  You are correct.  The line should have read 'https://azureopenaispeechjuneeastus2.openai.azure.com/openai/deployments/wmcazureopenaiwhisper/audio...

 

Here's the full flow to pass the contents from a Power App Mic control to Azure OpenAI Whisper and then pass the transcribed text back to the Power App.  In this case the flow is called 'AzureTextTranscription1.0'

 

This was called using the following line in the 'On Stop' setting on the Mic in the Power App.

 

Set(TempRecordingVar,MicrophoneQ1.Audio);
Set(TempJason,JSON(MicrophoneQ1.Audio,JSONFormat.IncludeBinaryData));
Set(StringB64Var,Mid(TempJason,25,Len(TempJason)-25));
Set(flowResonseQ1, 'AzureTextTranscription1.0'.Run(Q1UserDetails,", ",StringB64Var));

 

 

 

Richard_Wray_3-1718345200375.png

 

 

Take the feed from Power Apps

 

Richard_Wray_1-1718345127556.png

Convert the audio content

 

Richard_Wray_2-1718345168910.png

Compose

Richard_Wray_4-1718345237383.png

{
"$content-type": "multipart/form-data",
"$multipart": [
{
"headers": {
"Content-Disposition": "form-data; name=\"file\"; filename=\".mp3\""
},
"body": {
"$Content-type": "audio/webm",
"$content":
}
}
]
}

 

HTTP Action (I've removed my key from this image)

 

Richard_Wray_5-1718345300208.png

Parse the response

Richard_Wray_6-1718345334656.png

 

{
    "type": "object",
    "properties": {
        "text": {
            "type": "string"
        }
    }
}
 
Respond back to the Power App with the Azure Open AI Whisper results
 
Richard_Wray_7-1718345392843.png

 

It works great on browsers, but still a few issues on making it work through ther the Power Apps app on my Android phone.  Works fine on the browsers on the phone though.  Weird!!!!

 

That will be later though.  I think it is to do with how phones store audio.

 

Richard_Wray_8-1718345977002.jpeg

 

 

 

 

Hello @Richard_Wray 

Microphone control output format for mobile is different and depends upon device used. For Canvas App Microphone control, the audio formats are 3gp for Android, AAC for iOS, webm for web browsers.
Microphone control in Power Apps - Power Apps | Microsoft Learn

You could use connector to convert audio file type like a CloudConvert -
https://learn.microsoft.com/en-us/connectors/cloudconvert/