Generating Captions with Video Indexer and Logic Apps

Wouldn't it be nice if you could set up custom workflows with connectors to effectively index and generate captions from a large volume of video and audio files, without needing to write a single line of code?  With Azure Media Services Video Indexer, you can utilize Logic Apps and Power Automate connectors that are compatible with our API, without coding! The resulting solution provides better visibility into the health of your workflow, while making debugging easier. 

 

Azure Media Services Video Indexer v2 REST API supports both server-to-server and client-to-server communication. Video Indexer users can integrate video and audio insights (including captions) easily into their application logic, unlocking new experiences and monetization opportunities.

 

In this tutorial, you will learn how to:

• Trigger a Logic Apps / Power Automate flow every time a video file is added to a specific OneDrive or OneDrive Business folder.
• Use the flow to automatically index and generate captions for the video file.
• Set a second flow to save the video captions to a different OneDrive folder.

 

Prerequisites

1. Create an Azure subscription if you do not have one. You can create a free account here.
2. A Video Indexer account, with access to the APIs via API key.
3. A One Drive or OneDrive Business Account. In your OneDrive, create two folders – one to store videos and the other to store the captions generated by Video Indexer.

 

In this tutorial, we will use the term “flow” to describe both Power Automate flows and Logic Apps automations. Since the editors and capabilities are almost identical in both solutions, please note that the diagrams and explanations are applicable to both Logic Apps and Power Automate.

 

Index video automatically by copying it to OneDrive

This scenario is comprised of two separate flows that work together. The first flow is triggered when a file is added to a OneDrive or OneDrive for Business account. The flow uploads the new file to Video Indexer with a callback URL to send a notification once the indexing operation is completed.

 

The second flow is triggered from the callback URL. It saves the generated captions back to a JSON file in OneDrive. This two-flow approach can effectively be used to support async upload and indexing of a large volume of files.

 

You can create a similar flow that will be triggered when a blob is added or modified in an Azure Storage account to save the captions to your blob.

 

Set up the file upload flow

The first flow is triggered whenever a file is added to your OneDrive folder. When triggered, it will create a URL that enables you to upload and index your file in Video Indexer. Let’s start by creating the following flow in Logic Apps or Power Automate. Below is a workflow diagram from Logic Apps editor, but since the Power Automate editor is practically the same you can follow the steps in both solutions.

 

Note that every OneDrive trigger or action is available both for OneDrive and OneDrive for Business. Make sure to choose the one you use when you create the flow.

To set up the first flow, you will need to provide your OneDrive credentials and Video Indexer API Key.
“Connection name” can be a value of your choosing.

 

 

 

Once you are connected to your OneDrive and Video Indexer accounts, go to the “When a file is created” trigger and select the folder where you want your video files to be stored. In this example we’ve called the folder “VideoIndexer”.

 

 

Next, go to the “Create share link” action.  Select “File identifier” from the Dynamic content options to send the new file to the “Create share link” action and set the Link type to “Direct” so Video Indexer could access the file in the next steps.

 

 

Fill out your account Location and ID to get the Video Indexer account token. Access tokens are needed to use the Video Indexer APIs for security purposes. This ensures that all calls are either coming from you or those who have access permissions to your account. You can read more about access tokens here.

 

 

For “Upload video and index”, fill out the required parameters and Video URL. Select “Add new parameter” and select “Callback URL”.

 

 

Leave the callback URL empty for now. You will add it after you have finished the second flow and when the callback URL is created.

Add the parameter “Indexing Preset” and set it to “AudioOnly”.

You can also choose the default value for the other parameters or set them according to your needs.

 

Click “Save”, and you are ready to move on to configuring the second flow, which will be used to generate the captions once the upload and indexing process is completed.

 

Set up the caption's Generating flow

Once the uploading and indexing from the first flow is completed, it will send an HTTP request with the correct callback URL to trigger the second flow. It will also retrieve the captions generated by the Video Indexer. In this example, the output of your indexing job is stored in the OneDrive folder you set up. However, it is up to you to decide on how or where you would like to store the output.

Create the second flow separately from the first one:

 

 

To set up this flow, you will need to provide your Video Indexer API Key and OneDrive Business credentials again. Update the same parameters as you did for the first flow.

 

For your trigger, you will see the HTTP POST URL field. The URL will not be generated until after you save your flow; however, you will need the URL eventually. We will come back to this later.

 

Fill out your account Location and ID to get the Video Indexer account token.

 

Go to the “Get Video Index” action and fill out the required parameters. For Video ID, put in the following expression: triggerOutputs()['queries']['id']

 

 

This expression tells the connector to get the Video ID from the output of your trigger. In this case, the output of your trigger will be the output of “Upload video and index” from your first trigger.

Note: This flow can be built without the “Get Video Index” action but using it will provide the "Video Name" field which we will use later to name the captions file.

 

Go to the “Get Video Captions” action and fill out the required parameters. For Video ID, put in the same value you used in the last action: triggerOutputs()['queries']['id']

 

 

Go to the “Create file” action and select the folder in which you want the captions to be saved. Every video you upload will create a separate VTT caption file. You can set the caption file name to be identical to your video name by using the Name field from the “Get Video Index” action. For File Content, go to the Dynamic Fields of “Get Video Caption” and click on “See more”. Select the Body field.

 

 

Click “Save flow.”

 

Once the flow is saved, an HTTP POST URL is created in the trigger. Copy the URL from the trigger.

 

 

Now, go back to the first flow and paste the URL in the “Upload video and index” action for the Callback URL parameter.

Make sure both flows are saved, and you’re good to go!

Try out your newly created Logic App or Power Automate solution by adding a video to your OneDrive folder. Go back a few minutes later and you will see that the captions appear in the destination folder.

 

Clean up resources

After you are done with this tutorial, feel free to keep this Logic App or Power Automate solution up and running. However, if you do not want to keep this solution running, turn off both of your flows if you are using Power Automate or disable both flows if you’re using Logic Apps.

 

Next steps and further reading

For further exploration, read our tutorial on how to use Video Indexer with Logic Apps and Power Automate to automatically index videos on your blob storage and extract the insights JSON.

 

This tutorial showed an example of Video Indexer connectors. You can use the Video Indexer connectors for any API call provided by Video Indexer. For example, you can upload and retrieve insights, translate the results, get embed-able widgets and even customize your models. In addition, you can choose to trigger those actions based on different sources like updates to file repositories or emails sent. Next, choose to have the results updated to our relevant infrastructure/application or generate other action items.