Overview
In order to prepare content for delivery by adaptive bitrate streaming, the video needs to be encoded at multiple bit-rates (high to low). In order to ensure graceful degradation of quality, as the bitrate is lowered, so is the resolution of the video. This results in a so-called encoding ladder – a table of resolutions and bitrates, as you can see in some of our fixed encoding presets, such as “H264MultipleBitrate1080p”.
Interest in moving beyond a one-preset-fits-all-videos approach increased after Netflix published their blog in December 2015. Since then, multiple solutions for content-aware encoding have been released in the marketplace – see this article for an overview. The idea is to be aware of the content – to customize or tune the encoding ladder to the complexity of the individual video. At each resolution, there is a bitrate beyond which any increase in quality is not perceptive – the encoder operates at this optimal bitrate value. The next level of optimization is to select the resolutions based on the content – for example, a video of a PowerPoint presentation does not benefit from going below 720p. Going further, the encoder can be tasked to optimize the settings for each shot within the video – Netflix described such approach in 2018.
In early 2017, we released the Adaptive Streaming preset to address the problem of the variability in the quality and resolution of the source videos. Our customers had a varying mix of content – some at 1080p, others at 720p, and a few at SD and lower resolutions. Further, not all source content were high quality mezzanines from film or TV studios. The Adaptive Streaming preset addressed these problems by ensuring that the bitrate ladder never exceeds the resolution or the average bitrate of the input mezzanine.
The experimental content-aware encoding preset extends that mechanism, by incorporating custom logic that lets the encoder seek the optimal bitrate value for a given resolution, but without requiring extensive computational analysis. The net result is that this new preset produces an output that has lower bitrate than the Adaptive Streaming preset, but at a higher quality. See sample graphs below, that show the comparison using quality metrics like PSNR and VMAF. The source was created by concatenating short clips of high complexity shots from movies and TV shows, intended to stress the encoder. By definition, this preset produces results that vary from content to content – it also means that for some content, there may not be significant reduction in bitrate or improvement in quality.
The present currently is tuned for high complexity, high quality source videos (movies, TV shows). Work is in progress to adapt to low complexity (eg. PowerPoint presentations) content, as well as poorer quality videos. This preset also uses the same set of resolutions as the Adaptive Streaming preset – we are working on methods to select the minimal set of resolutions based on the content.
Using the experimental preset
You can create Transforms that use this preset as follows. If using a tutorial like this one, you can update the code as follows:
TransformOutput[] output = new TransformOutput[]
{
new TransformOutput
{
// The preset for the Transform is set to one of Media Services built-in sample presets.
// You can customize the encoding settings by changing this to use "StandardEncoderPreset" class.
Preset = new BuiltInStandardEncoderPreset()
{
// This sample uses the new experimental preset for content-aware encoding
PresetName = EncoderNamedPreset. ContentAwareEncodingExperimental
}
}
};
Next steps
Now that you have learnt about this new option of optimizing your videos, we invite you to try it out. You can send us feedback via the link below, or engage us more directly at amsved@microsoft.com