Hello BlueRaja , please see below for your answers:
- The codec model can be configured for different resolutions. The resolutions cited in the blog post are typical examples used in multi-party video conferencing scenarios, where both encoding and decoding must run in real time while using limited NPU capacity. Higher resolutions can be achieved by allocating more compute to the model if you are flexible with the system constraints. For example, running only the decoder (a streaming scenario) or utilizing the full NPU capacity when no other workloads are competing for it could allow the model to run at a higher resolution. I encourage you to try it out and raise an issue to discuss the possibility of running it in the manner you are interested in.
- The above examples run at 30 fps on Mac and Intel Lunar Lake and Panther Lake NPUs, utilizing about 50% of the NPU capacity. The compute cost is the same for encoding and decoding in the current design, so this should give you an idea of how much compute each requires.
- Model sizes are listed on the GitHub page. The large version is approximately 18 MB, and the small version is approximately 5.4 MB, plus the C++ library.
- Currently, both encoding and decoding require the same model version, so some form of versioning is needed to maintain stream compatibility within an application. We welcome efforts to standardize this aspect so that interoperability can be achieved across the board.
Microsoft