Ahead of IBC 2022, I joined a host of industry experts in an hour-long Faultline webinar to discuss the future of video codecs. We assessed some of the core characteristics of Versatile Video Coding (VVC), Essential Video Coding (EVC), Low Complexity Enhancement Video Coding (LCEVC), and AV1 and considered where they might fit in various media ecosystems.

Today, there are more competing codecs than at any time before. We are watching more content, across more devices, from multiple countries, and in various languages – and video codecs have continued to evolve to address these needs. There are unprecedented demands in video quality, resolution, fidelity, and frame rate.

And there are also many different viewpoints! I, Fabio Murra (V Nova), Jan de Cock (Synamedia), Janne Pelkonen (IdeaNova), and our moderator, Tommy Flanagan, enjoyed an engaging and respectful discussion throughout our session – which you can watch on-demand here.

What’s the outlook for VVC?

One of the main takeaways I was looking to project is that VVC is currently the pinnacle codec. It is undoubtedly the most efficient video coding standard available today. It drastically improves compression performance and is particularly important for higher resolution and frame rate content from 4K/UHD and above. That’s why we, at MediaKind, have focused our innovation development and research efforts on this codec, knowing that from those efforts we could also implement EVC if there is sufficient market demand due to the commonality between the two codec designs.

The roadmap for VVC adoption is already in place as it will be fundamental in enabling future formats, such as 8K broadcasts. The first chipsets with VVC decode and for 8K TVs have already been announced, while the broader rollout of devices supporting other codecs (which had been standardized earlier), such as EVC and AV1, were affected by COVID and silicon implementation and supply chain challenges.

VVC is also currently being implemented into the latest broadcasting standards, such as Brazil’s SDTVD and Digital Video Broadcasting (DVB) also became the first standards body of its kind to add VVC to its core specification for implementing video and audio coding in broadcast and broadband applications. While VVC will be essential for 8K video, it can and will also be relevant for lower video resolutions. In the first part of this blog, my colleague, Julien Le Tanou looks at how MediaKind successfully deployed and validated a full ABR profile ladder for a live OTT UHD channel using VVC with Aquila Streaming, using MediaKind’s aaS platform on Microsoft Azure.

Given its inclusion in next-generation broadcasting standards then the path for adoption of VVC in Broadcast seems clear. VVC will open doors for broadcasters to achieve significant spectrum efficiencies, enable greater connectivity, and ultimately, widen the adoption of 4K – and particularly 8K – content.

The future codec landscape for Streaming

One codec that was created to become the format of the future more rapidly was EVC (Essential Video Coding). However, as our panel discussed, this has not been the reality. The development of EVC was an attempt, back in 2018, to run a fast-track standardization exercise with a limited number of companies involved. Some of the points raised about EVC during the webinar focused on whether EVC will fit specific use cases that VVC and AV1 won’t already cover.

For example, as Jan, my fellow panelist, mentioned, AV1 does not have ubiquitous support when using an Internet browser as the decoder device. This, therefore, stalls other industry use cases where AV1 might otherwise be adopted, so another codec might have a bigger impact if it can achieve ubiquitous decode support across devices and browsers.

Despite the delays in chipsets for battery-efficient hardware, accelerated decode AV1s adoption at major social media and the key streaming platforms will see it have a place in device decode capabilities and the streaming codec landscape; at the very least for non-live on-demand based content.

For live streaming, there is a strong desire to achieve the widespread use of a unified common codec, as we have with MPEG-4 AVC today, for both the HLS and DASH worlds. Achieving such a desire will require one or more of these next-generation codecs to be adopted into the HLS specifications and into future Apple phones and tablets. Backing from such a major industry player will surely set the direction of the streaming codec landscape.

The question of sustainability 

A question that affects all codecs is the ability to ensure users can continue the industry-wide endeavor to be more sustainable. Greater workflow efficiencies inherently lead to better sustainability. For example, newer codecs, such as VVC, can reduce bitrate and bandwidth consumption, ultimately leading to a lower carbon footprint. This is due to a reduced requirement for storage, streaming capability, and caching. Just-in-time operations can also reduce CO2 emissions in power usage and costs.

The reduction in bitrates will likely require fewer profiles (i.e., the number of different resolution and bitrate versions of the video) required in the ABR ladder. This will go some way to offset the total compute required vs today’s processing needs.

Codecs get a bad rap regarding sustainability due to stats like “each new generation comes with up to 10x increase in computational complexity.” This is certainly a grey area, as this statement assumes all the available toolsets of a new codec standard will be fully utilized to achieve the maximum possible bitrate savings. By using slower than real-time encoding then this would be plausible in the early implementation of file-based processing.

However, for live encoding the implementation has to work in real-time within a finite amount of processing power; either the CPU capacity of a physical server for on-premises or the available node sizes from cloud service providers. Early real-time encoding implementations will use less compute power by using a subset of the toolset of the new codec while trying to achieve the best bitrate saving possible; this is the ‘secret sauce’ magic of live encoder vendor implementations.