How to manage the codec diversity (I)

The proliferation of video consumer devices, including STBs, smart TVs and handheld devices has produced a large diversity of software and formats. The industry is far from reaching an agreement to standardize, while each device (or family of devices) must be served with specific protocols. Each content publisher has to find their own customized solutions to reach its business KPI. This segmentation in the industry around protocols and devices offers a great opportunity for improvement.

The solutions available in the market are addressing this problem with a one-size-fits-all approach. However, there’s an opportunity to find better paths preparing the content depending on the users’ devices capacities, sizes and networks conditions. Encoding technologies and algorithms are evolving very rapidly, and there’s no perspective whatsoever that there will be a predominant format among H.264, H.265, VP9, or AV1. Therefore, it becomes essential to find solutions that cope with this diversity.

One format to rule them all?

For now, it seems clear that there will be no winner in the codec war, moreover when 4K and 8K become more the norm than the exception. Actually, the different players in the industry are supporting their more convenient standards. The silver lining of segmentation is that it fosters competition and, with it, R&D and evolution. But when your business depends on reaching your viewers with the best quality experience regarding your business rules, managing this complexity and trading between workflow and distribution costs vs. QoE may be the difference between succeeding or not.

Within the existing codec diversity, we can observe two large groups. On one hand, we have the most classic industry with H.265 / HEVC as a reference codec. It carries a load with patents potentially hindering its full adoption. On the other side, we find a community developing patent-free technologies, pushing codecs such as AV1 or VP9. Each side of the balance has its pros and cons. In general, every codec option aims to provide as much backwards compatibility as possible, at the risk of losing efficiencies.

The long tail of devices

The final piece of the puzzle is the devices themselves. Each manufacturer supports a subset of codecs according to their strategy. Among browsers, smart TVs, consoles and the likes, smartphones deserve special attention. According to Statcounter, along the last few years mobile video traffic has approached desktop consumption and nowadays it represents more than half:

When we double-click the two major OS for smartphones -where there is an exponential rise in video consumption- we clearly notice that IOS is opting to use H.265, keeping H.264 as a legacy for some of its most extended devices that need to be supported. In the case of Android, Google decided not to include H.265 and to use VP9 as the present codec, with AV 1 in their future scope. And so it happens with the rest of devices where every operating system, developer or manufacturer have made or will make their own decisions. If it wasn’t complex enough, we can’t ignore legacy devices and codecs and compatibility implications they carry.

In this entire environment there is one codec that can operate on virtually every device: H.264. This codec has two main issues, though. First one, H.264 includes some patents, so royalties have to be satisfied if used for business purposes. The second one is that the good old H.264 is, precisely, old. Its first release was in 2003, and although it has evolved with subsequent versions, its efficiency in terms of compression is limited. There are currently many codecs that provide far improved bitrate performance.

Quality expectations

In the last few years we have experienced a new paradigm in the consumption of online video, now demanding a high quality in streaming services. This has occurred because of platforms like Netflix and their pricing and content delivery policy, as well as an improved offering. Viewers demand more and more quality, becoming more and more demanding, as well as new experiences.

This is why, despite its ubiquity, H.264 is not a suitable choice as it is not sufficiently efficient to cope with the demand from the audience. In other words, to achieve the expected visual quality, H.264 wastes bandwidth when compare to newer codecs. For this reason, the most appropriate approach is a multi-codec strategy in order to provide the most suitable one to every device. This can be accomplished by using real time metrics and a technology that analyzes those metrics and takes the smart decision of trading between cost and quality in real time according to business needs.

How to manage?

Regarding the diversity of codecs, if anything, it will only get more complex. Each codec has its own advantages and weaknesses, and more likely than not, the community and the industry will find newer and better solutions to efficiently encode and transmit video over the internet. When it happens, we should learn not only how to make them coexist, but even to embrace the new options and get the most of each.

Don’t wait for a winner in the codec race. Instead, become the winner by using all of them based on your business rules and your user’s devices universe.