Netflix is working on a new video compression strategy, and they’ve published a very detailed blog post about it.
We’ve spent years developing an approach, called per-title encoding, where we run analysis on an individual title to determine the optimal encoding recipe based on its complexity. Imagine having very involved action scenes that need more bits to encapsulate the information versus unchanging landscape scenes or animation that need less. This allows us to deliver the same or better experience while using less bandwidth, which will be particularly important in lower bandwidth countries and as we expand to places where video viewing often happens on mobile networks.
The technical details go way over my head, but the basic premise seems to make sense to a layman such as I.
Isn’t that what Daala is using with lapped transforms already? Except it goes even further. It’s not encoding per title, it’s dynamic encoding for anything. I.e. compression applied more strongly to areas which are less noticeable and less so to areas which are more noticeable. Same idea is used in Opus for audio.
Edited 2015-12-23 23:02 UTC
Sorry but you are completely missing the point. Using some jargon, the idea here is constant quality instead of constant bitrate. All MPEG-like encoders (even JPEG) have a switch called quality or quantizer. Typically when you want to encode video at a certain bitrate, the quantizer changes from frame to frame because each frame has different difficulty for compression. What Netflix is doing now is that they do not intend to reach any given bitrate for videos but rather have a constant quality/quantizer across different titles. If this means that some slo-mo animation will get half the bitrate of a fast-action title, then so be it.
They are basically doing what video enthusiasts and movie pirates have been doing all along (two pass encoding – the first finds out, which frames need many or not so many bits, the second pass does the encoding).
Not even close. Pirate releases have fixed target sizes, aka fixed overall bitrates (for movies of the same duration)
Netflix is taking about different overall bitrates depending on the content/scene complexity even for movies of the same duration.
Not sure if they do 2-pass encoding, they can do the optimization they talk about with or without it, but even if they have two pass encoding, expect the variations between parts of the movie to be smaller than pirate releases because they want to avoid bitrate peaks, because they want to stream the videos over internet connections,which have more or less unchanging bit/s. In plain English,if they have 9mbits overall bitrate,they want it to be streamable over a 10mbit connection,and a 11mbit scene is not going to help (buffer underun). So even if they do 2pass, expect the variations to be smaller than what pirate releases have.
Edited 2015-12-24 00:23 UTC
Depends. If a low-bitrate section is followed by a higher-bitrate section the higher-bitrate section can already begin buffering while playing the lower-bitrate one and as such it wouldn’t matter if the bitrate went over the limit temporarily. The length of these sections, the buffer – size Netflix uses for streaming on client-side and the order of high – and low – bitrate sections dictate the ladder the encoder will use, ie. the budget allowed.
Daala and Opus use dynamic (variable) bitrate and instead aim at certain perception quality level, so how exactly is that idea radically different?
Edited 2015-12-24 02:18 UTC
You, Fishb8 above you and many others don’t seem to grasp the idea. It’s not the fact that Netflix is using CQP that’s the big thing here, it’s the analysis-part that’s the important part; they have a lot of constraints to take into account, like e.g. they already have a large amount of players out there that they have to maintain compatibility with, the section-over-section bitrate cannot exceed the budget except for buffer-length, the sequence of the high- and low-bitrate sections is a big factor in deciding the budget and so on.
It’s easy when you’re watching content from a local disk or whatever where you have plenty of bandwidth available, and it’s an entirely different thing when you are working with limited devices, limited bandwidth and limited transport-infrastructure — you need a budget and predictability.
All that functionality in conjunction with CRF has been available with ffmpeg+libx264 for quite some time. You can cap bitrates, limit bandwith variance, etc all while using CRF.
Also CRF (Constant Quality) != CQP (Constant Quantizer)
Perhaps the gist of the article is less about inventing / developing the functionality, and more about actually employing it.
shmerl,
I agree. The headline is a bit misleading in setting the tone for something new to the industry, but the article itself is basically saying the technique is new for netflix.I imagine that most video encoders who are meticulous about encoding parameters will have gone through very similar steps to experiment with getting the most out of bandwidth distribution.
And why not, there’s no need to waste bits on scenes that don’t benefit much from them. If instead they can throw additional bits at more demanding scenes without changing the average bitrate, then it’s pretty useful.
Edited 2015-12-24 03:32 UTC
What customers hear: “More bitrate for action movies, instead of providing the least bitrate they can get away with”
What Netflix means: “From now on, we ‘ll work on having the least bitrate we can get away with on slow movies too.”
Edited 2015-12-24 00:13 UTC
Seems like they’ve reinvented CRF (constant rate factor).
Surprise!
I’m not sure why this is to be considered so novel. libx264 has had CRF encoding since forever. It’s basically doing that analysis real time while encoding.
Cartoons and movies with a lot of solid colors are being re-encoded for lower bitrates since they don’t need to have the higher bitrate of a movie with action scenes and millions of colors. It’ll be fine, especially if it lowers my bandwidth usage without sucking.
Netflix is trying to figure out if they have to send you 5800k streams for everything, or if they can get away with sending you 2800k without you noticing. It will probably work.
Meanwhile, most modern people still believe that even 1000k is too much to bother with for music because they generally devalue audio data and overvalue video data.
I just wish someone would stream hi-res audio already. Tidal is close and Apple is rumored to be investigating it.
Proper stereo audio playback of good music takes between 800k-6000k bandwidth but we’ve been sold the lie that 256k is good enough.