Given that there is sufficient archive-wide support for zstd, Ubuntu is switching to zstd compressed packages in Ubuntu 21.10, the current development release. Please welcome hello/2.10-2ubuntu3, the first zstd-compressed Ubuntu package that will be followed by many other built with dpkg (>= 1.20.9ubuntu2), and enjoy the speed!
Sometimes, it’s the obscure changes that can have a big impact. This change will speed up the installation of .deb packages.
zstd, one of the few good things to come out of Facebook…
What problem is this supposed to solve? Just saving a little package installation time at the expense of increased storage and bandwidth use for all users and hosts? The vast majority of users must be installing packages far more often and in quantities than I ever imagined for this to make sense. It’s too bad we don’t have faster computers that can handle the monumental task of decompressing packages, and faster ram and storage bus speeds to they aren’t bogging down the process.
same problem optimizing a program’s performance by 0.01% does. if that is a hot codepath.
Plenty of people deploy system images, either via docker or automated installs, and there benefits of zstd are sizeable. You download a little bit more data, but decompression is vastly quicker.
See how it’s working on archlinux. System updates are now truly blazing fast. Even on hdd storage.
Who knows, this may translate to shorter downtimes in server maintenance, and energy savings in datacenters.
What matters here isn’t so much that zstd is fast (and it really is _seriously_ fast), it’s that it’s exceedingly fast proportionate to how good it’s compression ratios are. If speed of decompression was all that mattered, then we’d just be using snappy or LZ4. The reality is though that the size of a package file really matters because that dictates how much network bandwidth is needed and how much space is needed for the package archives (and how many packages you can shove into an installation CD), so packaging has historically tended towards favoring compression ratios over speed. DEBs started with gzip because that0s all there was, then moved to bzip2, then xz. Zstd gets compression ratios that are competitive with bzip2 and xz, but decompresses an order of magnitude faster than either.
The net result here is that package installation runs measurably faster but only uses a tiny bit more network bandwidth and the package archives only need a tiny bit more storage space. While it’s technically a microoptimization, it’s a tradeoff that makes sense in this context, especially for people who are doing lots of system installations (which is _very_ common in cloud settings) or working with particularly low powered hardware (which is becoming increasingly common as well).