Guest post by Linux 4.5 has been released. This release adds a new copy_file_range() system call that allows to make copies of files without transferring data through userspace; experimental Powerplay power management for modern Radeon GPUs; scalability improvements in the Btrfs free space handling; support GCC’s Undefined Behavior Sanitizer (-fsanitize=undefined); Forwarded Error Correction support in the device-mapper’s verity target; support for the MADV_FREE flag in madvise(); the new cgroup unified hierarchy is considered stable; scalability improvements for SO_REUSEPORT UDP sockets; scalability improvements for epoll, and better memory accounting of sockets in the memory controller. There are also new drivers and many other small improvements.
There are also new drivers and many other small improvements. Here is the full list of changes.
Nice. The copy_file_range() syscall looks interesting, especially for nfs.
Should be interesting.
…mandates a stable ABI for the kernel, then Hell Will Freeze Overâ„¢.
Don’t tell anybody, but that announcement is only 16 days away !!!
Git your hell parkas here:
http://www.amazon.com/Browning-Hells-Canyon-Primaloft-Parka/dp/B018…
Johann Chua,
He always says he doesn’t want to be locked into a fixed kernel API/ABI, which I understand. Personally I’ve always thought a good compromise would be to stabilize those for features between major versions.
Honestly, end users won’t be too bothered one way or another, but when I used to maintain some kernel code it was a problem that I needed to adapt to API changes almost every new kernel. IMHO, there was no good reason for it.
What I found especially frustrating was that I frequently had to fix problems with 3rd party out of tree modules (ie for aufs) to make them compatible with the linux flavor of the week. I enjoy working at the kernel level but I lack the resources to keep up with arbitrary changes that didn’t benefit me. If the kernel were stable for 6 months-1 year then I could have spent a day or so per year instead of per month or less.
Sounds like the BSDs. It’d be a nice way to go about it, except that Linus then has no control over what Red Hat, SuSE, etc will change anyway. It’d be a step forward, but I doubt it’d be enough. Red Hat, SuSE, and Debian at least, might honor the stability but others like Canonical will do whatever they damn well please regardless of what Linus might do. It’s a monster at this point.
I do agree though: it is the user and developer facing APIs and ABI that need to remain stable. What goes on internal to the kernel itself isn’t nearly as important as our interfaces to it are. The ABI is important however, not just the API. The need to recompile external drivers needs to end.
Edited 2016-03-16 14:30 UTC
Or stabilize between LTS releases… this might make backports to LTS alot easier as well if the API changes at the LTS release.
Edited 2016-03-16 15:00 UTC
Isn’t Linus’ (and the kernel community’s) view that things should be userspace as much as possible where the ABI there is stable?
And isn’t that ironically what most CS purists want – everything in userspace?
Are you suggesting turn Linux into a microkernel?
Views are one thing. Actions are another. Linus may have that view, but so far there has been no movement towards anything of the sort. Words are cheap. Actions count.
kwan_e,
Ah, the Tanenbaum versus Torvald debate
True CS purists would have been more likely to appreciate Minux. Linux was more pragmatic (at least at the time).
Edited 2016-03-16 16:00 UTC
I agree with what you wrote, but at the end I think you meant to write “MINIX” rather than Minux. The latter is a Linux-based operating system.
Haha, yeah.
I don’t remember ever hearing about “minux”, the linux distro, before:
https://sourceforge.net/projects/minux/
“ISO Image only 24 MB”
That’s tiny even by minimalist standards, my own linux build is on a 64MB iso.
Meh.
I remember booting Linux 2.4 from a pair of floppy disks. That included basic user space with a shell, ls, etc, and the hard disk installer.
I believe 2.0 could boot from one floppy disk.
24 MB is gigantic!
zlynx,
Yea, but those where the days when hardware conformed to (pseudo) software standards. It’s what made OS dev fun and relatively trivial back then. These days, if you want to support lots of hardware, you need hundreds of drivers. All of them do the same thing, but accomplish it differently. If we want to hard code driver support and reverse course on supporting decades worth of software abstractions, then we could go back to 1MB days (many indy operating system projects do it), but otherwise it’s not really practical any more.
I get you are being tongue-in-check though, and to that end: the guys today just can’t code like we used to
😉
Frankly, I dont understand this. All other OSes have stable kernel ABIs, without problems. For instance, Solaris have had stable kernel ABIs for decades, and ordinary software runs back to Solaris 2.11 unmodified. Oracle guarantees this.
Linus says the drawback with frozen ABI/API is that innovation will lag behind. That is not correct. Look at Solaris, it has brought us innovations such as DTrace, ZFS, SMF (systemd has copied this), Containers, Crossbow, etc etc. It IS possible to have stable kernel ABI/APIs and still innovate. Solaris is the most innovative OS today, everybody else copies or ports the Solaris tech. For instance, DTrace:
-Mac OS X has ported it
-FreeBSD has ported it
-QNX has ported it
-IBM AIX has copied it (“ProbeVue”)
-VMware has copied it (“vProbes”)
-Linux has copied it (“Systemtap”)
-Linux has ported it (“dtrace” in Oracle linux distro)
-etc
All the major OSes have copied or ported DTrace. It is the big killer. Not ZFS. And systemd is a copy of Solaris SMF, but done wrong. As BTRFS is done wrong when copying ZFS. It is possible to innovate with frozen kernel api/abi – if you know what you are doing.
The problem Linus Torvalds faced, when he released Linux as a university student after reading Tanenbaums book, was that Linus did not have experience of how to make kernels. So he tried this and that. It would be a mistake to freeze kernel abi/apis, as a unexperienced programmer. So he kept it open. Today Linus has the experience, so he should be able to freeze the abi/apis today. In the beginning he knew nothing, his Linux code was very amateurish.
Linus has said that trial and error is superior of design. Trial and error has evolved humanity from animals. So Linus has said officially that Linux will evolve by trial and error and become superior to all other OSes. “Linux has no design and will never have”, he said. It is better to experiment all the time, he says.
This has the drawback that large parts of the kernel is rewritten all the time. And new code is never stable. Whenever some part of the Linux code matures, it gets thrown out. So the Linux kernel is a constantly moving target. Making it very difficult for device drivers to update their drivers. This is why the Linux kernel model is broken, as you can yourself see from all forum threads “Ive upgraded the kernel and now my soundcard/modem/etc does not work”.
This makes LTS a broken concept too. If you want to upgrade or install new software, you likely need new library. The new library will trigger upgrades of other software, that requires new libraries, that triggers upgrade of other software. In an avalanche you have upgraded everything and left LTS. The only way to use LTS is to never upgrade nor install new software. That does not work.
Kebabbert,
Yes of course. Innovation and stable APIs are two different variables, one doesn’t follow from the other.
The problem with stable APIs, over the long term, is that they get stale regardless of whether they were innovative or not. Some parts of posix have not aged well at all, same goes for win32s and java’s class library, c++ libraries, android, etc. Previous assumptions no longer hold, yet because software depends on them now, legacy designs sort of ‘cement’ an API. What usually happens is an API starts to become redundant and bloated. Panzi’s example of copy_file_range() and sendfile() is a perfect example of this. Some historic API assumptions are notoriously problematic. Yet sometimes we are forced to do the best we can with the frameworks that were designed decades back.
I don’t have to tell you, there are definite downstream costs when things are never stable, which I can attest to myself. I don’t think Linus cares enough to do it, but he ought to compromise and keep things stable within major versions, lets say. So code written for X.Y should at least continue to work on X.Y+1. X.Y may not run on X+1. So kernel developers may need to update once a year or two, it’s much better than potentially every release.
Certainly. Every other OS than Linux have stable API/ABIs, even OS/2 had it. Linus Torvalds is alone with this driver model, and maybe he is smarter than every other OS hackers, but I doubt so if we look at all the Linux driver problems in every forum.
Regarding freezing the ABI/APIs, yes sometimes they become obsolete but the advantages outweigh the disadvantages in my opinion. If you are a good designer then you can make a good API/ABI that will stand the test of time. For instance Unix is from 1969 and it is so well designed it is the most successful OS model out there. Sure, Unix has it’s quirks, but the Unix designers knew their stuff. It is still valid today! So you can design well, if you know your stuff. If you dont, you do like Linus Torvalds, breaking everything all the time.
Regarding freezing the Linux kernel ABI/APIs in between major revisions, that is a very good idea, but Linus dont know where Linux is heading so he can not freeze Linux: “Linux has no design and will never have”.
Kebabbert,
Yea, I think it would benefit linux to have more planning, but ultimately it would require changes in the community that won’t happen if Linus refuses to go along.
I love how the traditional summary of the most important changes have become the “full list of changes”
What’s the difference between copy_file_range() and sendfile()?
panzi,
I kind of wish there was more attention to asynchronous use cases, who knows when these features will become available for AIO development. AIO in linux always seems like an afterthought.
Edited 2016-03-16 22:42 UTC
1
Browser: Mozilla/4.0 (compatible; Synapse)