The FreeBSD project has published its latest quarterly report, and there’s some good changes and improvements in there. First, there’s the project to allow FreeBSD to be built on non-FreeBSD hosts – Linux and macOS specifically. This project has made major headway.
As of September 2020 it should be possible to use the buildworld and buildkernel make targets to build a fully-functional FreeBSD installation on macOS and Linux hosts. We use this in our continuous integration system to build and test CheriBSD disk images for multiple architectures. I have also committed a GitHub Actions configuration upstream that takes approximately 10 minutes to build an amd64 kernel. This will ensure that changes that break crossbuilding from Linux/macOS can be detected easily.
Another major improvement is experimental support for little-endian PowerPC. Note, however, that this does not mean big-endian support is going away or being deprecated.
As of r366063, experimental support for little-endian PowerPC64, (PowerPC64LE) is available in -CURRENT for POWER8 and POWER9 machines.
There’s a lot more stuff to cover, so head on over and read the whole report for all the details.
Excuse the ignorance, but what exactly is the difference between big and little endian PPC. I thought PPC processors supported both?
So do ARM chips. It’s just that while big endian is prettier to debug (even though ides displays be and le transparently now), little endian have more advantages in the long run (different value size at the same address being the most obvious) hence several data format have adapted and were turned natively into little endian. Thus, being little endian prevents you from flip/flopping bits around all days long during time/power consuming endianess conversions. Big endian is pretty much mandatory in tcp headers, but nowhere else to be seen these days…
Kochise,
If someone is macho enough to be looking at memory dumps, I’d say they’re macho enough to do mental endian conversion, haha. This should be a non-factor.
Well, I think in a vacuum the technical benefits are mostly arbitrary. However if we aren’t in a vacuum and most of the data is big endian, then having a big endian architecture is more advantageous in order to avoid the flip. If most of the data is little endian, then having a little endian architecture is more advantageous in order to avoid the fiip. Note that this is the exact same justification for big and little endian – it mostly depends on the network effect (yes, that’s a pun).
The123king,
In practice, the byte order is rarely a consideration for local development. The compiler does it’s thing and the programmer doesn’t even have to know the byte order. Technically if you analyze the memory you can see the difference but just working normally with classes and structures you’d never see it. It’s like analyzing the bytes in a floating point value. Obviously there is a structure, but it’s abstracted away and we don’t usually care.
The big challenges occur when we start looking at copying structures to foreign systems since those systems may not be using the same byte order. This is a near universal problem in network programming and networking code has to take care to use the right byte order, often using standard helper functions to convert between host to network ordering:
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.bpxbd00/htonl.htm
As long as these helper functions are used consistently, they allow a programmer to write portable code.
The C socket API is big endian, which means programmers have to call functions like hton just to use the API, which is extremely annoying and IMHO a bad design. Obviously the network byte ordering is big endian, but it doesn’t mean the API running on the local host should. Most languages since C use a local byte ordering API. (And then there’s the amount of cruft, but that’s a different matter).
C…
https://www.geeksforgeeks.org/tcp-server-client-implementation-in-c/
Python…
https://steelkiwi.com/blog/working-tcp-sockets/
So, in the C version, you’ll see that the API exposes the ports in network byte ordering whereas in the python version the API treats them as local integers, which is far more rational IMHO.
This is just background info…but to the question of where PPC and other multi-endian processors fit in…well a program HAS to be compiled for big endian or little endian and in either case it will usually work the same way as long as standard programming conversions are followed.
But now there’s another quirk to talk about, what if the kernel is running a different byte endian than the user process? I believe some CPUs support this because they can change endian on demand, but if this is allowed, then all the integer bytes in syscalls will be in the wrong order. I don’t actually know how (or even whether) this is solved…? It seems like there are two obvious possibilities:
1. The kernel is able to check the endian mode of the process and has code to perform the conversion on its side of all syscalls. I don’t know whether any kernels actually do this? It would be a lot of work for relatively little gain and some overhead.
2 The kernel insists on a specific byte ordering. It’s compiled for big or little endian and expects the userspace process to comply with it’s byte ordering for syscalls. I think it would be possible to have a syscall shim to do this conversion mostly transparently, although it would need to be synchronized and maintained along with new kernel API developments. There are some syscalls where this is more complex because it isn’t a simple value that’s passed to the kernel but a whole structure.
https://www.man7.org/linux/man-pages/man2/poll.2.html
2.1. The shim could convert syscall structures in place on entry and exit from the kernel (This would probably work, but might hypothetically break something if multiple threads use the structure concurrently expecting values to remain constant throughout).
2.2. The shim could copy syscall structures on entry and exit from the kernel. This adds more overhead, but I think it would replicate semantics with less risk of usespacing seeing the structure in a modified byte endian state.
I’m interested to hear if anyone has any actual experience with this.