On December 28th, 2005 – a day which will live in anonymity – OSNews published an editorial of mine urging hobby and research operating system developers to implement Project UDI, because otherwise we (the small/ non-mainstream/ hobby/research OS community) would always wind up stuck with mutually incompatible sets of drivers for doing the same exact things. I also proclaimed that I would implement UDI for my own operating system kernel. Bad decision.As things turned out, the UDI specification turned out describe a small operating system which – at the expense of being sheer, painful hell to work on or with – made drivers binary portable. I wondered if I’d read the specs earlier while drunk (and I don’t drink)! So I decided not to implement the dirty crock. Still, I had promised about 61 people (a rough estimate of how many people commented on the article) in writing, and felt some obligation to fulfill that promise. I also still thought that the idea of a portable driver interface was a good one, so I began trying to construct a better driver interface.
This driver interface needed to be small, covering only things which all drivers and operating systems truly had in common; extensible, allowing people to add hardware- or OS- specific features in a way that didn’t destroy driver portability; and obedient to common sense, aiming only for source-portability in the first place. A new driver interface had to work without making implementors its beasts of burden. It needed do for driver interfacing what L4 has done for operating systems: provide a small, stable, and flexible core supporting standard functionality alongside an ecosystem of extensions and personalities.
I have tried to create such a thing, and I have named it the Extensible Driver Interface (EDI for short). EDI specifies the interface between a runtime – which abstracts operating system functionality – and a driver – which uses the facilities of the EDI runtime to abstract a useful device.
To abstract but allow extensions, a language-independent object system rests at EDI’s core. EDI runtimes and drivers create instances of classes = owned by themselves or the other party – and call the methods of those objects to interact. Any driver or runtime can transparently extend any class they’re already expected to implement, or add entirely new classes. Standard classes now exist for the runtime to reasonably abstract system hardware, and EDI also standardizes a class each to represent block and character devices. More devices abstracting driver functionality may come later, if necessary.
EDI also calls for a partial implementation of POSIX threading, and allows drivers to take advantage of multithreading. I’m very sorry about having to standardize this, but most systems implement some kind of threading and many drivers require it.
Note however, that for runtimes EDI standardizes only partial POSIX threading, the object system’s API and several hardware-handling classes. Runtime developers implement everything as they please. This means that the EDI runtime on Microkernel System X may actually be a stub which implements EDI by shuttling messages around. It can also mean that Monolithic Kernel System Y can implement an EDI runtime by loading drivers into the kernel and letting them call directly into real kernel code. Since EDI’s guarantees only source-portability of drivers, they become portable across kernel architectures.
EDI standardizes even less for drivers: only two classes. POSIX-like block and character device classes provide a baseline of abstraction over driver functionality: you can kludge just about everything into acting like one or the other. And when drivers need to provide a better interface to their functionality, they can. In a most likely typical case, adding a non-standard initialization method to the sample driver supplied with EDI will make it far more elegant. However, I leave it up to runtime and driver developers what kind of non-standard interfaces their software should accept and/or provide.
By standardizing as little as possible, EDI should give operating system and driver developers a common tongue to speak in while still allowing local variations and dialects. That has brought EDI to where it is today. You can download our (I’ve had much help, even if nobody else’s name is on the files) attempt at headers and documentation for an Extensible Driver Interface and take a look. The tarball includes C headers and documentation in both HTML and LATEX (thank you Doxygen). It is the fourth minor revision of the third major revision, and it’s time I let the internet public know about it.
Especially since it’s the internet public I hope will implement and use EDI. So in hopes of strengthening the community ties among operating system developers (such as exist on the net), EDI is copyrighted to me and placed under the GNU Free Documentation License version 1.2. Take it, change it, poke it with a stick, it’s free! However, implementors are also free (If words on the net can constitute legal permission, this does.) to license their EDI drivers and runtimes however they please. Just note that the few-guarantees design of EDI makes it simple to implement an EDI runtime which would log all calls to itself, helping to reverse engineer drivers.
Hey, if nVidia really wanted to they could “extend” their drivers to only run on Trusted Computing EDI Platforms or some such nonsense. That’s allowed, though improbable.
Since nobody fears the open-sourcing of serial port drivers, I’ve put the sample driver in the public domain. I only wrote a cheap, fast, but sufficient proof-of-concept, and feel no need to hold on to any rights.
So please, every reader with a word to say, come and comment! More eyeballs are good; hands and brains even better! I especially welcome eyes and/or hands already experienced in the development of systems software. You get a free smiley face. EDI needs people to improve it – and eventually to implement it in their own systems. Indeed, you can best give feedback by explaining what EDI needs to be for you to want it implemented or (better yet) for you to implement it yourself. If that sounds like a marketing pitch, it is. I apologize for my incompetance at herding cats, or in this case operating system geeks. My country has inferior potassium.
Note:
One comment which two people have made (and which I therefore address here) is “Why don’t you write the object system as a set of normal C++ classes?” The answer is language and compiler portability. Turning the entire deal into a set of C++ classes would then require the ability to declare member functions as imported from the runtime, which extremely few languages and compilers allow. Furthermore, attempting to solve that problem through a set of M$ COM-like abstract interface classes would tie the project down to a particular the implementation of virtual method tables for only one pair of language adn compiler. EDI should be only as difficult to implement as any other piece of an operating system, so it has no business requiring people to change their compiler internals in order to use it, even to make EDI’s architecture much cleaner. Besides, people use C for most operating system development, and C-kernel-hackers would have to hand-build such “virtual method tables”. However, all that matters to EDI now is a few function pointers passed on class declaration to EDI specifying the class’s methods and a means of creating and destroying objects of that class. Programmers can actually implement those X+2 functions in practically any language that compiles to binary. However, current EDI standards assume the C calling convention, since it is the most common. That assumption can change.
About the author:
Eli Zachary Gottlieb develops his hobby kernel named Glider in his Copious Free Time while trying to get into college. Yes, he’s in his senior year of high school. OSNews published an editorial of his, “The Reasons Nobody Gets Any Help” last year, which sparked his later quests into the land of device driver interface development. Flames and trolls get dumped into the Atlantic Ocean, so speak reasonably unless you want to pollute the Earth.
If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.
Wasn’t there an article somewhere about someone implementing a Solaris driver in Java?
Now that Java is going to be open it would be nice.
Well, except that even if Nvidia, ATI, Netgear, or whoever could write one driver that could potentially work with 20 operating systems they wouldn’t because Java can be decompiled pretty easily.
A driver written in Java is not yet a driver being cross-platform. It is bound to the interface provided by the operating system.
The Java driver infrastructure provides a set of abstract classes and interfaces that the driver developer needs to extend/implement; so, the infrastructure is very tied to the underlying OS, but the classes and interfaces provided not.
a day which will live in anonymity
What the heck does that mean? Are you sure you don’t mean “infamy?”
On topic, though, I would love to see this become a reality. A large part of why some new OSes don’t get off the ground is because they work on such limited hardware.
No, it’s humour
.
“What the heck does that mean? Are you sure you don’t mean “infamy?””
It’s called wit. I understand a lot of geeks lack it.
I thought it was kind of clever.
Don’t be so damned smug. It usually comes off making you look like a fool.
60-something people read it. Hardly anonymity, but certainly not especially witty.
The only nitpick I have is that the name EDI already has a meaning (Electronic Data Interchange – ANSI X12).
Maybe you should call it XDI instead? It has an ‘X’ in it, so it has to be cool, right?
“The only nitpick I have is that the name EDI already has a meaning (Electronic Data Interchange – ANSI X12).
Maybe you should call it XDI instead? It has an ‘X’ in it, so it has to be cool, right?”
Eerr… http://en.wikipedia.org/wiki/XDI, http://www.xdi.org etc. I guess any three letter acronym is used somewhere, as long as the context is clear, it’s no problem, imho.
Edited 2006-12-05 09:54
implement a set of APIs that mimics Linux’s APIs?
(Yes, Linux changes very fast, but then you’re not forced to follow absolutely all the kernels)
I have to admit, it works pretty well for Syllable. You don’t have to implement a complete Linux API either; Linux tends to offer a whole bunch of little macros and functions that tend to do very simple things, and you can actually do without a lot of it. You can still make larger structural changes to your kernel and still maintain a fair level of compatibility with the important parts of the Linux API’s. Something like a spinlock can only be implemented so many ways, or a set of atomic operations, for example.
Now that’s not to say that something like EDI couldn’t make things even easier, but I suspect the API’s would eventually end up looking pretty similar to existing systems.
There are a couple of reasons not to implement Linux APIs:
1.If you mean Linux’s user-mode API, I think it likely that device drivers will *implement* file system operations and should therefore not rely upon them.
2.If you mean Linux’s in-kernel driver API, it can’t be emulated or implemented because it doesn’t exist. Linux’s internal driver code runs on a continuous ad-hoc agreement between kernel hackers and driver developers, utterly without a specified API.
As we’re discussing driver development, I’m talking about the API between the driver and the kernel. Yes, Linux does have one. It may technically comprise every single function and table in the entire kernel, but it is still an API.
The thing is, you don’t need to emulate every single Linux kernel function. You just need to support some areas of functionality in a way that is close enough to the way Linux does them that you can port a driver from Linux with a few small changes. This is exactly how Syllable does it; there is a well defined driver API, but where functionality intercepts with Linux the implementations are similiar.
For example our atomic functions are almost identical, because there’s only so many ways to implement an atomic increment operation on IA32, so why not just do it the same way? Memory management, mutexes: all of the low level functionality, is similiar enough to Linux that it’s easy to port a driver. This sort of stuff doesn’t change often within Linux.
The few places where implementing Linux functionality would be cumbersome are covered by a simple system header file “linux_compat.h” which wraps Syllable functionality to present a set of functions and macros that work well enough to be used with a lot of Linux drivers.
So yes, it can be done and we’re doing it. Not only are we doing it, but it is in fact working out very well for us!
>As we’re discussing driver development, I’m talking about the API between the driver and the kernel. Yes, Linux does have one. It may technically comprise every single function and table in the entire kernel, but it is still an API.
Except that implementing Linux’s driver API also requires that drivers code for a monolithic kernel. Oops, there go all those interesting microkernel projects where the drivers can’t access symbols inside the kernel.
Linux’s driver API also can’t work on microkernels because you would have to code a runtime driver API layer that polled “kernel variables” in order to send messages to the real kernel. Probably technically possible, but very hard and not likely to happen.
EDI, in contrast, requires only functionality that a user-mode library could translate into messages or even implement itself.
Edited 2006-12-01 15:43
Except that implementing Linux’s driver API also requires that drivers code for a monolithic kernel. Oops, there go all those interesting microkernel projects where the drivers can’t access symbols inside the kernel.
I don’t see how this is relevent. The situation is identical on Syllable: a driver can not access symbols inside the kernel unless they are made available via. the kernel API (O.K, sure, the driver shares the same memory space as the kernel so you could hack your way around it, but that’s irelivent to this discussion)
Linux’s driver API also can’t work on microkernels..
Agreed. Getting it to work would be possible but you’d have to make a lot of compromises. Although doesn’t HURD try to offer some sort of Linux-like driver API?
I don’t see how this is relevent. The situation is identical on Syllable: a driver can not access symbols inside the kernel unless they are made available via. the kernel API (O.K, sure, the driver shares the same memory space as the kernel so you could hack your way around it, but that’s irelivent to this discussion)
Except that not all systems have the driver share memory spaces with the kernel. A good driver interface doesn’t impose design decisions on operating system developers.
Although doesn’t HURD try to offer some sort of Linux-like driver API?
At that point you might as well implement EDI, which will run fine on a microkernel with hacks and crocks to imitate drivers existing in the kernel’s address space on a microkernel, where keeping drivers out of the kernel is a conscious goal.
Furthermore, I don’t see exactly why Linux’s driver API is all that great. Imitating it on systems which don’t deliberately imitate Linux (as Syllable does) would have the same effect that porting C and Unix to every single hardware platform did: You wind up with too many mutually incompatible versions of the same thing, none of which were actually designed well for their task, all created in the name of portability.
That is what you get from standardizing on big interfaces.
Edited 2006-12-02 19:23
I didn’t understand the technical details of this entirely (slightly over my head) but could this be used to write a driver for 3d hardware that protects the trade secrets and whatnot, that can be combined with the other half of the driver – the OS half, so that a single binary object (per ABI and hardware platform of course) could be compiled into many OSs (like Linux, Windows, Haiku, etc.)?
It would be awesome if there could be at least some practical separation (even if it isn’t necessary or enforced) between the hardware side and the OS side (I assume that’s what this is all about, but I’ve been wrong before) that would allow the hardware manufacturers to guard their precious (mostly hardware related) secrets by releasing the compiled libraries that go with their hardware, while allowing the OS part of the driver to be Open Source, for all to port and poke at.
Even better if there is an easy tool kit that will help hardware makers write their drivers to run on all of Linux, Windows and Mac OS X.
I downloaded the headers, but they fail to decompress.
The archive seems to have had its line endings changed (been eaten by mingw or cygwin, perhaps?). I’ve put up a fixed version at http://www.8325.org/EDI-3.4.tar.bz2.
Edited 2006-12-01 00:07
Please, somebody managed to find the URL for the code?
Thanks
Edited 2006-11-30 21:22
Well it’s supposed to be on sourceforge.net.
http://sourceforge.net/project/showfiles.php?group_id=148100&packag…
However, I’ve tried several mirrors so far and the EDI archive is utterly corrupted. Ugh.
Same here, download is corrupted and appears empty.
I’m very interested in looking into this though. A few years ago I too read the UDI specifications (of which there are hundreds of pages of it) in high hopes of making my fledgling OS interact with others, but eventually discarded it as the author did after realizing the sheer magnitude of the specification and what all was required. The idea of someone creating a similar but simpler idea is very appealing.
Hiya all.
First of all, the archive is indeed corrupted. Sourceforge seems to eat every tarball I upload to them. I sent a new one to no avail. Anyone know a better hosting site for a broke high-school senior’s coding projects? It also ate my kernel release.
nick8325’s fixed mirror at http://www.8325.org/EDI-3.4.tar.bz2 works. I don’t know what I’d do without kind guys like Nick. http://www.mytempdir.com/1092730 will also work. I don’t know what to do about the official link in the article.
I hate Sourceforge. They never remember my password correctly, and now they require that everyone dos2unix (or the reverse?) source tarballs.
This is not at all what I hoped for the public introduction and release of EDI.
Edited 2006-12-01 05:43
is OSS, can be revitalized and there can be a licence where driver developers on UDI should give their source code. UDI is not for trade secrets, its for ease of development of OSS drivers. Imagine you have to write a different FireWire or BlueTooth stack for Linux/OpenBSD Haiku? Wasted effort. But UDI can help and is extensible. I think I should make a closer look.
Are you talking about Project UDI? The defunct Project UDI that got abandoned by its industry sponsors because it was as bloated as CORBA and requires implementation of almost a whole virtual machine?
I have looked at the specification, and UDI looks very C-based. How is this going to work with JNode (Java-based), House (Haskell-based) or any other OS that des not use the Unix paradigm of kernel, processes, address spaces, and so on?
It seems to me like this is going to hit the same trap os OSKit and OSLib did: Too specific to one way of thinking. If you want to restrict programmers so tightly, why not just write the final OS?
And this one too shows what I am talking about:
> EDI also calls for a partial implementation of POSIX threading, and
> allows drivers to take advantage of multithreading. I’m very sorry about
> having to standardize this, but most systems implement some kind of
> threading and many drivers require it.
C’mon, POSIX threading in a Haskell-based OS?
We need an aproach that is *independent* from the OSes where it is going to be used.
Maybe we could use a CORBA interface between the driver and the OS, so that the interface would be truly language independent.
😉
My “suggestion” was just there to show that if you try to provide an API which would work for also for a Java, Haskell OS, there is a real possibility that the result is an unusable monster..
Solving the driver problem for OS which can interoperate with C easily looks good enough to me.
Drivers can not be O/S-agnostic because each one is tailored to the needs of the operating system is about to run on…but perhaps if there was a central site will cleanly written documentation on hardware, the problem of drivers would not exist.
Thanks… that’s what I’m trying to say. There is already some hardware documentation, but it’s often leaky, and I know of no central point to get it all. Ralph Brown’s “interrupt list” is the best I know, but I think many new HW pieces are not described there.