AtheOS is a modern, free (GPLed) Operating System written from scratch in C++. A big chunk of the OS is POSIX compliant, supports multiprocessing and it is GUI-oriented (fully OOP). Today we are hosting an interesting interview with the AtheOS creator, Kurt Skauen. Kurt is talking about his views on binary compatibility in future versions, multithreading and the future of his OS in general.1. Especially after the release of 0.3.5, AtheOS received an even bigger popularity. It is amazing to see new software on Kamidake appearing almost everyday. How do you feel about this? Can you handle the “load” and responsibility that this generates?
Kurt Skauen: It’s nice to see that people are interrested in the project but I don’t feel responsible to deliver anything in particular. If giving away something automatically meant you where responsible to fullfill
all demmands from everybody who accepted the gift I don’t think there would be many free software projects out there. If people like AtheOS that’s great but if they don’t there is a lot of other OS’s out thereto save their day. If none of them fit they can always write their own.
2. AtheOS is surely still young. If, for example, you would have to break binary compatibility for the general good of the OS (ie. a switch to gcc 3.01), would you do it?
Kurt Skauen: Indeed. I expect to break binary compatibility every now and then. At this stage I will break all compatibility from one release to another if it is needed to get some new features or changes implemented. I hope to be able to keep a good overlap in the future though so that people get time to recompile/update their applications.
As a side-note I can mention that even the latest versions of AtheOS is mostly backward compatible with binaries built for the first released version of AtheOS. Just for the fun of it I unpacked the ATerm terminal emulator from AtheOS V0.1.1 and tried to launch it and it started and run happily on the current “development version” of AtheOS.
3. I see a whole lot of BeOS refugees in your developer’s mailing list. Most of them developers. How do you feel about this? Do you welcome the opportunity of lots of BeOS software may even be ported soon?
Kurt Skauen: Yes there seems to be a lot of BeOS developers on the mailing list. I have no problem with this as long as they don’t spam the list with requests for turning AtheOS into a BeOS clone. That have not been a problem so far. I believe most of them know where I stand and have accepted that. If some of them decide to port some of their software to AtheOS that would sure be nice. AtheOS lacks a lot of important applications so that would indeed be welcomed.
4. What I am going to say is not widely known among the BeOS people, but it is true among the Be engineers: BeOS has a real problem when it comes to multithreading. While multithreading is great on paper, it makes it extremely hard to program larger projects, especially if your project has more than 1 window to manage. Large projects and (especially) ports were doomed to failure and to non-existance under BeOS just because of (the good and bad at the same time) multithreading (most of the programmers out there have no clue how to manage multithreading correctly in their code). Now, my understanding is that AtheOS is based on the same principles regarding multithreading. How will you be able to overcome this problem for your developers, especially without a debugger/profiler in place?
Kurt Skauen: I can see that the multithreaded nature of both AtheOS and BeOS can be a problem for porting large applications from a single threaded environment (ie. just about anything else). Converting any piece ofsoftware larger than hello-world from singlethreading to multithreading will always be very hard.
I don’t see any big problems for large projects that is designed from scratch to be multithreaded though. All the required syncronization primitives are there and the threads that are “forced” on you have quite specific tasks. Then again I dont have any experience with writing large applications for either of the OS’s so there is probably unforseen issues. I will just have to deal with them as they come.
AtheOS already have ways to chicken-out of the multithreading though.
I do have first-hand experience with porting large projects to AtheOS and I do know that making a foregin application taking advantage of the multithreading is very hard.
The by far largest GUI based project I have ported to AtheOS is KHTML. The HTML renderer used in Konqueror and ABrowse (the native AtheOS web-browser). KHTML is not multithreaded and rewriting it to do efficient multithreading would indeed be very hard.
By default all “active” objects like Windows use their one mutex to protect the object whenever any user-code is run. It is however possible to tell the window (or any other objects running in their own thread) to use a user suplied mutex instead. In ABrowse I just create one global mutex that is passed to all browser windows thus forching all the windows to be 100% syncronized. This will give the impression of a single thread of execution and made it possible to run most of the KHTML code unchanged under AtheOS.
5. AtheOS has come a long way even if it is still conceived as “young”. And most of the work, is all yours, which I find quite amazing to come this far all by yourself. Please allow me to say that the only part of AtheOS that “standards” are not there yet is a Media Kit implementation. Any plans for a Media Kit?
Kurt Skauen: No, I don’t have much plans for this yet. It is still a bit into the future. I have not spent much time thinking about things like sound and live video yet. This sure are important things but there is many other things above them on my todo list yet.
6. A lot of people seem to have a bit of trouble dealing with GRUB. Is there an easier setup procedure in the works?
Kurt Skauen: Yes. I started to work on a graphical installer a long time ago that will automate the installation process. I would really like to get some more work done on the installer both because it would make it much easier to install AtheOS and because I personally find it a very interresting project. Unfortunatly it continue to be pushed down the list by other “must-have” things. Like the web-browser 🙂
7. Any plans to jump off to gcc 3.x? (which reportedly generates much better code?)
Kurt Skauen: Absolutely. I have not had the time to look at it yet but it will be included in AtheOS and the whole OS will be compiled with it as soon as I manage to get it to build under AtheOS.
8. I am sure that there are parts of AtheOS (as with any project) that were written a long time ago and you would like to replace. If this is the case, which these parts are?
Kurt Skauen: AtheOS have been totally rewritten several times already but now things seems to have stabilized quite a bit. I’m quite pleased with the current foundation.
The TCP/IP stack need a large overhaul but I don’t expect a full rewrite. There also is a lot of things that need to be cleaned up and fixed inside the kernel but there is no large modules that I would like to replace at the moment.
9. What are the cool things are you working for AtheOS 0.3.7 and for the future in general?
Kurt Skauen: The next release will be almost as boring as the previous. Mostly bugfixes and optimizations and very few new features. I have spendt a lot of time fixing bugs in and optimizing the TCP/IP stack. It now performs a lot better on connections with high latency. This used toscrew up the timing and kill the troughput in older versions.
The most interresting new feature is support for scroll wheel in the GUI. Catalin Climov sendt me a patch to the PS2 mouse driver that added support for IntelliMouse mouse wheels some time ago and I have now added support for mouse wheels throughout the GUI toolkit. Many of the standard GUI components (like list-views, multiline text views, scroll-bars, spinner controls) take advantage of the it without any extra support from the application and I have also added support for the mouse wheel to the terminal emulator and the web browser.
For the future in general the most imidiate project of some size is the desktop manager. I have a lot of plans for it and I’m looking forward to start working on it.
Hm! This is interesting indeed. Eugenia, could you coax JBQ to speak up on the problems with multi-threading under BeOS? You say there are serious problems with it, but what do you mean? Is it that BeOS itself has problems, or is it that having a pervasively multi-threaded OS creates problems that become really troublesome with large software projects?
As for AtheOS, I’m dying for it to start supporting sound, but I guess Kurt has his hands full at the moment and it’s dangerous to start building that kind of infrastructure without leadership from the ‘main guy’.
I don’t think there is anything technically wrong with BeOS’s multithreading per se; it’s just that lots of (Windows/Mac/Unix/etc) source code assumes that all windows will run in the same thread, whereas BeOS and AtheOS both require each window to run in a separate thread. This makes it hard to port said applications to (AtheOS/BeOS).
I don’t really feel like getting into deep details.
In a quick nutshell, BeOS makes it hard to use synchronous “communication” (function calls) between BLoopers of a same team, because the BLoopers lock themselves when dispatching messages, and because you’ll naturally want to lock a BLooper before calling a function on it.
Asynchronous communication seems to be the only way to go, but it’s pretty annoying on its own, and the fact that SendMessage can fail under non-catastrophic conditions (write_port failing) creates solutions that are almost impossible to deal with, if not entirely impossible.
Multithreading sounds cool, but it’s an unnecessarily complex beast to master, and the fact that it’s asynchronous by nature makes it fairly hard to debug, and creates some behaviors that vary a lot between machines. Plus, people learn and like to write synchronous code.
Just my 2 cents.
–jbq
My experience shows that multithreading isn’t good or bad by definition; It just depends on the developer who’s using it. I myself grew up with multithreading, and I constantly keep an eye on race conditions, reentrant code, semaphores etc. where I knew lots of very skilled developers with Windows heritage that don’t even know what semaphores are or what reentrant means. I’m sure they had a hard time programming the BeOS, where I tend to make heavy use of threads even in the Windows environment – and often I wish the commercail software vendors would, too.
Allowing for multiple threads isn’t bad in itself. There are a few cases where it makes sense, because it can allow to simplify code, to speed up code, to use multiple CPUs inside a same application.
However, there’s a great distance between allowing for multithreading and forcibly making every application perversely multithreaded at its core. Especially when you can’t guarantee that asynchronous messages are properly delivered. This is an issue already when writing new code, and can make it a nightmare to port applications from other platforms.
–jbq
Very nice interview! Keep up the good work on AtheOS, Kurt!
I can see the problems with forcing people to use multiple threads and how it’s hard when porting large single threaded projects. However, I must say, I was forced to learn a better way of designing my applications. In my applications, asynchronous messages mostly originate from user generated events, and I have never experienced any dropped messages. The only problem I have come across is app_server dropping mouse events because my code is too slow that deals with them. Thus you sometimes see views keeping hold of the mouse while you already released it. I suspect I should make sure MouseMoved() is a little speedier. I don’t think synchronous code would be any better off, because it would force the user to keep the mouse still once in a while to let the app catch up…. hehe.
If someone can find the link to the “diary” of the original JAVA port to BeOS, I highly recommend reading it, it tells a story of how JAVA ended up being/feeling much better then even when running on a Sun – by redesigning it to be multithreaded. Was it John Wätte doing the port? http://www.b500.com ?? Anybody?
-Stephan
Just a note about the state of multithreading … nearly all games (including ANY DirectX game that does aysyncrounous blitting and sound playing), and all well written internet tools for Windows use multithreading … otherwise you would not be able to click your email viewer to read a message while it was still downloading … or the AI in a chess game wouldn’t be able to compute moves, while letting you pick up pieces and type chat messages. Multithreading is helpful, if not necessary, for almost any complex multimedia or asyncronous communications application. For GUI apps there is almost always a GUI thread and a computation thread … for games it’s: Drawing, Logic (game code and physics), and AI. These are just about the bare essentials to keep decent behavior possible under heavy loads.
Well, losing a mouseup event is a fairly big deal in my book, and not something that might happen. What if, instead of losing a mouseup event (which is already annoying), you lose a message that you had to reply to within a certain timeframe (e.g. a BDirectWindow or BWindowScreen connection message)? When that happens, the app_server will instruct the kernel to kill your app. That’s bad. And that’s only one simple example.
As for Windows stuff, the little I have learnt from Windows when porting a Windows app to BeOS is that it seemed entirely possible to have an application open several windows and not have to use as many threads. Furthermore, the idea about multithreading for networkign is pretty invalid in my opinion. A well-written application will use asynchnonous I/O to deal with that kind of situation. Async I/O allow to keep a good level of responsiveness without having to deal with dozens (hundreds, thousands) of threads.
The problem with BeOS isn’t the fact that the kernel allows for multiple threads. The problem is the fact that developers are forced to use multithreading under BeOS, that they are forced to use the BeOS locking mechanism, and that various bugs and limitations in BeOS make all that even harder than it should already be. Sure, a computationally intensive application might (and probably will) benefit from spawning an extra thread. Sure, a game that needs to send feedback to different output devices might benefit from putting the game engine in a separate thread. But in many cases, it’s just overkill.
–jbq
Well, none of my apps got ever killed for not responding to a message, and I’ve written a couple, have used a vast amount more of them – and never seen a thing like this happen. The apps that I have written, especially eXposer, which I’m in the process of releasing on Kagi, _greatly_ benefits from multi threading. Forcing people to have multiple threads running, at least for every window they open, also means forcing them to rethink their design, if it somehow means trouble to them. And the results is better apps and a smoother computing experience for the user. That’s what all the people keep saying about their BeOS experience. I hope you’re not justifying our tragic loss (and maybe I’m a bit early on that) of our favorite OS with the attitude “BeOS is dead, oh well, it’s not that great anyways.” It is though.
Maybe you could give us a real world example how you got trapped into the “kernel kills you because you don’t respond to a message” situation while working on a BeOS port or native app.
And like I said earlier, if I’m too slow processing mouse messages, it’s not getting me anywhere, multi threading or not.
-Stephan
Well, one classic example of things that can go wrong would be to open a terminal under BeOS R3, and to run something that’d print lots of text, like “yes”. At some point, one update message can get lost, causing the windows to never ever update again. You only need to lose that message once, and the game is entirely over for your window.
When working on native apps, the worst case I ever saw was bdb. Some parts of bdb work such that a window sends itself messages, while the debugger stub sends more messages. Under heavy load, some of those messages get lost, coming both from the stub and from bdb itself. The result was, bdb “forgot” some symbols, making it unusable. Just in case you wonder, there are many cases in BeOS where a window will send a message to itself, the most obvious being the default behavior of a BControl.
When working on a port, the worst case I saw was a Mac application that assumed that it could do anything. “anything” included being able to call any method on any window at any time, including during an update. On BeOS, during an update, the window is locked, possibly twice or more if the update was triggered as a side-effect of a GetMouse. On BeOS, updates can really be dispatched to several windows at the same time. On BeOS, during an update, you are forcibly constrained to drawing inside your dirty region. Now, back to our Mac application which wanted to be able to synchronously draw in any window as a side effect of getting an update message in one of the windows. I let you imagine the mess.
And, to precisely answer your exact point, I did indeed get into situations where I would lose a message related to a BWindowScreen::ScreenConnected(), such that the app_server would kill my app.
Indeed, if you’re consistently too slow processing messages because you are just too slow, you’re not going anywhere. If you’re just too slow because the kernel is busy paging your code bak from disk, and you lose messages because of that, there’s a real problem with the OS. Guess what happens under BeOS.
–jbq
Why speak about be.. dont you have forums for that?
Or is it that Beos is so dead that nobody listens
If you want to compare..go on.
but dont start speaking bout Be here.
be is dead. remember!
Hermes… don’t count your chickens before they hatch! BeOS is not dead and if it was my computer is still running it and as long as the hardware holds up, BeOS will live on!!!
If Palm does nothing with BeOS or kills it entirely, the fact that BeOS can still be run on your computer will be utterly meaningless. Why? Because it still has it’s flaws/weaknessess, it still has no browser support that can go to the places you can in IE5, it still doesn’t have BONE, it still doesn’t have hardware OpenGL, it still doesn’t have the wide support of the public, etc. ad nauseum.
I jumped ship long ago and I am grateful I did, for look where we are today. I sure wouldn’t want to be still holding onto my BeOS R5 Pro CD today, no siree. I’m hoping Palm continues support/upgrades for BeOS, but if it doesn’t, we have few options but any freeware/opensource offerings in the taste of AtheOS, BlueOS, OpenBeOS, etc. All very poor replacements for the original BeOS, considering none of them are anywhere near as far along as BeOS is/was.
OpenBeOS…”we’re still in discussions”. *sigh* The most hopeful I’ve been in months… and I’m brought to the realization that no one is doing anything but TALKING ABOUT IT?!? Arrrgh! Whatever… at least I have an OS that I *can* use to do anything and everything I need/want.
Windows 98 SE… “I hate you, but you’re all I’ve got”.
…I can do everything I want with BeOS and AtheOS currently.
AtheOS has the browser, BeOS the rest.
If I really need a mainstream OS, for example to play Quake, I would boot up Linux. Win2k is installed, but I really hate to boot it and I rarely have a reason for it.
Most times I boot BeOS and AtheOS. Quite often, but that’s not the problem, I really like their fast boot times. That’s very cool if you do a lot of OS switching.
And it just “feels good” to turn on the PC. No endless wait anymore…
People keep harping on about BONE and OpenGL (no one seems to mention the new media kit..), but to be honest, it’s all out there if you look in the right places (..BeShare..)
BONE is not the answer, sure it looks pretty, and it connects to the Net on one of my 2 BeOS boxes, but it kills Tracker at boot time on the other (this could be related to multithreading ;-). It’s not finished, so releasing it will simply trash a load of peoples BeOS Intallations, and maybe make 60% of the population happy.
Not found OGL/Media Kit yet, but to be honest, I doubt I’ll bother if BONE is anything to go by.
If anyone knows of a sollution to Tracker dying, please let me know. I’m going insane!! (first it was KDL because of the PCTEL Model I have on the mobo.. solved that one by deleting the driver 😉 You end up with Tracker running for about 2 seconds, and then you get an error window telling you something along the lines of Tracker doing something illegal. If you debug, it mentions some ‘init????V10( *void, *void)’ call being bad. That’s totally from memory btw.
At last a free alternative to Windows that’s all in one piece… by that I mean not in the extreamly complex mess that is Linux.
Are the problems with communication between BLoopers an unavoidable part of a “pervasive multithreading” environment, or do they reflect some avoidable limitation in implementation? For that matter, has any other OS tried to use threads so heavily, and if so have they run into similar issues?. I’m pretty sure that BeOS is unique among OSes that have actually been available to consumers in this regard, but maybe an experimental OS at some university has addressed this.
It’s an interesting discussion – I had previously bought into the idea that pervasive multithreading is great, and it is interesting to read these caveats from a really knowledgable engineer.
Well, having multiple threads will always require some level of communication between them, there’s no way around that. If the threads were totally independent, there would be absolutely no point putting them in the same applications
First, let’s assume that threads are meant to maintain or improve responsiveness. From that point of view, solving a problem by “polling” (i.e. looping until a condition is met) is unacceptable, as it causes both CPU usage and delays.
In any multithreaded system, data protection is an issue : because threads are asynchronous, some form of data protection must exist, to make sure that two threads don’t access the same piece of data at the same time. As you explore real-world cases, you will quickly notice that getting multiple threads to access the same data at the same time causes some locking problems, quickly leading to deadlocks. A good way to deal with that is to use asynchronous communication between threads, which is what the BeOS messaging (BLoopers) tries to achieve.
Asynchronous messaging, though, is fairly tricky to deal with. As a BLooper could be receiving messages from pretty much anywhere, and as the only way you can communicate without deadlocking is through asynchronous messages, the developer needs to keep track of the state of all the “communications” that a BLooper is currently processing. This kind of code is hard to write, and extremely hard to test thouroughly, as it is asynchronous by nature, and the number of possible states grows exponentially with the size of the data the BLooper deals with.
Practically, BeOS makes all this even harder than it needs to be. BLoopers have a very primitive locking system (a single lock for the looper, automatically acquired before dispatching a message). For BWindows, it is even worse, because the same lock is used to protect the session between the interface kit and the app_server (which means that drawing in threads other than the main BWindow thread has some interesting locking constraints).
Also, and that’s probably the worst issue with BeOS, the kernel doesn’t guarantee that it’ll let messages go through, and the application kit doesn’t do anything to work around that problem. A messaging system where messages aren’t guaranteed to go through is a nightmare to deal with.
And, finally, there’s a big reason why I think that the BeOS “pervasive multithreading” is just smoke and mirrors. Developers know that, to keep an application responsive, BLoopers should all stay responsive, which means that they should not eat too much CPU power and quickly process and respond to messages. If BLoopers don’t eat too much CPU power, they are not competing among themselves for CPU power, even on a multi-CPU machine, which means that they don’t need to be running in multiple threads.
Furthermore, the “responsiveness” that people are talking about is responsiveness of the UI. That can be split in two parts : the speed at which applications respond to input events, and the speed at which they redraw when windows are moved around. Big surprises here : there’s usually only one mouse, only one keyboard, such that cases where several applications have to respond to input events at the same time are marginal, unusual cases. As for redrawing, all the redrawing events are sent to the graphics chip, which can only deal with one acceleration at a time anyway.
I personally believe that a good level of responsiveness can be achieved with single-threaded applications, through the use of signals and asynchronous IO.
Contrary to what people believe those don’t make the applications harder to write, especially because people aren’t forced to use them.
In my opinion, seen from an OS designer’s point of view, the BeOS way is a fairly lazy way to write an OS : force the developers to handle asynchronous events through threading, such that the core services of the operating system only handle synchronous requests.
–jbq
The communication problems JBQ mentioned does not have anything to do with multithreading. It is just that BeOS have a to small limit on the message queues and does not block waiting for space in the queue but AFAIK throws away the message if the queue is full.
This problem would arise if the GUI was single threaded aswell. In fact if the GUI was single threaded it would happen all the time. Take a look at a regular Windows or X11 applications. Whenever they have something to compute they just entirely stop responding to events of all kinds often for an extended periode of time. If windows had the same limitiations in the message passing mechanism as BeOS most of the applications would have died whenever you moved another window over it while it was busy processing something.
Seems like JBQ did beat me at responding to this
I was referring to JBQ’s first post not his last in my previous post in case someone was wondering (if your still confused thats ok to
.
Well, Kurt brought in some extra details.
BeOS can block on a port write when the port itself is full (each port can only hold a fixed number of messages). The real problem is that the BeOS kernel allocates the data associated with each port message from a separate (small) memory pool, and write_port will fail if that pool is full, returning B_NO_MEMORY. Annoyingly, this “feature” doesn’t seem to be documented in the Be Book. Filling up the memory pool with have some very negative effects on the system, usually causing the app_server to deadlock.
Even if write_port could be guaranteed to block forever and never return any error code, the problem wouldn’t really be solved. Applications wouldn’t lose messages any longer, but would deadlock instead.
Kurt is right indeed when saying that the problem has nothing to do with multithreading. It would (and does) also occur when communicating between single-threaded applications. The fact that BeOS forces multithreading onto developers makes that bug much more annoying, because developers will have to go through that codepath even for messages that stay inside the application, and because a developer usually won’t expect a message sent inside his applications to get lost, especially when no documentation warns about write_port() (and thus SendMessage() and PostMessage()) possibly returning B_NO_MEMORY.
So, let’s try to summarize :
-One of the highest priorities for a message-based OS should be to make sure that no messages should get lost, and steps should be taken to avoid that case.
Specifically, the kernel should only lose messages under cases of “catastrophic failure”, and one of the available options in that case is to kill a random application. BeOS doesn’t consider that issue as a priority.
-BeOS forces multithreading on developers, putting them in situations where they have to extensively use messaging, implicitly or explicitly.
-Combining those two problems makes BeOS an unnecessarily tricky platform to develop for.
–jbq
Combining those two problems makes BeOS an unnecessarily tricky platform to develop for.
For professional sized development maybe, but for me I find it *clean*.
Ditto here.. I’ve done a bit of Swing coding (single-threaded, bleh), win32, X11, and I know I’ll neer return back, single-threadedness
is just too annoying. BRS for example would be terribly annoying to code with a different framework. To concur with some of djaybee’s points though, I only use asynchronous messaging and find it a natural thing to do. I dont have to keep track of states so there’s no “explosion combinatoire” in my case.. Bottom line (at least in my case) is probably, the BeOS model encourages you to copy data from one thread to another, just because as JBQ pointed out otherwise acessing data yuor thread does not own leads to problems. The trick is to design your app so that there’s not too much data copying of course, to save time and memory. Then your app is well designed. Note: I’m not trying to say people advocating otherwise just are poor app coders :-), it’s not my point, especially with a well known alpha-hacker like JBQ, just trying to track down possible isundersandings and giving my (humble) point of view after designing a couple dozen more or less interesting applications for the last 3 years under BeOS.
As to Hemos: nice troll, but try harder to be original.. we have enough of this type of gremlins on be forusm already.
Forgot to mention the one thing I really really agree with though: not being sure that a BMessage will be delivered is something that keeps me up at night 🙂 Well almost.. it never happened to my apps AFAIK but the mere fast that PostMessage() returns a status_code is bad IMHO (I don’t even take taht into account when writing code or it would be unecessarily complicated/blaoted) and other discussions of a few montsh/years ago like “what happens if you delete a BHandler and create a new one at exactly the same address, or a BMessenger” prove that the Be API is not perfect of course.
Still, the fact that it’s light years beyong anythign else (that I know of, that is) speaks volume about the suckyness of the CS industry nowadays. /end of rant. So nothing else than Be at home for me in the forseeable future. Or maybe sci-fi books, but certainly not Windos or QNX.
Multithreading is good. Period. If the OS has bugs in its support for it, that is bad of course; but this is not an argument that MT is bad in itself. The fact that BeOS “forces” it onto the app developers is also a good thing, or all the hordes of new programmers would not have used it. Of course it’s harder to use than coding a single-threaded app. It’s also probably simpler to code in an unprotected, DOS-like OS where you can just do everything; and I’m sure lots of “hard-core” coders have cursed the fact that they weren’t allowed to do direct I/O to hardware when they first coded on a protected OS.
The single most annoying thing when I (have to) use Windows is apps that block completely whenever they need to do something. You then can’t even move or resize their windows. I’d much rather loose a message or two.
i indeed too think that forcing to use a thread per window is not a good thing. on the other hand it can be quite difficult to program a thread per window in an other os, but that would normally not be necessary. i would rather use threads for “lengthy operations”.
also the default behaviour of bcontrols to send a message when invoked without containing the new value made some of my porgramming hard. by the time i would get that message and read the value from the bcontrol that sent it, the control could have been changed by another thread and in my application this wasn’t what i needed.
what i do like about the beos is its scheduler with realtime and non-realtime scheduling. also system_time() is very accurate, much more accurate than most operating systems provide. it would not be that usefull if there weren’t also snooze_until(), which is super-accurate. i think this is because the kernel dynamically programs the pc clock chip?
anyhow, it’s not all bad, but without updates it’s not going anywhere either. they could switch to gcc3 breaking binary compatibility and at the same time rewrite the interface kit to support single threaded applications, but that is not going to happen.
— martijn
Since this discussion started out as an interview with Kurt about AtheOS, I hope that the conclusion is “make asynchronous messaging in AtheOS as fail-safe as possible”. Maybe extensive use of threads isn’t quite all that Be and BeOS made it out to be, but it would be nice if at least one OS keeps carrying the pervasive/perverse multithreading model forward to see where it leads.
JBQ-
Wow, thanks for all of you comments. Makes me rethink jumping on the BE Development bandwagon, as some of the projects I have in mind might be a little too difficult to implement from the sounds of it.
I do have one quick question though: UNIX and Linux are also multithreaded development environments, if I am not mistaken, so how has the problems you speak of been delat with by developers for those systems? Seems to me, with the surge in GPL’d sftware, that there is aloit of development going on there, so there must be a workaround.
Thanks
JW
unix traditionally wasn’t multithreaded. and most linux programs are not multithreaded either. when talking about porting single threaded applications to beos, these often are unix/linux applications. there is a big difference about multiprocessing and multithreading.
i’m not sure but i think linux threading support is even exceptionally bad and it doesn’t support true kernel threads, but i may be wrong about that.
— martijn
cedricd wrote : “I don’t even take taht into account when writing code or it would be unnecessarily complicated/bloated”. See, we agree. That’s where part of the real problem really lies. That “unnecessary” code becomes necessary if you want to charge hundreds (or even thousands) of dollars for your code.
cedricd also wrote : “it never happened to my apps AFAIK”. Well, maybe you didn’t test your apps enough. It’s fairly easy to put BeOS in a situation where problems happen. Getting write_port to return B_NO_MEMORY only needs 5 lines of code in another app (but running that kind of stress usually kills the app_server). Getting write_port (PostMessage/SendMessage) to block or timeout is a bit more interesting to achieve (the easiest way is to script-flood one of your loopers from another application). If you’re really serious about testing your code, this is something that you should try to do.
Sander Stoks wrote : “Multithreading is good”. Sure it is. Asynchronous code is good. Having multithreading as an option to write asynchronous code is good. Making it the only option is not. Hordes of new BeOS programmers have had to use multithreading without understanding it, giving us the Baxters and Scoobys of this world. Those beginner programmers should have the option to code pure single-threaded synchronous code, and then, when they become more confident in their skills, switch to a more asynchronous style of programming with either multithreading or asynchronous I/O (or even signals).
If a Windows application stops dispatching messages, it’ll become unresponsive. Guess what? the same thing applies under BeOS. And I equally dislike a BeOS application that freezes on me as I dislike a Windows application that freezes on me.
As a developer, I really don’t feel like living with the fear of losing messages. What about the message that says that the document must be saved? Your user won’t be happy. What if your photoshop loses the message that transfers ownership of a 100MB undo buffer? You leak 100MB of RAM. What if you lose a B_QUIT_REQUESTED message? the shutdown process gets stuck. What if, simply, your user clicks a button and nothing happens? Your user gets unhappy about your app, thinks that it’s unreliable.
martijn sipkema wrote : “bcontrols […] send a message when invoked without containing the new value”. I thought they did. Check for a field called (if memory serves right) “be:value” in the message you get from your BControl. Or, in doubt, to a BMessage::PrintToStream() on that message and run your app from a terminal.
OK, to be clear : I’m not blindly against multithreading, I am all in favor of tools to write asynchronous code, and multithreading is one of several tools available for that ; multithreading is needed on SMP systems, but multithreading also can’t take care of everything (try to run an IRC server with one thread per user).
However, I am convinced that one thread per window is overkill, especially when messages can’t be guaranteed to go through. I wouldn’t wnat to have to sign off on apiece of code in such an environment.
–jbq
“However, I am convinced that one thread per window is overkill, especially when messages can’t be guaranteed to go through.”
So what is your real opinion about “one-thread-per-window” if messages ARE guaranteed to go through?
What would be the big problem in this case?
I guess most of us are very interesting in hearing why forcing multithreading could be bad in general (although you’ll never know the definete answer like the microkernel vs. monolithic kernel discussion)), instead of why it is bad in BeOS.
Especially, cause this is the AtheOS thread. 
However, thanks for the informations (same to Kurt)!
Well, if messages are guaranteed to go through, it becomes easier to write code, but it remains convoluted.
Writing code with multiple windows that need to interoperate still requires to go through some asynchronous channels, and the code to do that is still harder to design, write, test and validate than purely synchronous code ; the benefits that you might get (if any, I still need ot be convinced about that) are in my opinion unlikely to outweigh the extra cost of development.
–jbq
While I really like to read more about BeOS, dont you think that it is a bit impolite to discuss something only related to BeOS under a story of AtheOS?
alencious: i don’t think it is unrelated and it is
interesting. and i for one do not think jbq’s posts
were impolite. i don’t like yours however. just my
opinion.
jbq: i’ll have to look if bcontrol includes a be:value
in it’s message. it would make sense. but i don’t
remember seeing it in the bebook, could be wrong though.
i do want to add that perhaps one of be’s larger problems
is also the documentation. it is not at all good.
— martijn
Alencious : well, I was asked a specific question, I answered it, and the thread went on… And I seem to understand that what we are discussing could be fairly relevant for AtheOS anyway
martjin : I’m pretty sure that BControl will copy its value to the message. One of the aspects of BControl is to keep track of that 32-bit value for you. And, yeah, the documentation has pretty much always been an issue with BeOS. It was good in the beginning, but I guess that the documentation team couldn’t really catch up with the engineering team after they did the paper versions of the BeBook (PR2 and R3 times).
–jbq
martijn asked about linux threading
Linux includes a very versatile process clone system which permits models all the way from traditional Unix multi-processing to an extremely fine threading not normally seen in a conventional OS (and probably useless to 99% of users)
and of course in any combination.
GNU/Linux uses clone() to provide POSIX threads using libpthread, they are used by projects like Mozilla and modern versions are sufficiently robust to support even our exceptionally large and complex image processing programs on a 6-way named brand compute server.
JBQ is right, I sometimes use threads to separate “responsive GUI” from “CPU crunching backend” but adding another thread for every stupid window I create would not be… sensible.
It does seem like a neat idea when you draw it on a white board though, that’s always the trouble with “start from scratch” OS design, you won’t find all the wealth of past OS implementation experience in any book.
(That’s why Microsoft hired Dave Cutler, and why I was surprised to find out that there wasn’t a famous lead kernel hacker at Be Inc.)
Yep, I’m behind Nick Lamb on that one, even though I didn’t want to speak first, not being a Unix guru (but, I mean, not at all). Linux provides a number of methods to write asynchronous code, including threads, select() (which allows for asynchronous IO) and signals (which allow for synchronous delivery of asynchronous messages).
As for “lead kernel hackers” at Be, well, I’d say that people like Erich Ringewald (co-author of the MacOS multifinder) and Bob Herold (former leader of the Mac firmware group, IIRC) would definitely qualify. And what some other people have done at Be should now also qualify them for the title of “famous kernel hacker”.
–jbq
While I’m no fan of multithreading on UNIX, oddly I love the way BeOS and Atheos
do it (know only what I read about Atheos, but looking forward to learning more
if I can line up the hardware.) I hope people who are new to these systems won’t
be unduly put off – you need to pay some attention to the concurrency issues, but
this is not outside the reach of people with normal intelligence. (And according
to the interview here, you can serialize Atheos GUI threads with a locking gimmick,
if concurrency isn’t your bag.)
Lately I have been fooling around with language level support for asynchronous
programming. Nothing much to report yet, but the place to look for this kind
of thing seems to be “functional” languages like Haskell (e.g., O’Haskell’s
“reactive object” feature, also see Erlang, etc.) It would be interesting to
integrate something like that with Atheos’ API, for a more elegant way to tackle
asynchrony than the usual C++ misery.
First about the meta-discussion here: I haven’t seen any AtheOS message that would have been flooded/hidden under an hysteria of BeOS-related messages yet. At all. All I see is a board that is animated only by BeOS folks and where the (supposedly) AtheOS folks do not contribute technical stuff but only meta-discussion. And as plenty of people said, this is relevant to AtheOS as well, given how similar (cough cough) their APIs are.
JBQ, I don’t have the “silver bullet” argument about how though PostMessage returning a status_t is something that annoys me, it’s not something I perceive as inherently bad about the OS; but maybe I can try a comparison here:
The new operator. Do you always #include new.h, turn off the exception throwing, and multiply your code size by two by adding tons of success-checking lines?? I know I’ll probably never do that. I wouldn’t know what to put in the if(!ptr) block, simply because i don’t want that to happen, period. If that happens I prefer to SEGV or whatever and have my app exterminated.
Same goes for PostMessage: _theoritically_ it can fail just like new can fail, and even in real life is using some special code around write_port, as you explained, but my apps don’t exhibit this behaviour and I don’t want them to.
As to pervasive (“perverse”? Lol. tats a good one) multithreading, I fear I’m hell bent on the same side as Sander here and cannot agree to disagree with you
To drive the point home, it seems to me the reason why win32 apps suck ass reg. responsiveness even though Window sNT _does_ implement multhreading well is that MS _never forced MT on developers_. So if BeOS was the same people would be lazy and don’t bother with doing an “optional” enhancement and the state of things would be the same as on Windows. Devs are lazy (wll known fact).
The blame would be more on lack of evngelizing this new “paradign”, or lack of emphasis in the documentation as to how you have to code good MThreaded code, or lack of sample code that gets devs started on this topic (except the super famous BMessage’d boucning balls), or lack of good development tools that would make it easier, but not the pervasive aspect IMHO. But what do I know… Just that when I made the transition from Basic to C or C to C++ I felt roughly as much “increase in expression power” as when I eventually master decently MT’ing
after doing only some single-threaded code.
An area where single-threadedness makes sense (or at least is used in 99.999% cases) is games though. We still have to see a follow-up to Q3’s dual threadedness :-)… But a game takes over the screen and interface by definition, and re-invents the wheel (custom interface, custom widgets ..etc).
I still think the “fear of losing messages” which JBQ keeps talking about is related to a buggy implementation of the OS, not of MT per se. I still haven’t heard why MT is inherently bad. And “incapable programmers can’t write good programs” doesn’t qualify.
Perhaps having two threads for each window is overkill, but I think it’s absolutely silly to have things like moving the window go through the main message pipe. Look at Windows – there are apps there of which I can’t even move the windows while they’re busy. That is totally unacceptable to me.
About “safe” MT, I am still pondering on the best paradigm. I am leaning towards using handles instead of pointers; sort of like only using BMessengers instead of calling win->PostMessage() directly. The fact that you keep pointers to other threads’ stuff around is the number one source of problems, in my experience.
Someday, I’ll write my own OS (like everybody else is doing nowadays 🙂 and it will definitely use “perverse” multithreading.
[well, I’ll have to re-type this message, my laptop died on me… because of a deadlock in the app_server. gotta love complex multithreading code.]
I’m not arguing against multithreading, I’m arguing against the way it is forced onto BeOS developers, by making BWindows spawn threads, by having a messaging system that’s too unreliable, by not providing services to make developers’ lives easier. Multithreading is a good tool for certain problems. It is used under BeOS to solve problems that it isn’t well-suited for, and sometimes to solve nothing at all.
cedricd :
-Well, you’re the living proof that devs are lazy, and that an OS designer should make sure that the OS is as easy as possible to write code for. Not checking whether a memory allocation failed and letting the application crash will cause your users to lose data. In many cases, application crash with data loss is considered “no-ship” priority. The support costs associated with even one such crash will far outweigh the cost of writing proper error-checking in dozens of places as you write the code.
-write_port, under BeOS, is far more likely to fail than new, and it is quite easier for an app to make all write_ports to fail system-wide than to make new fail in a single other app (other than, allocating all the system’s RAM, which BeOS doesn’t handle graciously either).
-BeOS applications, just like Windows applications, won’t behave well if you start spending too much time processing a message. It’s a fact of life. Costly processing should be deferred to other threads. Network activity should be made asynchronous. That is true under BeOS, under Windows, under Linux, under MacOS. And it’s unrelated to the number of threads you have to run each window : when a windows freezes after the user clicks a button, the user is unhappy about it.
-Your mere request that documentation should explain how to write multithreaded code shows that it is not an obvious task. There should be an obvious way to write correct code, and optional ways to make it better or faster. BeOS doesn’t offer obvious ways to write a correct multi-window app.
-Some games other than Q3 are multithreaded, even on Windows. I know, I ported one of those from Windows to BeOS. Guess what : most of the bugs that were left in the shipping Windows version were threading/locking bugs.
Sander :
-Yes, the fact that BeOS can fail to deliver messages is an implementation bug in the OS, not an inherent problem of multithreading. That was the whole point of the question that was asked in the interview, and of the followup questions.
-If your API is such that developers that are capable of writing correct code under other OSes are incapable of writing correct code for your OS, maybe something is wrong with you, not with your developers.
-As I said, I can have the same argument about BeOS that you have under Windows : “Look at BeOS there are applications that can’t close their windows while you hold the mouse button down.”
-I know a very good MT paradigm, provided by BeOS itself (heh, it also has a few good things). BLockers.
–jbq
JBQ,
there is facts, and then there is how you attribute them. Two different things. I think your argumentation really lacks, not because you are giving false facts, but because of how you describe them with phrases like “worse yet” or “smoke and mirrors”. The smoke and mirror passage is especially crappy, because BeOS’s multithreaded windows _do_ make the user experience more fluid and responsive. It does so, because access to the _one_ accelerating graphics chip is distributed among windows much more evenly. Frankly, when I use BeOS, Linux, MacOS and Windows in comparisson, I simply don’t see your “BeOS is a lazy designed OS” having any real world basis.
-Stephan
stippi : well, I like and dislike your post.
I like it, because it is somewhat true that my previous posts go along the lines of “trust me, it sucks”. I’ve had enough first-hand and second-hand experience with issues porting and/of developing code on BeOS to know that multithreading can be an issue. A few names just for fun : SimCity3000. Cycling74 Max. Macromedia Flash. Mail-it. Nuendo. bdb. In every one of those cases, the BeOS multithreading (multiple message queues with multiple messages being processed simultaneously) caused more problems than it created solutions. So, I think that you can indeed “trust me on that one”.
I don’t like it, because I don’t think that your argument holds very well when it comes to multithreading windows. For 3 reasons :
-At some point, Be experimented with that problem, one of the paths that was followed was to make all BLoopers share the same BLocker. It created some obvious deadlocks (intra-team inter-looper synchronous messages), but many apps actually worked, and responsiveness was unaffected.
-The graphics chip can’t be shared by multiple threads. As a matter of fact, access to the hardware acceleration is traditionally restricted by a mutex (and, before you say that it’s inefficient, well, it doesn’t matter : filling the FIFO isn’t the limiting factor). Access to the framebuffer is usually not restricted at the software level (except for lame-ass cards), but the hardware takes care of it. Experience shows that accessing the framebuffer from several CPUs at the same time doesn’t improve the throughput (it could even have the opposite effect, especially with fast PIIs and above combine with fast graphics cards).
-One of the big changes in the app_server between R3 and R4 was to make the locking inside the app_server more fine-grained. Interestingly, this made the machine feel quite slower. Owners of slow BeBoxes remember that time. I myself had my BeBos running R3.1 until fairly recently. As it turned out, the locking until R3 was so gross that no two app_server threads would ever try to touch the graphics card at the same time at all : the rendering part of the app_server was essentially single-threaded. And it was fast.
OK, maybe the word “lazy” wasn’t well-chosen. But many parts of BeOS never got the polish that would have been needed to make BeOS a better OS. That wasn’t caused by laziness, but rather by lack of time, the list of tasks as hand being always so big that in many cases it was chosen to ship features that were 90% functional instead of spending twice as much time to reach 99% functionality. I would put the “one-thread-per-window” issue into that same bag : it just made it easier to write the app_server and the interface kit that way. And it is “good enough” for most situations. I could give other examples of issues where a “90% solution” was chosen, BeOS is full of those.
–jbq
JBQ,
You seem to make a few assumptions that dont really hold. When I first implemented memory-protection in AtheOS I had to make a server that dealth with the GUI (or put it in the kernel but that was never an option). The first version of AtheOS with memory protection (each process running in a separate address space) did not have support for multiple threads per process so the server had to be single threaded.
I later added support for multithreading and rewrote the server to use one thread per window and boy did the responsiveness increase! Even with the rude locking schemes I had in the first version of the multithreaded server the sensation of speed-increase it gave was stunning. This was of course long before AtheOS got support for SMP
and there is no doudth in my mind that the number of clock-cycles needed to update the screen after moving a window increased quite a bit when introducing multithreading but still the fact that certain operations got attention a bit earlier made the OS feel *so* much more responsive. Even though I only had one mouse.
The same goes for multi-window applications. Like I mentioned ABrowse is essentially single-threaded and it can open multiple windows. Just out of curiocity I opened 10 browser windows using a single instance of ABrowse on one desktop and then started 10 instances of the browser on another desktop. All browser windows displaying the same web-page. I would not have any problems passing a “responciveness blind test” between the two desktops
The fact that more finegrained locking made BeOS run slower on a BeBox does not surprice me one bit. The PPC’s in the BeBox was not designed to run in SMP. Be threw away the second-level cache and added a second CPU instead. The L1 cache syncronization model this gave caused a *lot* of cache flushes essentially halting the CPU’s when the two CPU’s was working in the same memory area. CPU’s designed to run on SMP boards have much better characteristics in this situations. I would not be surprised if ripping one of the CPU’s out of your BeBox would also speed up certain operations.
At some point in a not to distant future I will add support in the kernel for a generic “wait for multiple event” kind of call almost like select() but that can wait on all kind of objects (pipes, sockets, message-ports, semaphores, thread-deaths, etc etc) and that have a register-object/unregister-object type of functionality so not the entire list of objects have to be passed to the kernel each time you want to wait for a set of events.
I have often been thinking about using such a system to allow multiple loopers to be run by one thread. Either by making it possible to link loopers together or by having another kind of thread-object that you can add loopers to instead of Run()’ing them. I can absolutely see that this would be of great help for porting single threaded applications but I’m very reluctant to make it “to easy” to write single threaded applications under AtheOS since I have first-hand experience with how much multithreading increase the responsiveness of the GUI.
Kurt Skauen
Hi Kurt, thanks for the informative post.
On the server side, I don’t see why multithreading should not be used, especially if/when it makes sense. After all, I don’t see why a multithreaded server would have any impact on the number of threads a developer might have to deal with on the client-side. Furthermore, part of the graphics server will be CPU-consuming tasks, for which threads are an obvious remedy (it is well-known that you don’t want to perform time-consuming tasks in a thread that’s also used to dispatch messages).
Could you describe the architecture of ABrowse in a bit more detail? When you mean “essentially single-threaded”, what do you mean? Do you mean that you run all the windows in the same thread, or that part of the HTML engine is protected by a giant lock. Do you also mean that the HTML engine is called into synchronously as a response to getting certain messages. If that’s the case, the real problem isn’t with the number of threads that run message loops, it is the fact that window message loops are asked to do some processing, and that message dispatching is slowed down by that processing. A good architecture would be to have one master message loop taking care of the “small” tasks, and one slave thread calling into the engine. That’ll give you enough responsiveness.
Hearing an OS developer being reluctant to make it easier for application writers to write certain kinds of applications makes me shiver. That’s in my opinion the last kind of thought that an OS developer should have.
–jbq
Re select on events – Yes, please do it! That will be a big help for all kinds
of applications, aside from singlethreaded loop dispatching. DEC VMS had a
“logical event flag” for that, if I remember right after all these years, just
a one bit value that the caller optionally supplies with a long system request.
Re multithreaded application architectures – how could you get optimal performance
with a browser with less than one rendering thread per window? If each rendering
window has its own thread, then when one is hanging on disk I/O or something, another
can run. But single threaded, it’s just dead time. A single slave thread handling
it for the whole application isn’t going to add any performance.
I’ve had a very interesting IRC chat yesterday with Kurt, during which we cleared out quite some misunderstanding, and analyzed the results that Kurt had seen.
About the AtheOS app_server first : the app_server has to deal with various tasks that take some significantly different processing times. Passing a keydown event around will take a few microseconds, whereas doign a big (software) screen-to-screen blit can easily take hundreds of milliseconds. Processing all those sequentially means that the “small” events are slowed down by the “big” ones.
About ABrowse : the HTML engine isn’t thread-safe, and each instance of ABrowse can only access the engine from a single thread at the same time.
When each window runs in a different process, the bahavior of each window is very similar to that of the app_server described above : calls to the HTML engine to parse and/or render HTML will slow down the passing of messages. Fortunately, this goes mostly unnoticed, as the windows stay independent from each other, and as it it pretty “normal” while using a browser to not use the window while it is parsing or rendering HTML (there’s nothing to interact with at that point!).
When all the windows run in the same process, things suddenly slow down. The major reason is that some small events that would normally be processed quickly but require to lock the HTML engine suddenly become much much slower, because they have to be processed sequentially with some long events.
Interestingly, the “single-process” ABrowse still has one thread per window, as many as the one-process-per-window ABrowse, but is significantly less responsive, proving that one-thread-per-window doesn’t automatically imply responsiveness.
ABrowse feels slow when running all the windows in the same process, for several reasons :
-because it processes small events and long events sequentially within the same window (even though this is not a major issue for a browser).
-because, through the HTML engine, it processes small events and long events sequentially across windows.
-because, since the big events have to be processed sequentially, it can’t use the second CPU efficiently.
There are a few ways ABrowse could be made more responsive :
-If the HTML engine is kept entirely “as is”, any event that needs to be processed through the engine should not be processed by the message loop itself, but should be deferred to a worker thread.
-If the HTML engine can be made “a bit” thread-safe, such that a “small” event and a “big” event can be processed through it simultaneously, only the “big” events need to be deferred to a worker thread.
-If the HTML engine can be made entirely thread-safe, it would be possible to have several worker threads to process the “big” events, where the number of such threads would be chosen relative to the number of CPUs.
If would then be possible to achieve near-maximum responsiveness with a single message loop processing all the “small” events, and dispatching the “big” events to a pool of worker threads, where the number of threads would be chosen such that the (unix-style) load created by the worker threads would be equal or slightly greater than the number of CPUs.
A few rules of thumb to choose how the threads should be chosen, assuming a reasonable level of thread-safeness :
-A thread should not process messages that require different amount of time to be processed. Both I/O time and CPU time count in that case.
-If an application is expected to have to deal with messages that take different amounts of time to be processed, it should have several threads (or pools of threads) each dedicated to processing messages that take similar amounts of time.
-The number of threads withing each pool will vary depending on the number of CPUs and the kind of tasks that the threads are expected to have. A good rule of thumb is to try to keep the (unix-style) load approximately equal to the number of CPUs there are in the machine.
-The priority of threads should be chosen such that the average likelihood of a thread being scheduled is inversely proportional to the time it takes to process the tasks that that thread deals with.
–jbq
I’ve spent he bulk of my career working in Windows and OS/2, so my perspective on the MT issues addressed so completely by both Kurt and JBQ are probably a little different. <p>
The problem as I see it is that MT is percieved as a ‘quick’ answer to providing UI responsiveness. This is fairly accurate, but it’s got nasty ramifications. Most code has a pretty specific flow. In the case of Windows and OS/2 everything runs in a single app, uses a single thread by default. Under NT that changes marginally, but it’s more of a semantic change than a functional change. Those of you that have used Windows XP will have noted that Console Windows are not themed, the semantic change doesn’t effect them, they run in a single thread :-). The nasty ramifications are that communications between threads become unreliable without significant efforts to prevent this. In Windows, I personally go out of my way to implement threads only where appropriate, ie, when you start an application, run the UI on the default thread, creating windows, populating them, arranging them etc. This can take a while in a complex application, think Productive or the defunct e-Picture startups. During that time, it makes sense for the other long running processes that run in that time to be doing other things on other available CPU’s, in my case I do alot of DB related work, so I have the data access running in it’s own isolated thread. When that data thread returns, I need to then push the data from the thread into the UI thread. That means synchronization. Lock the UI thread and post the data. Let’s say the JBQ’s issue of dropping a message because of poor system conditions occurs. The customer experiences Data loss, the app loses data integrity, and the user experiences a ‘bug’, one that I as the developer cannot reliably reproduce, and therefore almost any fix I put in place is pure guesswork.<p>
How real is this situation? Very.<p>
There are several solutions out there. I’m not going to come off the fence and say that any one of them is better than the other, but for grins I will let you in on one of Windows dirty little secrets. Windows does COM and DCOM via Window Messages and DDE behind the scenes. It’s all smoke and mirrors too. Along the way, the learned that the Single Input Queue that OS/2 had created ‘perceived’ lockups when the system was still in fact functioning (but an app wasn’t clearing it’s message que), so they used mutliple ques, and in Windows there are literally hundreds of invisible little windows handling COM / DCOM messages in thier own little asynchronous ques to make inter-process and inter thread communications easier and more reliable.<p>
In other words, while I love the BeOS, I find it MT to be massive overkill to a problem that wasn’t that severe.
dru
Hi,
One can argue that forcing MT threading is helpful or not for performance.
I long thouht it was obviously positive. I don’t think so anymore.
Can MT be good? Sure! But MT doesn’t support non-excellent implementation (either you get stability or unpredictable results, or you get poor performance, or both!). JBQ says that MT in the app server slowed it down at first? That’s probably a good example.
Yes, it can help. But for which application? It is easy if windows are truly independent (no sync needed, no/little sharing of information) like for a media player or Tracker, and will help if several windows have high speed requirement (media player). But if the goal is to display preference panels, simple UI stuff, there there is just no point, and it will get in the way. And this is the case of so many applications.
Where MT can be very useful, is when the app has a lot of activity going on. Something like Nuendo. But then, you have a *huge* deal of inter-dependency, with values shown in different ways in different windows at the same time. And there, MT goes in your way if it is not done the way you wish.
BeOS forces the way MT is done in you app, and this is the real problem. I want to design my app the way *I* wish depending on *my* needs.
Take Nuendo (again). How did I implement it (following the recommendation of Be experts who refused to help in other ways): I killed down BeOS’s multithreading by basically forwarding every UI message in my own event loop, and then, I used Nuendo’s internal multithreading, which is already pervasive. In the end, I had an app which was amazing. Fast redraws and everything, and no compromise. But with its own private multithreading. JBQ will correct me, but I don’t think that any app on BeOS was nearly has “live”, fast and powerful as Nuendo.
The war is not: MT is good or not. MT can help greatly, but it has to be well done. BeOS forces the way MT is done in your app, and this is really bad. From discussions I have had with other developers who made “big” beos software with multi window UI, this fact alone killed (or delayed so much) so many projects, that it certainly was the reason #1 why so few “great” software appeared for BeOS.
-geb, [email protected] (please cc by email your answers..)
Okay, my two cents??
While it is good to use MT throughout a system, I believe there should be some automation of coding… in other words: Allow programmers to program in a familiar language, but all that would be necessary for a new thread, is to create a new *.h or whatever file… I am not a C programmer, but I feel that this wouldn’t be tooo bad for the developers…. If they wanted to do things in a seperate thread, just create a new file..perhaps a thread file to include in the final compilation similar to the following:
# Thread File
!/thread_prio=normal # duh…to set the thread’s priority…can be set in any way, of course
!/thread_app_id=MyApp # To link this thread inherently, and quickly to your App
!/thread_id=ThreadName # To be called from other threads like: CallThread(thread_id), thread_data
IF
“$thread_data” == “” do/then
do_null() # do_null() would be set in the compiler the same as return_app, this would be used to avoid running any more checks than needed…
“$thread_data” /= “” do/then # /= is does not equal..so if the thread_data has anything in it, it will be processed
do_thread_work($thread_data) #this would of course do the dirty work of the thread, defined in any variety of places…
else
return_app() # this causes a quick termination of the thread, without an error.. return (0) I think is similar, but this will keep it simple-in-the-head…
FI
:do_thread_work #could be used to define work to be done…
IF
$thread_data size > 32 do/then # this will be to do small work…
goto_thread_id(large_work) # just making the hard work go to another thread…
else
do_dirty_stuff($thread_data)
FI
:do_dirty_stuff # again, just defining the work
DO WHATEVER WORK THAT NEEDWS TO BE DONE…
Do you guys get it??
And, yes, I realize that I could easily just set the main app to filter the thread_request() to the correct filter, but I feel that this could probably be done in a totally new thread, so do_dirty_work would simply be another file or sub… Pretty simple…and fast, I would think…
Would anyone be interested in something like this?? OF COURSE, THIS WOULD HAVE TO BE MADE BY A PROGRAMMER… I have looked at code, but I have never written my own C code… HTML, JAVASCRIPT, BAT(CH), Q/BASIC)…but NO C….so no jokes…. 😉
I would be VERY interested to hear what y’all have to say… Maybe this can be used to help Atheos in the future… 😉 A little tie-in would be nice…seeing that VERY FEW posts have been directly pointed towards to AtheOS. Of course, learning the down/short-falls of a GREAT OS such as BeOS during the developement phases of another, similar enough, OS could be very healthy for that OS….
Anyway..that is my $2000000000 worth…
Hi Joe!
What you are getting at is not the problem with threads, as has been discussed here lately. With an object oriented API, you usually inherit from some kind of thread class and in most cases end up creating a separate header and c/cpp file anyway. You can then launnch the thread or control its priority via simple function calls. I think what you propose would just add much more hassle and not solve any problem.
The real hurdle with threads is not setting them up or launching them, but managing communication and syncronization between them in a safe manner.
Best Regards,
Robert
1) Multithreading is just an event handling system where there kernel provides
various message queues. It gets most “useful” with preemptive scheduling.
This can all be implemented manualls if you feel like it.
[ Prof. N Wirth wrote a very readable paper against MT years ago. ]
2) Async IO is multi-threading with second class threads and a very inflexible
scheduler. If you don’t carefully lock data structures then the IO handler
can access inconsistent data structures, if you do lock them then youve
just (loosely) synchronised your async IO handler. 🙂
3) There is no such thing as async messaging.
The appearance of Async messaging is achieved by having a queue managed by
a separate (and synchronous) worker thread enqueue and dequeue messages.
The policy used by the worker thread to handle full queues is important.
If the policy is to drop messages, and the worker thread is buried in the
kernel and the policy is hard coded in the thread, then you end up with an
unreliable OS.
4) Having the kernel allocate message queues is a _bad_ idea, it makes proper
resource accounting very hard. I believe the late Prof Jochen Liedtke was
correct in his L4 kernels in choosing only sync IPC. However I believe he
was mistaken in not providing a semaphore for shared memory synchronisation.
[ Note MS Windows has both PostMessage() and SendMessage(), the later is
at least partly synchronous. ]
5) Synchronous messaging (and a predicatable scheduler) makes debugging easier,
as it is possible to predict what happens.
6) Handling locks is tricky, which is why well designed languages provide high
level synchronisation primatives where the compiler can verify many basic
properties (e.g. locks get unlocked), and where code analysis can verify
several more (e.g. consistent nesting) allowing formal analysis of deadlock
freeness (what a word :-).
7) The idea of separating UI code from application code is not new.
Its just a shame that X did not include a proper extension mechanism.