I have written (and continue to write) an application called Desktop Manager for OS X. It provides virtual desktops to those refugees from Unix like myself :). Recently it became apparent that it would no longer work under Apple’s forthcoming Tiger release of OS X. This proved a little bit of a problem for me since I have no copy of Tiger to test with and don’t wish to risk Apple’s wrath by obtaining a less-than-legitimate copy. Instead I have relied on a network of people I correspond with via my blog to tell me when things break. It also means I have to attempt to reverse engineer something I don’t have in front of me. I am indebted to these people and would like to thank them for their hard work.
I am often asked in email how I uncovered the API calls I use in Desktop Manager which are, unfortunately, undocumented. This article aims to give a little insight into the techniques I use to reverse engineer OS X in order to provide extra functionality to users and extra information to third-party developers. In this article all the utilities I use are a standard part of OS X’s developer tools which are freely available. Commercial disassemblers may offer more features but I hope this article will give a feel for how things can be done with free tools.
I shall assume some basic familiarity with the concepts of machine-code programming (i.e. registers, load and store instructions, etc) but in-depth knowledge of PPC assembler is not required.
The problem
One of the features that Desktop Manager has is the ability to use a transition effect when switching virtual desktops. These transitions, according to initial reports I was getting, were one of the things that didn’t work in Tiger. This was particularly unfortunate since a non-trivial number of people use Desktop Manager for precisely this feature. The existing code for transitions essentially boils down to this previously reverse engineered code:
void doSwitch(int transition, int direction, float seconds)
{
CGSConnectionRef cid = _CGSDefaultConnection();
CGSSetWorkspaceWithTransition(cid, workspaceNumber, transition, direction, seconds);
}
Perhaps a little explanation of these lines is in order; each graphical OS X application talks to a ‘Window Server’, which is responsible for actually compositing the windows on screen, via a ‘CoreGraphics connection’. The _CGSDefaultConnection() function returns a reference to the default connection used for the application. The second function uses this connection to talk to the Window Server and switches to workspace (desktop) number ‘workspaceNumber’ using a particular transition type, direction and duration (in seconds).
This method works beautifully in Panther but doesn’t in Tiger. Repeated fiddling with the arguments to CGSSetWorkspaceWithTransition() didn’t appear to get any where so it was time to find a new solution. I also wanted this solution to work on both Panther and Tiger. It was time to put my Reverse Engineer T-shirt on and get a little dirty.
Finding a way in
All was not lost as I remembered that during the Panther install there was an introduction movie that plays on the first boot up and then segues into the registration screen via the well-known ‘Apple cube’ transition. Those I knew with legitimate copies of Tiger reported that the Tiger install has a similar stage and therefore my aim was to find out how this application (the Setup Assistant) did the transition in Panther since I hoped that Apple would be less likely to change the APIs it uses itself when developing Tiger.
I, of course, had no guarantee of this but short of obtaining Tiger illicitly, or waiting for my pre-ordered copy to arrive, it was the only plan of attack I had.
The first stage was to see what other API calls there could be which dealt with transitions. All the undocumented APIs used in Desktop Manager come from the CoreGraphics framework buried inside the System folder. To see what else there was in there the nm tool was used to grep the symbols defined by the framework:
persephone:~ rjw57$ cd /System/Library/Frameworks/ApplicationServices.framework/Frameworks/ persephone:~ rjw57$ nm -g CoreGraphics.framework/CoreGraphics | grep -i transition 965521d8 T _CGSInvokeTransition 96553468 T _CGSNewTransition 96554b1c T _CGSReleaseTransition 965571dc T _CGSSetTransitionAlias 96557d78 T _CGSSetWorkspaceWithTransition 96558bd0 T _CGSTileTransitionPoint
The ‘-g’ flag tells nm to print out only the external symbols (i.e. those that can be linked to trivially). There was some good news here, lots of likely looking API calls. The next step was to see if the Setup Assistant used any of these calls. To do this the otool utility was used to disassemble the Setup Assistant framework:
persephone:~ rjw57$ cd /System/Library/PrivateFrameworks/SetupAssistant.framework/ persephone:~ rjw57$ otool -tV SetupAssistant >~/sadiss.txt
The ‘-tV’ flags tell otool to disassemble and resolve symbols, i.e. make the disassembled code a bit easier to read. A little grep-ing later and a useful-looking Objective C method was found which used some of the interesting API calls.
Developing the solution
Before it can be seen if a particular API call will be useful one needs to work out how to call it. There is no public documentation for these functions and hence no header files that can be used. The first stage in analysis is working out how Setup Assistant calls these functions. Below are a set of extracts from the disassembly which were used to investigate the API. Each block relates to an interesting API call or argument thereof. We shall go through each block in turn and finish with a piece of C code which uses the API calls we are investigating.
Firstly, however, it may be worth going over the PPC calling convention. When calling a function the values for each argument are placed in general purpose registers starting with r3. For example, if we have the following function
int foo(int a, int b, int c)
then upon entry r3 contains the value of ‘a’, r4 contains the value of ‘b’ and r5 contains the value of ‘c’. The return value is placed in r3 when the function exits. When examining our disassembly we need to pay close attention to what happens to the values of r3 upwards because of this.
The start of the method looks like this (I have added some comments to aid legibility):
; Start of method -[NSAssistantController(Private) showNewPage:direction:]: 91f4313c mfcr r2 91f43140 mfspr r0,lr 91f43144 bl saveFP
This is pretty much standard boilerplate code for an Objective C method. Our first interesting call is in this block of code:
; Block 1 91f43218 bl 0x91f670cc ; symbol stub for: __CGSDefaultConnection 91f4321c lwz r2,0x478(r1) 91f43220 stw r3,0x378(r1)
This code calls _CGSDefaultConnection() which from previous work we know takes no parameters and stores r3 (the return value) in a particular memory location (r1 + 0x378). We can re-code this in C fairly simply:
/* Block 1 */ int t1; /* Temp. variable stored at r1 + 0x378 */ t1 = _CGSDefaultConnection();
This block is interesting because it sets up some memory used later.
; Block 2 91f4323c li r4,__register_frame_info.eh 91f43240 li r5,0x14 91f43244 addi r3,r1,0x40 91f43248 bl 0x91f670ac ; symbol stub for: _memset
We know from existing documentation that memset takes three arguments, a pointer to a region of memory, a value to set all bytes in that memory to and the length of the region. Looking at the disassembly we see that the area of memory to clear is at location r1 + 0x40 and its size is 0x14 (20 bytes or 5 32-bit integers). Our C version is therefore something like:
/* Block 2 */ int pt2[5]; /* An array which starts at r1 + 0x40 */ memset(pt2, 0, 20); /* 20 = 0x14 */
where I’ve avoided checking what value the memory gets cleared with since it isn’t interesting at this moment in time.
Our next block is interesting because it calls one of the API functions we don’t know about.
; Block 3 91f43314 addi r4,r1,0x40 91f43318 lwz r3,0x378(r1) 91f4331c addi r0,r2,0x74 91f43320 addi r5,r1,0x370 91f43324 stw r0,0x50(r1) 91f43328 bl 0x91f6708c ; symbol stub for: _CGSNewTransition
Straight away we can see that only r3, r4 and r5 are set implying that this API call takes 3 arguments. The first argument, r3, is loaded from the memory location r1 + 0x378. Referring back to block 1 we see that this is the value returned from _CGSDefaultConnection(). The second argument, r4, is set to point to memory location r1 + 0x40. As we have seen above, this is a 20 byte area of memory previously cleared by memset and then modified by the code between blocks 2 and 3. The final argument, r5, is set to point to memory location r1 + 0x370. Since this isn’t really set anywhere in the disassembly we can assume that this is a pointer to some temporary variable. We can now write a C version of this block:
/* Block 3 */ int t3; /* Temp. variable stored at r1 + 0x370 */ CGSNewTransition(t1, pt2, &t3);
We now investigate the last two blocks which contain interesting API calls.
; Block 4 91f433f0 lwz r4,0x370(r1) 91f433f4 cmpwi cr7,r4,__register_frame_info.eh 91f433f8 beq+ cr7,0x91f43404 91f433fc lwz r3,0x378(r1) 91f43400 bl 0x91f66fec ; symbol stub for: _CGSReleaseTransition ... ; Block 5 91f43484 lwz r3,0x378(r1) 91f43488 addis r2,r31,0x3 91f4348c lwz r4,0x370(r1) 91f43490 lfs f1,0xde60(r2) 91f43494 bl 0x91f66fcc ; symbol stub for: _CGSInvokeTransition ... 91f4357c mtcrf 56,r11 91f43580 b restFP ; End of method
By examining the values of r3 and r4 and comparing with block 3 we can easily write equivalent C for these blocks:
/* Block 4 */ CGSReleaseTransition(t1, t3); /* Block 5*/ CGSInvokeTransition(t1, t3, f1);
where f1 is some floating point argument. Comparing it with CGSSetWorkspaceWithTransition() we can probably be fairly sure it is the duration of the transition in seconds. The hard work is now over. We’ve discovered the number of arguments each API call takes and where their values come from. The remaining task is to combine these arguments with what we already know to give them names more meaningful than t1, etc.
From the call to _CGSDefaultConnection() it is clear that t1 holds a reference to the application’s connection to the Window Server. The variable t3 is set by CGSNewTransition() and used by CGSReleaseTransition() and CGSInvokeTransition(). We can therefore suppose that it is some handle which refers to the created transition. The memory at pt2 is passed only to CGSNewTransition() so we can suppose that it is a structure which specifies the various parameters of the transition. The call CGSReleaseTransition(), interestingly, happens before CGSInvokeTransition(). It seems unlikely that both of these functions would be called in that order so we may suppose that the transition is only released if some other operation fails and that invoking the transition implicitly releases it. Using this information we can flesh our our C version:
CGSConnectionRef cid;
int transitionHandle;
int transitionSpec[5];
float duration;
cid = _CGSDefaultConnection();
memset(transitionSpec, 0, 20); /* 20 = 0x14 */
/* ... modifiy contents of transitionSpec (between blocks 2 and 3) ... */
CGSNewTransition(cid, transitionSpec, &transitionHandle);
/* ... find out if we failed ... */
if(failed)
CGSReleaseTransition(cid, transitionHandle);
else
{
/* set duration */
CGSInvokeTransition(cid, transitionHandle, duration);
}
All that remains now is to work out the format of transitionSpec. After a little trial and error we find the following code works just as well as our initial code:
CGSConnectionRef cid; int transitionHandle; int transitionSpec[5]; cid = _CGSDefaultConnection(); /* Set up the transition specification */ memset(transitionSpec, 0, 20); /* 20 = 0x14 */ transitionSpec[1] = transition; transitionSpec[2] = direction; /* Set up the transition itself */ CGSNewTransition(cid, transitionSpec, &transitionHandle); /* Switch desktop */ CGSSetWorkspace(cid, workspaceNumber); /* A little pause to let the Window Server sort itself out ... */ usleep(10000); /* ... and fire off the transition */ CGSInvokeTransition(cid, transitionHandle, seconds);
Conclusions
The above code was added to Desktop Manager and a version sent to those people with Tiger. To my great relief (and suprise) people reported that transitions were working! There are still problems to be worked out but, because of this bit of reverse engineering, Desktop Manager on Tiger will soon be a reality.
Can we now write some documentation on these new API calls? Indeed we can. A little experimentation has shown that these are not just useful for virtual desktops. Indeed anyone can use them in a full-screen app to provide Keynote-style transitions. Simply call CGSNewTransition() which will freeze the screen, draw your new screen and call CGSInvokeTransition() to reveal it to the user.
About the author:
Rich Wareham is a 3rd year PhD student in the Signal Processing Group of the Cambridge University Engineering Department. When not playing with Geometric Algebra and proving theorems he enjoys learning about technology, experimenting with new Operating Systems and teaching undergraduates about Software Engineering.
If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.
A great technical article. OSNews should have more of these.
Hi,
If your application is actually bringing in enough money for you to spend on it’s development (no judgment here – I actually assume it is, since you’ve been around for a while), you might want to check this out:
http://developer.apple.com/labs/index.html
If you’re a premiere or select member of the developer program (or can convince one to give you their asset) you can use Apple’s lab to test your stuff – should have pre-release software installed for you to test against/develop against.
Regards,
TLFord
I agree with first-post.
This is one the best articles I’ve read on OSNews for a bit now.
Good job.
>OSNews should have more of these.
It’s not on our hands really. If people don’t write such articles often and not sending them to us, we can’t be possibly posting them.
is one of the greatest programs i’ve added to my iBook. for a while, the lack of multiple desktops was one of the main reasons i had for sticking with linux; especially on a 12″ screen. and it’s another reason i probably won’t upgrade to tiger soon if this is true.
really, if you use a mac, and don’t have this running, download it now and set it to startup at boot.
If you paid for your articles, people would write them.
Nobody likes to work for free.
Exactly 🙂
>>Nobody likes to work for free.<<
I know of tens of thousands of F/OSS developers that can contradict that.
people work for free all the time. People donate their time to charities (Christmas Red Kettle Campaign by the Red Cross)
The fact that you need money for motivation just show what kind of person you truely are.
I am not getting paid either, so I don’t understand how you expect me to pay others. OSNews is a voluntary site.
And please stay on topic.
This is an excellent article, although a titch over my head considering my lack of familiarity with assembly in general. I really love what DM has become too. I used it a while back when it was .2 or .3 and wasn’t really that pleased, but I’m glad to see what i’s become. I just hope that you’re able to get all the odds and ends tied up before the Tiger release. But anyways, kudos on the article and the program.
This article is one of those little diamonds in the rough that keep me reading OSNews. Thanks for the good article.
I have had to stop using Desktop Manager for now because some modal “sheets,” such as the save as dialog, were getting shoved behind windows and I couldn’t access them. Other than that problem I really liked the program. It was one of the first things I installed on my Mac when I acquired it.
This is undoubtedly one of the best and most amazing apps out there. When I show my PC friends at work they are floored at how fast the switch takes place. The affects are unreal. It actually makes me sick if I watch it to much..
Great app Rich, You have done a great job!
Powerbook 15″ 1.5
>>The fact that you need money for motivation just show what kind of person you truely are
A poor one?
A very nice article. I am currently using Virtue because it had a few features I want, but Desktop Manager is very nice as well. From the developers blog I believe that DM is getting a 2 dimensional layout of desktops, which is one thing I want. The other being the ability to watermark desktops, but that is not as big a deal.
(P.S. Thanks for porting Xine as well.
)
Great read. One of the better one’s in some time.
Thanks for posting.
[SNIP]
” …really, if you use a mac, and don’t have this running, download it now and set it to startup at boot…”
How did you set it to start up at boot? Thanks.
You drag the app icon to your startup list under the Accounts system Prefenece. It’s on a per user Basis.
To micheal i live pay check to Paycheck yet I still have been able to afforad a powerbook. I don’t make much i can just save a little at a time.
I have to have virtual desktops in Linux. Under OS X though I have my powerbook hooked up to a second monitor . I have two real desktops. When It’s just the one I don’t normally have as many apps running so expose works well.
How did you set it to start up at boot? Thanks.
Go to sysprefs, then to Accounts. Under the account you wish to have it start up at “boot” (more like start at login), there will be a tab named “Startup Items” you can add it there.
Awesome article, we need more of these, and fewer “My one-night stand with distro Y of GNU/Linux”. Desktop Manager thoroughly rocks, and is one of the apps I use to show off OS X to people who have never really used a mac. Unfortunately (or not?), I dont use virtual desktops in OS X, though I could not live without them in Linux.
DM and LB (LaunchBar) are essential Login Items that make OS X feel like a polished and well crafted musical instrument.
—
The article finally inspired me to donate…
Is ten quid too paltry?
That’s twenty bucks now for us living under Dubya’s glorious economic management.
I’ve gotta add Quicksilver to the list, that’s one _cool_ app.
Great article, Rich, and thanks again for your great app! I’m currently running Virtue (and can’t live without it) which is admittedly based on and made possible by DM which I also like greatly. Keep up the awesome work! You’re making OSX really look good and function well. I’ve left Linux behind!
…did something similar with VAX/FMS. We knew that FMS had to have an API to manage certain aspects of a created form (stuff managed “for the programmer”), but it was never documented so we had to find them ourselves. It was fun, poking through the shared library for the functions… then mapping out their parameters!
I agree, this was a great article and brings back memories!!!
I truly love DM; it’s one of those rare apps that you soon forget you installed, because it integrates perfectly into the workflow, and every time you use someone else’s Mac you feel something fundamental is missing.
I usually get amazed comments from complete strangers who see me “cube-spin” between desktops whenever I’m in some sort of public place. It’s a real eye-catcher.
Nice article, too.
I only just started using DM recently – and am totally hooked. The one essential partner to DM is witch – http://tinyurl.com/5eo3l – If you set it up right, you can use alt-tab to change between the windows on the current desktop. It’s a killer combination.
No really, the learning curve for programming is hard. And undocumented Calls and Libs should not be part of the OS, it is a crying shame because historically many features of the OS were originally written by third party CodeMaestros. Thinks like Clocks, PPP, Desktop Pictures were not in 7.1 but had to be stitched in.
*HOWEVER* Apple has worked a set of miracles in the OS, which is functionally free.
a.> with a new machine
b.> ‘friendware’ Apple is a HW company so software ‘piracy’ is not as major an impact as it is for say AdobeMacromedia.
– and stable and efficient in a way that other OS’s fear.
And all of this is done under the dual swords of M$oft with the treat (oops Freudian) threat of removing Apple’s ONLY reason to be in the Enterprise which is M$oft Office AND the microscope of the market which functions like those ‘(x) days since a fatal accident’ that you see in factories & such.
As far as some opinions that Apple has it easier than M$oft because of the reduced number of MB’s and compatible HW, but that is pure slag. This was true in 1994 but not as real of a factor in 2005. This is the job of the HAL and the Kernel. BTW my Java always works.
Great article, reminds me of when I first started working… mmmm kernel debugging over com ports.
I actually had to do a process similar to this one while working on bringing Chromium – http://chromium.sourceforge.net/ – over to OS X.
In my case, I was trying to figure out details on two undocumented functions: CGLSetSurface and CGLGetSurface. I only got as far as figuring out what the parameters were, however, leaving how to use the functions for a later time.
Apple may choose to ‘hide’ the functions for a reason, but it isn’t hard to find a reason -to- use them.
nice from a technical standpoint, but who needs virual desktops when you have expose?
First of all kudos for putting in so much hard work to build a great app.
I’m not really sure however, whether I see any added value in this application. As far as I can tell, the desktop on the Mac is managed pretty well. I feel no compelling need to add an extra step to the workflow. No doubt that’s because I haven’t actually seen the application. It may very well be the thing that we want. Apple has a history of honoring great design by incorporating individual developer’s ideas into the OS eventually.
I’ll give it a spin later [but I’ll be running with the Tiger soon so you might want to get your app into gear with the times].
As far as invoking the wrath of Apple… get yourself a developer’s license and you have access to everything you need to make a great app and more.
Besides, I think there are plenty of developer tools with the system disks, no? Maybe that’s not what you need.
Keep building a great app. Maybe I’ll point at it some day and tell my friends: this was actually created by a geek with a vision and Apple bought it. Innit cool?
Godspeed!
Why are these calls undocumented? Does Apple not want ‘norma lpeople’ to have apps that do these fancy effects?
If that is true, then I can understand kind of why not, because maybe they want to keep the system clean and don’t want overuse of these special effects by over-eager third party developers.
On the other hand, if it is true, it might show a lot of distrust for your developers, and a lack of respect for them, by suggesting that the developers are not to be trusted to have these APIs, only we at Apple can have them…
Do you think these scenarios are true? Maybe I have totally misunderstood the situation here!
Good article.
I can’t help but observe that keeping certain APIs undocumented to give your apps an edge over others is the behaviour Microsoft were always accused of. Is it really true that Keynote uses transition effects nobody else can? If so, shame on them.
Just goes to show that underneath the technology differences, Apple and Microsoft are equivalent.
exactly the same kind of tricks I’ve been doing on BeOS
Was using http://www.reveng.cjb.net/reveng/ which is very useful on C and C++ sources… it’s not BeOS specific btw, it works in linux also, and probably in OSX.
Now, it’s nice to reconstruct the frames, but even funier to actually tamper with them.
Used some nice let’s-overwrite-some-BFoo::Draw-methods-prologs to hook in libbe and get it skinned.
Also fun to get new syscalls that way.
Though much simpler with the source ;D
Thanks for a great article. However I’d like to point out a misconception you’ve presented in your article….
“I, of course, had no guarantee of this but short of obtaining Tiger illicitly, or waiting for my pre-ordered copy to arrive, it was the only plan of attack I had.”
If you purchase a ADC Select membership, you can legally download pre-release versions of Tiger and other Apple technologies.
It’s not cheap, but if your time = money, it can easily pay for it self in short order.
Why are these calls undocumented? Does Apple not want ‘norma lpeople’ to have apps that do these fancy effects?
If that is true, then I can understand kind of why not, because maybe they want to keep the system clean and don’t want overuse of these special effects by over-eager third party developers.
On the other hand, if it is true, it might show a lot of distrust for your developers, and a lack of respect for them, by suggesting that the developers are not to be trusted to have these APIs, only we at Apple can have them…
Do you think these scenarios are true? Maybe I have totally misunderstood the situation here!
its fairly normal. an API consists of many layers, and for 99% of the time, going deeper then the first will only result in more complicated calls. Also, if an API is documented, apple will have a responsability not to break it. so stuff under that top layer will change quite a bit more often, due to bug fixes/enhancements.
That being said, knowing what is going on behind the scenes can be invaluable for certain kinds of problems(like this, where there is no high level APIs to do what he wants to do), or debugging (if the bug is actually in the API rather then your code). Of course, what makes life even easier is to be actually able to SEE the source code, so you can find out how it works if you have to.
all that to say, the practice is fairly normal 😉
“nice from a technical standpoint, but who needs virual desktops when you have expose?”
I thought the very same thing myself until I actually tried out Desktop Manager for a few days to see what all the “virtual desktop” fuss was about. Now after becoming used to it, I can’t imagine using OS X without it. For one thing, I discovered that virtual desktops and Exposé actually complement each other very, very well. I use a 15″ PowerBook, and with limited monitor space, using Desktop Manager in conjunction with Exposé and WindowShade, I can perform related tasks on a separate desktop, keep the relevant applications open, and use Exposé and WindowShade to manage the windows.
For instance, I use four separate desktops, “Main” for *ugh* work (typically no windows open), “Internet” (Safari and NewsFire), “Mail” (what else, Mail), and “Media” (iLife, Graphics Converter etc). It’s a dramatic improvement in my computer use to be able to switch tasks without having to bring windows forward, hide others, shrink others to the Dock etc. And like other posters have mentioned, the cube transition has the added benefit of leaving PC users breathless when they first see it. 🙂
It’s now an absolute must-have utility for me, and I’m greatly relieved that the developer got it to run under Tiger. Kudos to Rich Wareham for a wonderful piece of software.
This article shows up the fact that even now there really isn’t a good way for developers to obfuscate code such that it can’t be reverse engineered.
I remember doing such things back in the NeXTSTEP days. As a result, I “obfuscated” my own product code to such a degree that it became a real pain for me to maintain (“What the heck was I doing here?” and “What on earth is that reference for?”). Even with comments, most of the code was doing bogus things to try to hide the “meat”; it all became unnerving. I eventually ripped it all out and shipped a clean app.
Hmm.. all this time I’ve been using Space.app http://space.sourceforge.net/ But Desktop Manager certainly seems like a slicker, flashier, more “in your face, Windows!” way to go…
Will the DM update allow any CoreImage transition to be used in Tiger? I’m envisioning a ripple effect or dissolve as a more subtle, yet still visually impressive, transition.
write for free.
its a question of passion, too.
But the problem not doing it is time… when you dont manage to update your own homepage, how would you wanna writeird party articles…..
its a sad truth we spend considerable time generating our income in mostly boring ways, drawing away from the interesting things *g*
oh well for some at least ..
great article by the way.. about every assy looks nice to me, but i still can’t really read any c code *lol*