“Since its release a few weeks ago, curious developers have been sniffing through the source code for Google’s new Chrome web browser. Chrome’s source is interesting for a variety of reasons: there’s the new V8 JavaScript virtual machine with its boasts of near-native code performance, the WebKit rendering engine that does all the hard work of understanding and displaying web pages, and (last but not least), Chrome’s secure sandbox designed to minimize the impact of any security flaws that might exist in both the browser and plugins alike. It is this secure sandbox that has piqued the curiosity of some observers, and for a reason that many may find surprising. From reading the source, it looks as though Google has reverse-engineered Windows, and that’s explicitly prohibited by the Windows EULA.”
I like Chrome. I have not looked into it at that depth. But it let’s me run Apple’s Mobile Me on a Vista box and it is clean and fast – but if that is true or even half true then I really want ringside seats when these elephants start wrestling
—
Smells like napalm to me tho’
If this proves true, then it would be interesting to see if this adds any weight to the calls for Microsoft to further release information on undocumented calls.
It could prove that Microsoft are deliberately hiding their best code their their own products (for want a crude description) thus hindering 3rd party programmers to better intergrate their applications with Windows (which is one of the many reasons MS have been taken to court over the years).
However, the realist in me expects little to come from any potential court case. Both sides are far too smart and have far too good lawyers to let anything significant harm (or good for their business opponents) happen.
That is just silly. It comes down to that a supported API has to be supported. If you want to use an unsupported API that is fine, nobody is stopping you, but don’t complain to anyone when it is broken in the next point release. Sun does it in java, Apple does it in OSX, Linux doesn’t really do it, but the linux devs dont maintain backwards compatibility, so the point is moot.
Secondly, something like DEP is something that should be managed by the system 99% of the time. The capability exists in post XP SP1 systems to manually muck with it in those 1% of corner cases. Public facing apps that execute arbitrary code off the internet are in those cases, so they wanted to use it. Since this ability was only really exposed to the world post XP SP1 (which was primarily about fleshing out security), they had to rely on a hacky way to do it.
This is not proof that MS is “hiding its best code”, it is chrome trying to utilize an obscure feature of the APIs in operating systems that came out before it was publicly supported. I would say more then anything it is a testament to their thoroughness, since most people would have just said “we dont support pre xp sp1.”, as it was a major upgrade that any xp user should have.
Linux devs don’t do it *at all* because *everything* is open and visible. It is unclear exactly what you mean by “Linux”. The user space API of the kernel is sacred, and programs written for 1.0 will still run against the 2.6.27 kernel. (Its only the kernel’s internal API that changes.) If you are talking things like GTK. Well, GTK’s backwards compatibility has been the subject of much discussion here. The GTK 3.0 plans are all about breaking some things, and its big news.
Yeah, all I was saying is that it is a very different world when your development team is distributed and your code is under the GPL. For commercially supported frameworks developed in a non-distributed fashion, targetted at business users who do not ever want to have to re-write code to have it function on newer versions, there is a very big difference between supported and unsupported functions.
The crux of what I was trying to say was that an unsupported function for manual DEP control on vanilla windows XP without any updates is hardly proof that MS is trying to pull one over on the world.
uhm, no sorry. linux (kernel wise) is not backwards compatible in any way. the kernel interface even changes between minor versions. this does not really matter since all drivers are included in the kernel anyway.
libc/gtk/qt/x11 and stuff like that are quite stable and properly versioned.
but more on topic. the first time i experienced windows undocumented calls was with emm, somehow windows was able to take over emm but i was not. sucks to be the one left out.
You are confusing the internal API (which is used by drivers and other kernel code) with the user space API. Research the matter and you will see that I am right. In fact, it is because so many people hold the same misconception that I took care to clarify the matter. Otherwise I would not have bothered.
If you still disagree, please point out a specific example for us to examine.
Edited 2008-09-23 23:13 UTC
a long time ago i looked at a webcam driver. to see for yourself take a look at :
http://alpha.dyndns.org/ov511/download/ov51x/ov51x-1.65-1.12-mark/o… . This is what i meant with the linux (KERNEL WISE) not being backwards compatible. The internals drastically changed, and it shows. freebsd has for example ABI binary backwards compatibility for the major releases. you have to also specifically enable this. i could not find such a thing for linux.
and as for the kernel api : i also highly doubt libc for linux 1.0 also runs on linux 2.6, but it might be possible. when using binary backwards compatibility the structures you give to the kernel can not change. i highly doubt linus did this.
as a simple test i took siginfo.h and checked if it changed between 2.4 and 2.6 and it did. the siginfo structure is NOT binary backwards compatible. changing int’s to longs makes a lot of difference. it not even source level compatible. and this is the first file with the first large strucure i could find which is used by a the kernel api (the sys_* calls). maybe i’ll not allowed to use the variables deeper in the structure. but guess what, it’s not documented and linux always tells me to read the source.
could you maybe point out somewhere where linus states he preserves backwards compatibility? i couln’t find it.
please note that binary backwards compatibility is also about supporting code relying on bugs and stuff “you are not supposed to do”. so if it runs on linux 2.4 it must also run on a newer version, no matter what.
i have to admit the kernel api is small and more backwards compatible than i expected. i’ll give you that one
Sorry, but that’s not the whole truth.
Linux userspace APIs don’t ever change, except if the get deprecated. What’s changing in Linux are the structures withing the kernel. But that is good and to be expected from a developing product.
Linux devs DO mantain backwards compatibility, and it would precisely apply in this case – they would never break sandbox of Chrome from the kernel side (except unintentionally, but when that happens, it is fixed as soon as it’s discovered).
And usually results in an entertainingly colorful quote, not always suitable for mixed company, from Linus himself, to grace the kernel page of LWN’s Weekly Edition. 😉
Then obviously you’ve missed the NGPL vs. NTPL vs. Old Threading, where application compatible was not assured when Linux moved from the old threading model to the (which is used now) NPTL. I remember wine breaking because it relied on features based on the old fork() process idea of creating ‘fake threads’.
The comment only says that one *can find* this information by disassembling:
// Completely undocumented from Microsoft. You can find this information by
// disassembling Vista’s SP1 kernel32.dll with your favorite disassembler.
It does not say that is how Google found out about the api. It’s very easy (easier than disassembling, really) to find secret, undocumented calls in Windows’ API. Just run Microsoft apps under Wine. Wine will tell you all about it when the program makes a call that is not implemented in Wine. Go to Microsoft’s docs, and if it is not documented there, then… well, there you have it. That should be trivial to set up as a live demonstration in a court of law.
Edited 2008-09-23 16:12 UTC
I think the key thing here is that uninformed people deserve the right to post wild accusations and make unfounded claims.
After all, if they were to research the matter slightly, or not jump to conclusions, then where would we be on a slow news day? It’s hard to incite users to post incensed messages by a story entitled something like “Undocumented Windows APIs: You don’t have to be a monopoly to take advantage of them (but it helps)”, even if such a story could impart the same information.
The reason function calls are hidden is to protect the developer, not frustrate him/her. If a function is hidden there is a good chance that the parameter list could change in the future.
Back in the day my boss and I “reverse engineered” some FMS functions (DEC FMS – Forms Management System) because we wanted to fine tune our application. It was easy enough to do, you could simply edit the shared image, look for the entry points and the number of parameters were evident.
Then it was a process of elimination (add dummy parameters and see the compiler/runtime responses). Took a day and we had access to a couple of very powerful functions that were not listed in the documentation at all.
We did it because we used to work at DEC and knew that there had to be functions that performed some handy tricks behind the scenes for the API to do what it did. Anyway, we called the FMS guys and they wouldn’t give us the documentation for those unmentioned functions…
Exactly. If you write any non-trivial program, there is going to be a lot of code that is not part of the published interface, simply because a) it just doesn’t make sense for it to be published and b) it gives you much more freedom to change the function signature down the line.
There are reasons why certain API calls are not documented. People seem intent on ascribing the most malicious of these reasons to Microsoft.
Well, yes, but its also very tempting for developers at Microsoft to use those undocumented OS api functions in their programs ( word, live antivirus, IE, Media player). And they often do, leading to an unfair advantage. They can’t claim to be protecting the third party developer, if they themselves are using it in an application. BTW, apple also does this a lot.
IMHO, they’re their APIs, they can do whatever they want with them… they do not have TO publish and document every function their APIs provide, and, though the Win32 APIs are always criticized, they have better documentation than a lot of other commercial APIs…
Edited 2008-09-23 19:00 UTC
IMHO, they’re their APIs, they can do whatever they want with them…
If we were talking about anything other then Win32 APIs, I’d agree with you. However, to think that we should be grateful for documentation this good… well, it’s a tad naive.
We’re talking about an API that 90% of all applications (or more?) need to use. Windows is The platform to develop for. When you’re API is that critical to the software ecosystem, you had better document everything and it had better be documentation that makes Kernighan and Ritchie cry tears of solid gold.
It depends on what you mean with Windows. Win32 or .Net APIs? It’s clear .Net is the future, so all hacks around Win32 are dangerous. It’s going to be deprecated some day, or unsuported, whatever. Besides, your software must really suck if you need a hack to make it better.
If it doesn’t ship in the Windows box, it can’t use APIs that are not doc’ed in MSDN.
Yes. But on the other hand, if there’s something desirable that you can achieve only by using undocumented calls, it basically means that as an API provider you have failed the developers.
The security features implemented by chrome using undocumented calls are desirable. So maybe the calls won’t work anymore in the future, but then again microsoft is at fault for not providing an official and documented api to access these features.
First, it’s not so clear that anything in Chrome itself is reverse-engineered from Windows, only one Windows-only section that makes an undocumented system call.
Further, Google could have developed the code in the USA, in which case the EULA clause prohibiting reverse-engineering doesn’t apply (there’s lots of case law in the US on the subject — reverse engineering of technologies for the purpose of making novel uses of something, integrating with it, or making it compatible with something else is protected).
Last, the EULA only applies to end-users. You need not use Windows to disassemble and analyze it. Heck, you don’t even need to install it anywhere (you’re only presented with the EULA after installation).
I find EULAs that you have to accept after paying for a product highly questionable and unethical, comparable to blackmailing or theft. There should be a law against this! Are anyone aware of any laws (in any country) that specifically mentions this issue?
If you have to accept an EULA to get some software working after you’ve paid for it, it should not be considered a legal contract, as it’s just a necessary action to get what you paid for. Thus it should have no legal consequence if you break the EULA.
If you have to accept an EULA before paying, or if you don’t have to pay, the situation is different.
http://src.chromium.org/viewvc/chrome/trunk/src/sandbox/src/dep.cc?…
Edited 2008-09-23 18:20 UTC
…this will never see a court case. Microsoft and Google may be ‘adversaries’, but the point is Microsoft won’t press Google on a EULA. You have to remember: Chrome is windows only at the moment. (I say this as my laptop is compiling the latest linux codebase) If anything I can see Microsoft volunteering more information to Google, to help “improve integration” with windows.
Think about this like someone who owns a business: Your competitor has spent a good deal of money and time promoting a new product. They’ve stirred up a veritable hornet’s nest of opinions, but for now, their new product only runs on top of your product. An opportunity exist here. You can ‘parter’ unofficially with your competitor, work towards ‘interoperability, meanwhile riding their coattails, and making sure that they’re as locked into your product as possible.
If Microsoft is foolish enough to pursue anything along the lines of a lawsuit, I will be shocked and appauled.
Somewhere – can’t find the source right now – it said that these code snippets were from portions Microsoft released as open Source software. Noone did any reverse engineering here.
Sure, there is a reason that you create hidden functions, but when you do it to make your application software faster than the competition, it’s an unfair advantage. If it’s merely used for other internal processes and not used for applications, fine.
Microsoft has shown, with Office, that they’ve had an unfair advantage time and again and it’s usually the first software to break during an operating system beta, for some reason.
We’ve seen enough people dis-assemble memory on various products and while it’s again each of the EULAs, it happens anyway. A lot of people used to buy the Periscope hardware debugging boards in order to see what they couldn’t see otherwise. It’s one thing to steal trade secrets and another to just put yourself on even ground with a monopolistic company.
That said, I trust Microsoft almost as much as I trust Google…and I’m getting to that point with Apple.
From the article, commenting on the EULA:
“except … to the extent that applicable law expressly permits”
Except law is prohibitive, it doesn’t permit things. It is permited as long as it is not prohibited. I think. Besides, an EULA is an agreement, it is not law applicable. You don’t have to agree to the EULA. You can unpack the kernel32 and dissasemble it and you are not breaking any laws. You can’t be forbidden to do things except by law.
UPDATE: more so, law doesn’t prohibite anyone from killing any other guy. You CAN in fact do it. Do it now, and you’ll see how the law fails to prohibite. But you’ll get punishment for that. That’s what the law brings.
Edited 2008-09-23 19:49 UTC
On the one hand im worried about Chrome using undocumented api’s as a hotfix or any patch could break it and it would need the Chrome dev’s to fix it ASAP.
On the other I know it has been said that many companies have software with undocumented API’s. However when you provide the operating system to 90% of the computers out there, you have to make sure that, that OS provides a level playing field to any application that runs on it. This is really the heart of the antitrust cases, is that microsoft unfairly slanted the OS to Microsoft’s own applications. Im glad that the various courts were able to enforce Microsoft to even the playing field.
Now unlikely, as MS will get the flak if this happens. But still, how can someone be worried about a web browser? It’s not life critical, and not the only one available.
Edited 2008-09-23 20:29 UTC
MS should either remove internal API or document it. Otherwise ti turns into a nightmare sooner or later, when they have to patch bandaids for the backwards compatibility issue.
The OS where this is much better handled is Linux. If it is suitable it goes in and becomes “official” API, if not it never goes in. OK, almost, since there were mistakes like first timerfd try and new stuff can sometimes be marked experimental, but nothing’s perfect.
Why would they disassemble kernel32.dll for an undocumented hack around something when they could just Google it?
“This internal and totally unsupported API is in the Microsoft Windows Internals 4th Edition, Chapter 6, on download.microsoft.com ( http://download.microsoft.com/download/5/b/3/5b38800c-ba6e-4023-907… ). It’s also mentioned in a Microsoft Research PowerPoint ( http://research.microsoft.com/workshops/FS2008/presentations/Polze_… ).”
In the very article Ars links to, in the very second paragraph below the code quoted.
Wow, just wow. Talk about fail to read an article.
Microsoft freely gave this information out.
The sad thing is the article itself is pretty good, just has a trollbait title. Goes to show how many people here rtfa.
Perhaps the lesson is not to put stupid and entirely misleading titles on your articles and perhaps not dedicate pages for baseless speculation?
I dare say that there exist no serious Windows programmer that has never made use of undocumented and unsupported Windows API’s calls and speculating that this would mean that the programmer reverse-engineered Windows is a bit far fetched.
I read TFA, I just didn’t Click TF link.
Very interesting. So they used something that is not officially documented, but was actually released by Microsoft.
Supposedly, reverse engineering for the purposes of interoperability is explictly protected under US law, and the EULA can’t limit your right to do this.
Not so sure about other countries, but Sega vs Accolade I think is a good case that highlights this principle.
I can’t say i’m completely current with this stuff, and it would seem to me like the Blizzard vs Glider bot guys is a bit of a backpedal from the ‘reverse engineering for the purposes of interoperability is ok’ position – but as far as I know, Google, and anyone else, are well within their rights to disassemble windows source code in order to interoperate with it, whatever the EULA says.
Chrome is a good operating system, it just needs a decent Web browser
http://www.dragon-tails.com/comics/archive.php?date=030118
Come on now…
Given that Google has released the Chromium code as
open-source, would they *really* have done that
(and so exposed the code to MS’ scrutiny) if they had
reverse-engineered Windows?
If you were going to reverse-engineer Windows (and therefore had something to hide), wouldn’t you keep
your efforts *closed*-source?
Edited 2008-09-24 00:48 UTC
Once I got to “EULA” snippet in the summary, I decided not to bother reading the article
End User License Agreement
The end user is not reverse engineering nothing.
Edited 2008-09-25 03:27 UTC
I think Google was able to steal a lot of talent form Microsoft and chances are these people think about solving these type of problems in that method. I have my own method of approaching problems. They get incorporated in the work I do. When I change companies variants of my methods get pass to the new company.