Submitted by “WinFS has been officially pulled out of Microsoft’s road map for products and services – permanently. People all around the web are shocked and complaining. But the thing is: who didn’t expect this? Although no one came out and said it directly, no one spoke of WinFS except as a distant memory, it was quite obvious that people didn’t buy Microsoft’s story of it shipping separately. If people believed, the shock and outrage today would be ten times as big as it was when the LH project was rebooted and WinFS torn out with the strings still hanging. But the question many people are asking these long years later is: what is WinFS anyway? And what’s the big deal if everyone already knew it wasn’t coming?”
performance.
I’m sure there’s security reasons, disk size reasons, as well as other reasons that people would be considering.
But there’s many a techie like myself who look at the hard drive as the “lagging indicator” if you will. That’s the slowest part. So if you can increase the speed of your filesystem it affects nearly everything.
ReiserFS anybody?
WinFS was never designed to increase filesystem performance on Windows. The most specific description of WinFS that most everyone can agree on is that WinFS was intended to be a relational database bolted on top of NTFS in order to expose structured data interfaces to the Longhorn shell. This would, at best, have zero impact on traditional filesystem benchmarks. Instead, WinFS was targeted at the real “lagging indicator” of desktop systems. Not the hard drive, the user.
By expanding the use of metadata facilities already present in NTFS, MS can eventually provide the illusion of a rich structured data environment on the Windows platform (without the use of a standalone RDB). However, the implementation will be no where near as elegant, implementing new data management features for the Windows platform will be more complex, and such features wouldn’t be able to achieve as much code reuse.
The heart of the matter is that WinFS was not ready in time to provide the ubiquitous “desktop search” features that were absolutely critical for Vista. The Windows group had to implement desktop search directly on top of NTFS, and now that the Vista architecture is (hopefully…) frozen, WinFS is no longer relevant for Vista. Many of the innovations resulting from the WinFS project are, however, extremely relevant to the MS SQL Server product. The MS blog claims that they are “super-excited” to be productizing these features in Katmai. However, these features would have been integrated into SQL Server even if WinFS made it into Vista. When they said “super-excited,” they actually meant “consoled that at least some of that work won’t go to waste.”
As an aside, I’ve used ReiserFS, Reiser4, and XFS extensively on Linux 2.6.x, and I’ve found that ext3 with the “dir_index” option offers comparable overall performance with significantly less CPU and power usage. It also seems more stable and more consistent in its performance. ReiserFS and XFS are both excellent choices, the former especially for lots of small files and the latter for lots of large files. Reiser4, in addition to causing data corruption and coherency problems on mulitple occasions, has as many noticeable regressions as it does noticeable improvements. Your mileage may vary!!
In the quest for better performance, though, I’d rather crash my system trying the latest process scheduler than lose my data trying the latest filesystem (even if the data is properly backed-up). The best thing I can say about a filesystem is that it is rock-solid. I have never had any data corruption or performance issues with ext3+dir_index, and ext3 is the most mature and well-tested journalling filesystem for Linux. It is rock-solid.
Unless someone develops a SQL plugin for Reiser4 or someone ports ZFS to Linux, I believe my days of experimenting with filesystems are over.
Reiser4, in addition to causing data corruption and coherency problems on mulitple occasions, has as many noticeable regressions as it does noticeable improvements. Your mileage may vary!!
Reiser4 is still very new. ReiserFSv3 had bugs when it was first introduced too. Now it is fast and reliable.
Unless someone develops a SQL plugin for Reiser4 or someone ports ZFS to Linux, I believe my days of experimenting with filesystems are over.
I guess your days aren’t over yet.
http://www.wizy.org/wiki/ZFS_on_FUSE#download
I agree with most of what you say, and it’s too bad that WFS is(was) never intended as performance enhancement.
The issues surrounding reiserfs are something that I’ve read about and without really getting into an argument about this(which is not what my intention was) you could very well be right about ext3+dir_index.
XFS is what I’ve been using for quite a while now, and from what I’ve seen it is a little bit better of a performer than the others.
And I have no interest in experimenting with FS’s either unless they are well tested. XFS appears to be beyond it’s experimental phase.
What is to be found at
http://osnews.com/200
I just wonder…
C/P error. Fixed.
Wasnt WinFS similar in concept to ZFS?
http://www.sun.com/2004-0914/feature/
It’s the kind of news that hurt Vista a lot.
This just adds a big “we failed AGAIN” no matter how they try to explain it.
As far as I know ReiserFS still hasn’t steal the thunder in the linux world but I don’t know the reasons.
———–As far as I know ReiserFS still hasn’t steal the thunder in the linux world but I don’t know the reasons.———-
It hasn’t. But that wasn’t really my point. It was a general theme. One of the things touted(that I’ve seen anyways) about reiser is performance.
It does make a difference. Filesystem I mean. Some filesystems are faster than others.
ZFS is entirely different in concept from WinFS. WinFS is, in sum, about being able to access the filesystem via a powerful SQL/SQL-like mechanism. It also offered the ability to create and store substantial metadata (not just flags).
No, WinFS is not at all similar to ZFS. ZFS is best described as a pooled logical volume filesystem. In other words, disks are sliced into logical volumes that can be dynamically created, destroyed, resized, and migrated to other disks. ZFS logical volumes on any attached storage devices can be added to the pool at specified mount points. The OS recognizes the entire storage architecture as a single filesystem containing all of the volumes that are currently in the pool.
ZFS is also a 128-bit filesystem. Many people thought this was a ridiculous feature that goes way beyond reasonable headroom for future storage technologies. However, when you realize that (in enterprise IT) massive arrays of storage devices can comprise a single filesystem, it doesn’t seem that unreasonable anymore.
The only filesystems that even begin to come close to the feature-set of ZFS are IBM JFS2 and HP JFS (also called Veritas VxFS). Neither of these are to be confused with each other or with the JFS in the Linux kernel, which is a really old and comparitively primitive version of IBM’s flavor of JFS. Got it?
The idea behind WinFS was that everything has searchable metadata. Microsoft wants to integrate a search engine deeply into the operating system. To do this, they decided to take code from MSSQL, their database server, and base a filesystem off of it.
Any modern Linux filesystem with extended attributes enabled (ext3 or reiserfs), nautilus with it’s virtual search folders, leaftag for file/folder tags, beagle search, and deskbar applet does the same basic thing today what Microsoft wants to do tomorrow. Instead of making one huge complex beast that does everything. They should focus on perfecting small components and combining them all together like Linux/Gnome did.
Personal Opinion: I think WinFS was overengineered and was therefor doomed from greatness in the beginning.
http://qub333.googlepages.com/deskbar-demo.gif
http://arstechnica.com/articles/columns/linux/linux-20060328.ars/2
http://chipx86.com/wiki/Leaftag
WinFS wasn’t/isn’t about building a search engine into the OS — Vista already has the Windows Search indexing system (derived from MSN search), which is entirely independent of SQL and WinFS (as well as the terrible XP and earlier built-in context indexer). Live folders, instant searches, and the like are still in Vista; the “death” of WinFS has no affect on them.
Excellent post. One link is dead and one is to a story that’s only 3 posts down on the front page. Good research too, obviously no one thought it was still coming out when they demo’d it at TechEd a couple of weeks ago.
http://www.devx.com/dotnet/Article/31693
Maybe you should add a ‘Don’t recommend’ link on the posts so we could skip over gems like this one.
Since when is it my fault when websites don’t pay their rent?
Get off your high horse, sonny. You’re sitting in my place.
That was a joke people. Now laugh.
It’s very likely that he went over a bandwidth limit when a lot of people started arriving unexpectedly at his website — perhaps directed by a link from a popular high-traffic website. Now I wonder, where such a site might be. . .
I think filesystem performance is something microsoft overlooked/ignored for so long that they are behind with developing something that competes. It was something they didnt give much thought about and now it has come back to bite them again. They better be thankful they at least managed to get NTFS out there or else it would really be ugly. If they had hunkered downand really went to work back in the days of hpfs instead of throwing it out and then playing catch up. But same ol’ same ol’ for microsoft,make it look good first, sell it and then go ack and try to actually make something from it.
Even the name follows the same formula. A fast, high-tech, database-like filesystem. That’s what WinFS was supposed to be, and that’s what BeFS already was back when Microsoft got around to releasing FAT32 (shipping separately.) Aggressive marketing beats out a quality product every. single. time.
BFS and WinFS are two completely different animals.
BFS had metadata which was searchable.
This is not at all what WinFS is about.
NTFS supports metadata and it is searchable in Vista without WinFS.
WinFS offers more flexibility in how to display, work with, and store your information.
BFS was not this.
http://en.wikipedia.org/wiki/Be_File_System
“it includes support for extended file attributes (metadata) with indexing and querying characteristics to provide functionality similar to that of a relational database.”
http://books.slashdot.org/article.pl?sid=04/05/10/181231
“BeFS has some advanced features – arbitrary metadata, attribute queries, and indexing.”
It doesn’t just support metadata. It supports arbitrary metadata, and queries. It looks like the only differences between BeFS and WinFS were some lofty goals about userspace integration. And, you know, one has existed for ten years and the other used to be vapor but recently moved up to canceled.
NTFS supports metadata and it is searchable in Vista without WinFS.
That doesn’t mean that anything else to come out of the same company has to be completely distinct. Microsoft is famous for reinventing the wheel because the first one they made was “round enough.”
WinFS offers more flexibility in how to display, work with, and store your information.
WinFS offers nothing. It’s imaginary. It was going to do those things, but it was going to do them by incorporating characteristics of a relational database: indexing, querying, and applying arbitrary metadata. The “flexibility in how to display” would have required the cooperation of every app writing out files to the point of using MS-defined filetypes for everything (a rather transparent vie for more platform dependency). On second thought, I’m not so sure that’s really all that “flexible.”
Actually, Be had an arrlier file system (in BeOS PR1 + it is known as OFS), which was completely database orientated. It didn’t just allow arbitrary metadata to be associated with a file, but files could be tables of data. One of the original goals was to have no directory structure what so ever, and list all files using queries into logical groupings – thought hat never got carried through.
The point was that this was a bad design. It shared far more with the WinFS design – database bolted on to filesystem. Be then did the right thing and developed BFS (BeFS was never used to describe BFS by the way – that is a post Be Inc folding invention.) They lost a lot of the Database features, but kept the important acpects – queryable FS with arbitrary metadata thet is indexable.
Most files are used by programs – they don’t need to be indexed or even searched in their whole lifetime. Performance is the only thing matters for them.
MS should just provide an user-friendly interface to put all the silly documents into SQL Server
I think WinFS is best compared to GNOME Storage http://www.gnome.org/~seth/storage/
Hello OSNews,
We’re the site that’s down right now, we’re really sorry about that. Just to clarify, it’s not overdue bills, it’s slashdot, digg, and two entries on OSNews, all in 48 hours. We appreciate your hits, and we’ll be back shortly..
Regards,
The NeoSmart Team
Thank you for your patience.
We’d like to express our apologies that this happened, but matters were indeed beyond our control.
http://neosmart.net/blog/archives/202
It’s back now, enjoy!
…in all likelihood, WinFS concepts/interfaces/code will probably get rolled into future products.
…in all likelihood, WinFS concepts/interfaces/code will probably get rolled into future products.
That’s exactly what’s happening. They’re applying the technologies across all data stores, basing them on the entity data model and using LINQ/eSQL as the common method for addressing the data whether it’s on the client, server, DB, or services.
Well, the article is down so I can’t exactly quote it, but, WinFS is a relational filesystem: something that can’t run on a proprietry layer on top of the existing operating system.
The reason why Vista was a one-time chance is that the kernel was almost completely re-written, giving MS an opportunity to put hooks for WinFS in the kernel. What’s the use of WinFS if the OS itself doesn’t support it?
Does it sound like WinFS is something that can be added to the kernel in a hotfix? or even a service pack? It’s going to have to wait to blackcomb.
Well, the article is down so I can’t exactly quote it, but, WinFS is a relational filesystem: something that can’t run on a proprietry layer on top of the existing operating system. The reason why Vista was a one-time chance is that the kernel was almost completely re-written, giving MS an opportunity to put hooks for WinFS in the kernel. What’s the use of WinFS if the OS itself doesn’t support it?
This is not correct. It didn’t require Vista-specific hooks. Beta 1 and 2 of WinFS ran on Windows XP. It’s a hybrid store that, in its standalone form, was used to store file metadata, non-file items (schema based objects like Contacts), and relational data while using NTFS’s filestream support for handling file-backed items like photo and media files.
While that’s true, you can’t deny that if it had hooks <em>within</em> the OS, it would be exponentially more powerful. Without those hooks, it’s something that Windows Desktop Search and Google Desktop come close to emulating: not a filesytem, but a storage database.
Once it’s actually a part of the OS, you can do a lot more things, namely instant integration with any other program or service, bidirectional direct interfacing with the operating system, and improved system performance.
WinFS <em>did</em> bog down XP, but not Longhorn 4074 (the only LH-project build to contain WinFS), because it wasn’t another layer on top of the filesystem, but a part of the Windows API itself, implemented with hooks deep down into the core.
While that’s true, you can’t deny that if it had hooks <em>within</em> the OS, it would be exponentially more powerful. Without those hooks, it’s something that Windows Desktop Search and Google Desktop come close to emulating: not a filesytem, but a storage database.
No. The goals of WinFS were different than desktop search. Desktop search is just about indexing file text and metadata and allowing fast search. It brings nothing to the table as far as allowing you to create relationships between data types, providing a common type system, storage for and extensibility of non-file types, nor any of the synchronization and management features.
Once it’s actually a part of the OS, you can do a lot more things, namely instant integration with any other program or service, bidirectional direct interfacing with the operating system, and improved system performance.
It was already part of the OS. There were some limitations in the betas such as where a store could reside, but this didn’t affect functionality. The way WinFS works, you can’t just magically get instant integration with any application beyond that of reading and writing files, which was supported in the Betas. For any meaningful integration with applications, the developer is going to have to actually code against the WinFS API so their application can use features like creating relationships between types and exposing that to the user in some meaningful way.
WinFS <em>did</em> bog down XP, but not Longhorn 4074 (the only LH-project build to contain WinFS), because it wasn’t another layer on top of the filesystem, but a part of the Windows API itself, implemented with hooks deep down into the core.
WinFS bogged down the LH Alpha (possibly the only public build to contain WinFS) moreso than it did XP. In any case, MS doesn’t focus on performance optimization early in development. They focus on getting the code correct first, then optimizing for the most common scenarios and the ones they’re primarily targeting. Expecting large performance gains from WinFS vs NTFS is unrealistic (sans non-file-backed cached data) as it used NTFS for many operations.
Physical architecture has little to do with the developer API. In both cases, however, WinFS was very much seperate from but dependent on NTFS. The fact that it used NTFS in either case was simply a technicality though (meaning why rewrite and optimize file stream support when you already have an optimized implementation in NTFS?). MS could replace NTFS with something else to fill that role at any time.
WinFS was a part of the Windows API. Whether in LH or XP, it was part of WinFX (now known as .NET 3.0 – though the revisions to ADO.NET won’t ship until the next framework release). The only thing about that API that’s changed from 2003 to present is that MS originally created a new API (ObjectSpaces) for addressing data. Developer feedback from and following PDC suggested that developers didn’t want to learn another syntax (OPath – they also asked for scenarios MS hadn’t considered, causing MS to build for those scenarios as well). So MS built the data API with SQL-like syntax instead (eSQL/LINQ) and made it part of ADO.NET so developers would have a more familiar experience. The Entity Data Model that will now be shipping virtually everywhere there’s a store for data is what WinFS was all about. MS just found that there was a more general approach they could take in productizing the technology so that you could use one method for working with data whether it’s exposed as objects, via WinFS or SQL, or from a web service.
Edited 2006-06-26 14:26
“Get ready for WinFS”.. “Learn how to program file transactions”… boy I am glad I didn’t go to those Microsoft tech events. It would have been a complete waste of time. If I was in the US, I would sue them…
“Get ready for WinFS”.. “Learn how to program file transactions”… boy I am glad I didn’t go to those Microsoft tech events. It would have been a complete waste of time. If I was in the US, I would sue them…
Those sessions wouldn’t have been a waste of time as it still applies to System.Transactions in .NET 2.0, and items built atop the Kernel Transaction Manager in Vista (Transactional NTFS, Registry, Installer). It’ll also still apply later when the WinFS technologies are exposed accross a number of products.
Of course the blog entry means WinFS is dead for a while. It’s really disappointing first because it is (was) a very good technology, expecially the way it was developed in Vista (it was somewhat part of the system).
Second, it’s very bad how things change in Microsoft. You can’t create such an hype (many, REALLY MANY people where considering WinFS real and were developing on those BETAs…) and then, in a matter of weeks, trash the whole idea it pursued in past 2 years.
I believe MS should change the way they’re dealing with communities because we all welcomed public betas, software and tools availability and so forth. We all welcomed MS being much more open on upcoming technologies. However, it can’t end up like this, with people developing and using those technologies for months only to finally know they won’t be delivered. Very disappointing. Better to keep such things closed until they got 100% sure they will deliver them.
Edited 2006-06-26 11:22
I think almost anyone can agree that once Vista is released, heads need to roll. They need to make managerial changes and structural changes. Microsoft is too big and it’s employees are getting lost in the size. They need to readjust to learn to be more efficient with the employee base that they have. Microsoft has so much more potential, but they’re really blowing it right now.
They won’t kill themselves, they are just slightly wounding themselves right now. Eventually it will click, and hopefully that time is now.
http://blogs.msdn.com/winfs/archive/2006/06/23/644706.aspx
“”The Entities features we are now building in ADO.NET started as things we were building for the WinFS API. We got far enough along and were pushed on the general applicability of the work that we made the choice to not have it be just about WinFS but make it more general purpose (as an aside – this stuff is really coming together – super cool).”
Remind you of anyone?
😉
http://movies.apple.com/movies/us/apple/getamac_ads2/work_480x376.m…
Weakness on NTFS:
1. Windows NTFS do not do well against FS fragmentation. (UNIX employ writting stragety to eliminate fragment and ever Mac OS X has builtin code to defrag in backgroud). I wait MS since MS-DOS’s era to do what UNIX already did. Waste users $ buy defragmenter software.
2. 4K default cluster size for NTFS quite awesome. We can define to use bigger than 4K cluster size for NTFS to lower no. of fragmentation and improve I/O performance, but 3rd party defragmenters do not allow to defrag bigger than 4K cluster size (I test it under Diskeeper 9 and O&O Defrag 7).
In my opinions, elimination of FS fragmentation and > 4K cluster size at NTFS default (i.e. FreeBSD has chagne from 8K to 16K cluster size) are more benefit for user than Database FS (WinFS).
You’re right. But also consider that Vista keeps a defrag software running in background. At least, that’s true for Vista BETA 2…
On Windows 2000 they don’t.
But on Windows XP they do. At NeoSmart Technologies all our workstations have 32k cluster size, and it works miracles.
WinFS is the same old Microsoft “Let’s announce awesome feature ‘Foo’ so that our OS seems so much better than our competitors”. Who knows if they even thought they could pull it off…