About 3 years ago I was looking around for something to add multimedia capabilities to my GNOME desktop. At that point in time there wasn’t really that much around. I think the most advanced video player for Linux in those days was XAnim, which was neither were moving quickly or could qualify as free software, except in the beer context. Projects like Xine and mplayer had either just started up or not come into existence yet.
Introduction
Also my interest wasn’t purely aimed at playing back media files on my own machine, but also to see if there was something out there I could help push forward to give Linux developers and users something competitive to Microsoft’s DirectMedia and Apple’s Quicktime. Which meant I was looking for something which would also allow developers to relatively easily do more advanced stuff.
Anyway I started looking around and discovered GStreamer. I guess what pulled me in was the screenshots of the pipeline editor which gave me the clear feeling that this was more than just a playback application, but something that could be used for a much wider range of applications. Based on that I decided to do an interview with the developers for the news website I was involved with at the time (now gone). I guess I never left after doing that 🙂
The Basics
The core concept in GStreamer is that of a pipeline system which your media streams through. This means you have one or more sources which can be anything like a file, an URL or a hardware device. Depending on how you construct your pipeline you can then have lots of things happening to that media stream before it ends up in one or more sinks at the other end of your pipeline. The sinks can be like the sources; a web stream, a file or hardware device; all depending on what plugins and elements you have installed.
So what can happen in the pipeline? Well the possibilities are almost endless. In GStreamer there are some different classes of elements. There is the basic stuff like elements for decoding different formats, demuxers for splitting the audio and video into separate streams, muxers and encoders for merging the streams back together and encoding them in a format of choice. Then there is a class of filter elements. Filters can do things ranging from technical transformations like colorspace conversions, stereo to mono and vice versa, to adding effects to video images like make them look old or psychedelic or make the video look as if a bug is looking at it. There is no limit to the number of elements you can apply to a given pipeline except the limitations your hardware imposes on you. For instance if you are doing an application that needs to do work in real-time, it puts limitations on what kind of things you can do, because if you do too much your machine will simply not be able to do the computations fast enough. The GStreamer architecture is designed however so that the pipeline system itself adds no latency to the pipeline. This is a prerequisite for many types of applications which demand low latency.
GStreamer also contains an advanced system for negotiating capabilities. This means that GStreamer itself can assemble a line of elements that takes the input it gets and transforms it into a form that the output device you are using supports. So if your output device is something using the I420 colorspace but the video stream coming in is in the RGBA colorspace then GStreamer will be able to automatically assemble a pipeline that converts the colorspace for you. Since GStreamer handles these things itself, a developer who wants to write media applications doesn’t need to learn about colorspaces, bitrates and sound card clock speeds; GStreamer provides you with an easy to use API that lets you focus on your actual application instead of worrying about what kinds of things happen at the lower levels.
Developing GStreamer has been a lot harder than I think was anticipated when the project started out. Unlike many other free software projects, GStreamer was not a simple re-implementation of something which had been done before. I guess we did what Steve Balmer claims free software never does: we innovated. The basic design and basic idea came from a research project at Portland University, research work in which GStreamer project founder Erik Walthinsen participated. It was loosely modeled on DirectShow. This research gave us the basics, but when you take something out of the lab and place it in the real world, many new issues tend to arise quickly. This means that the last three years, we have had many re-writes of core modules as the original design needed extending in order to allow people to use GStreamer for more and more varied stuff and as real world worries such as support for legacy formats reared their head.
It is important to know that GStreamer has always been focused on two things : keeping the core media agnostic and keeping it GUI independent. In fact many of the first commercial users of GStreamer used it on the server for things like audio formats conversion at a telecom, recording and storing clips of live news and archiving at a radio station, and similar applications.
GStreamer in the embeded world
From the early stages, GStreamer had a focus on keeping the core small and nimble enough to be useful on various embedded devices. GStreamer’s first corporate sponsor was the embedded company RidgeRun Inc. working on a version of Linux for devices uses CPU’s from Motorola. Unfortunately their cash reserves ran dry before they could get some products to market. The legacy of this work still remains however, and GStreamer can still today be easily slimmed down to fit on even small devices. GStreamer is also the media player back-end of the GPE media player from handhelds.org. Philip Blundell maintains this player, he also made the Tremor Ogg Vorbis plugin in GStreamer.
GStreamer on the server
GStreamer has long been used by various groups and companies on the server side. Known uses have been as the backend for icecast webcasts, format converter at a telecom and media archiver at a radio station. Since the core GStreamer system doesn’t contain any GUI code it is perfect for making server solutions.
There is also a new company being formed around GStreamer these days which will among other things produce server-based multimedia solutions based on GStreamer. We feel confident that the creation of this company, which already has its financing taken care of, will help boost our effort even further.
GStreamer on the desktop
The GStreamer team has from the beginning been very interested in seeing GStreamer integrated into the free desktop projects. As free software developers we are very keen to see the Linux desktop really becoming competitive with the commercial desktops out there and we feel that having top of the line multimedia capabilities is essential to making this happen. From the GStreamer point of view having a media player is just the tip of the iceberg in that regard.
GStreamer never had basic playback as a primary goal : it was, and to some degree still is (even if ‘real’ world concerns has made us increase the focus on it) a secondary concern to us. What has been our focus from the start is to make sure that GStreamer is powerful and flexible enough to allow people to make advanced media applications like non-linear editors, advanced audio mixers, sound editors, softh synthesizers, transcoders and so on. By having this focus we feel we have been able to make a much more clean and extensible design than we would have gotten if we focused on media playback and then added features to support other things as an afterthought.
As part of our desktop push we got GStreamer integrated into GNOME 2.2 and recently moved GStreamer CVS and website to be hosted on freedesktop.org. While the move has little practical effect we hope its sends out a clear message to developers that we see GStreamer not as the GNOME multimedia framework but as the Unix multimedia framework.
The use of GStreamer in GNOME
We introduced GStreamer at a basic level in GNOME 2.2 and the integration is planned to go much further in GNOME 2.6. This is what we are working on currently. A new addition that developers and users will notice in GStreamer 0.8.x is that it has a mixing interface designed by Ronald Bultje. This means that whenever someone writes an element for a sound system for GStreamer, all the core applications in GNOME will be able to do mixing and volume control on that sound system. Today if you want to add Irix audio volume control to GNOME you would need to add it to a lot of separate applications like the gnome-mixer, the GNOME volume applet, ACME (the gnome multimedia keyboard daemon) and any other GNOME application which adjusts volume and so on. No more.
In GNOME 2.6 we plan to have all these systems use GStreamer which means that if a Irix audio system plugin is written for GStreamer then all the basic applications in GNOME can adjust the volume on an Irix system. Currently we have an OSS and an ALSA plug-ins which support this mixing interface, but there are work being done on trying to get a Sun Audio one also. Others will be added as they are contributed. Note that this new interface is for hardware mixing, software mixing we have of course been doing from the beginning.
In GNOME 2.6 there will also be two new applications using GStreamer. Sound-Juicer is made by Ross Burton and is a neat little cd-ripping application. Rhythmbox, maintained by Colin Walters, is an advanced music playback application comparable to Apple’s iTunes.
We are also working hard on offering good video playback and video encoding, so even if we will not bundle any video player with GNOME 2.6 I do expect that some of the distributors will start to bundle Totem or gst-player during the lifetime of GNOME 2.6. And then if things go as planned I guess we see a GStreamer-using video player and maybe even a video recorder bundled with GNOME 2.8. Of course the main problem here in regards to distributors in the issue of a lot of the common media formats out there being covered by a host of patents in some countries, with the US being the one hardest to ignore, which makes it problematic for most Linux distributions and Unix vendors to ship GStreamer or any other media framework with support for these formats enabled.
This distribution problem is why supporting formats such as Ogg Vorbis, Ogg Theora, Ogg Tarkin, Matroska and FLAC both in regards to decoding, but also for encoding has been a top priority for us. We want to be a enabler for people in regards to moving to using free formats for their own stuff. In fact we have a really cool announcements for the world soon in that regard.
The use of GStreamer in KDE
GStreamer is not officially part of KDE today which means that there is naturally a lot less integration of GStreamer into the core of KDE. The first release that might see GStreamer officially adopted for KDE is KDE 4.0, as that is first opportunity where the current arts system can/should be deprecated.
This means that GStreamer use is currently at a more experimental level in the KDE world with
Scott Wheeler’s JuK being the only major application which offers using GStreamer as its backend. It uses the KDE GStreamer bindings developed by Tim Jansen which are hosted in KDE CVS currently.
There are some other KDE projects apart from Juk looking into using GStreamer out there, so hopefully when the times comes for KDE to choose the choice will be easy to make. Not only will such an event make sure that multimedia development effort is focused on going forward instead of redundant re-implementation, but it will be a portal to taking things a huge step forward in regards to cross-desktop interoperability and co-operation.
Plans for the future
We are pretty happy with the current feature set of the core of GStreamer. So our focus for the next half year will probably be on polishing what is there and updating old elements and creating new ones.
The only major missing component is a subsystem for doing MIDI with GStreamer. Actually it is probably just a set of elements, but until it is actually implemented we can’t be 100% sure that the core doesn’t need some changes to accommodate it. We have had several people expressing interest in taking it on, but no one has actually committed some code for it so far, but hopefully someone will get a start on it soon, as it is really something we need in before starting to aim for GStreamer 1.0.
The new development series also includes support for interactivity which is needed for enabling such stuff as DVD menu’s and Flash animations to work through GStreamer. The interactivity support was created by David Schleef and is currently being completed and extended by Jan Schmidt. I am not sure we will have a finished DVD playback application or complete Flash support ready for 0.8.0, but the basics are there so we should be able to offer those during the 0.8.x series lifetime. Of course the flash support depends on interactivity being integrated into the flash decoding library Swfdec that David Schleef are developing in parallel with GStreamer. Developers interested in flash should really take a look at swfdec as it is the only flash implementation available out there available under a license as liberal as the LGPL, and it can be used outside of GStreamer.
Another fun new feature is our advanced new support for metadata/tagging designed and implemented by Benjamin Otte. We have devised a system which should be able to preserve your metadata when you use a GStreamer based application to transcode your files. So for instance if you convert some of your FLAC files to Ogg Vorbis with GStreamer, the metadata tags should be converted along with the music. Making an easy to use application with GStreamer for doing media conversions might be an idea for someone who want to learn how to program with GStreamer.
Another interesting development is that we currently got a team of about 7 french students who are going to make a GStreamer-based non-linear video editor as the final year project. This effort will probably improve the framework in certain areas like adding new advanced clocking elements for doing things like double speed playback. In relation to this we just completed a major change in our system which enables, among other things, splitting a video signal into two parallel windows. This means that in the case of a video editor you can view the same video signal in two different windows; the first could be the original video for instance while the second contained your transformations. A great way to experiment with different transformations in real time.
We also hope to integrate a SMIL implementation done by Malcolm Tredinnick in the coming months. This will actually not be part of GStreamer as such, but it is designed to integrate easily with GStreamer. SMIL is a W3C standard using XML that is meant for doing interactive audiovisual presentations. This is very useful for accessibility for instance, where we hope to see GStreamer tools written that will help popularize the addition of accessibility information using SMIL to formats such as Ogg and Matroska. We are especially excited about this as this will be the first implementation of SMIL available under the LGPL.
Other interesting improvements are the work on Quicktime and ASF done by Jeremy Simon, our improved AVI and Matroska encoding by Ronald Bultje, the color ascii art plug-in based on libcaca by Zeeshan Ali, our improved error handling by Thomas Vander Stichele and the software video scaling and gst-player and Totem development done by Julien Moutte. A more long term project that has been discussed (long term unless someone outside the current group of hackers steps up to the plate) is writing a new sound server using GStreamer.
On the competition
Don’t want to start a huge flamefest so I keep this one brief in the hope that the level of toe stepping will be acceptable :). We really feel that GStreamer is at a sweet spot at the moment with little real competition. Other projects within our sphere of application are either much more narrowly targeted (like Xine or MPlayer, which are great backends for creating basic playback applications, but in our opinion not well suited as backends for a wider more complex range of applications), or they are much more immature (like MAS and NMM), or licensed in a way that makes them uninteresting to most free software developers (like Helix). Or, even, a combination of all of the above issues.
Another major advantage of GStreamer in addition to our relative maturity is our licensing. GStreamer uses the LGPL license which we feel is the perfect license for a library. It allows plug-in and application developers the freedom to use the license of their choice (our choice for the applications we make ourselves is the GPL), yet reducing the chance of major forking or fixes being withheld, which is what a more liberal license could lead to.
GStreamer is also highly portable; we had reports of people running it on Linux, FreeBSD, Solaris, AIX and Irix. Recently I also got a mail about the core compiling on OSX with some soon to be merged patches attached. The code also compiles with other compilers than GCC, for instance Sun’s Forte. Porting to Windows shouldn’t be overly difficult if GLib works well. Possibly the only thing that would need coding is a custom scheduler.
Summary
Anyway, back to what got me started with GStreamer. As said it has been a long and at times though journey, but I feel that we have now finally gotten to the point both with the library and with available applications using it, that I feel that we are able to deliver what I originally wanted. Just the continuation of the projects and efforts that are already underway is proof enough that we have succeeded in delivering what I was looking for those years ago. That said we are still not done and there are of course still areas needing work, but that is just the way of software; it is not something static but a continuesly evolving entity.
I hope I managed to get you interested in GStreamer and what it can offer developers and through them what it can offer users. If you want to talk you can find us at #gstreamer on irc.freenode.net or you can get in touch through the mailing list.
I remember not so long back having loads of issues with gstreamer, but since i’m running the 7.* development series things are working great and it largely replaced xine-based media on my system.
Most people will only see this glory when 8.0 comes out, but it will be very good.
to the global domination of free desktops.
Wow… GStreamer sure is starting to rock.
the lack of a system like gstreamer is what drove me away from developing for linux. however, the lack of a c++ api for gstreamer made me choose the BeOS Media Kit instead, knowing that it would somehow probably make it to other platforms as well.
personal issues made me stop developing my application before it reached alpha stage, but if I ever decide to start developing it again I might perhaps take a second look at gstreamer if someone would just build a c++ wrapper around it.
gstreamer is very important for linux as this will make developing media apps as easy as on Windows or BeOS, so perhaps we will finally see more of them popping up in the next few years. Windows is still a bad system for music production, and will probably never be a good one since MS doesn’t care about that market. Linux can be tweaked to perform a lot better in those situations, and will hopefully never have the audio interface hell that windows has.
So are there any commercial counterparts that really offer anything simmilar to gstreamers goal?
In other words, can we expects a big nasty blood fest in a few years?
This is great article.
OS news has been producing some great articles lately and this one is right up there. Great work.
i suppose you have read the article before you posted.
so you may have noticed, that there is a c++ wrapper above gstreamer, it is in kde cvs…
well, maybe you do not consider qt-c++ (moc and stuff) as c++, but you may really look at the api before (which you obviously did not, as nothing forces you to write qt-c++).
maybe you only want a stl-based wrapper. i can understand this. afaik, this is not yet done. and propably will not be in the foreseeable future.
and why would you want this. if you do want gui apps, you will have to choose a graphics lib, assuming you do not want xlib (or something exiotic like fresco), you will have to choose either gtk– or kdelibs, well, you do have a gstreamer api for that language flavor.
so what do you think is missing?
~ibotty
I am sure, you mean by 0.7.* and 0.8.0.. 😉
and so far, none of it working right-out-of-the-box-with-all-kind-of-(streaming media) as wmp does.
seems there has to come a vendor of a proprietary player first (real/helix) to finally make that happen…
sometimes, the fragmentation of linux-projects makes one mad (from a users perspective). gstremaer might be a nice framework for future media applications, but i for one would prefer a player that just works and integrates properly for a start!
🙁
At least visit their site before making uninformed comments. 2 clicks off the main page and you’ll find C++ bindings under development for gstreamer. http://www.gstreamer.net/bindings/c++/
What about GStreamer and web cameras? Are these in GStreamer’s domain (regarding capturing/playingback and/or streaming video)? I am writing this because I need an ov511 driver for FreeBSD 5.2 and the only available is one named “vid” but is just a frame grabber and not a video capture driver (so it can’t be used with Gnomemeeting). Apparently they can’t port the ov511 driver to FreeBSD because FreeBSD doesn’t have a Video4Linux infrastructure. So, is Gstreamer sitting on top of V4L or can it replace it and so these stuff can be more OS agnostic?
Additionally, where is freedesktop.org’s HAL fitting into the whole Gstreamer picture here?
quote: “and so far, none of it working right-out-of-the-box-with-all-kind-of-(streaming media) as wmp does.”
First of all wmp is just a player. Gstreamer also has _editing_ capabilities. There are already a couple of nice video editors available that use gstreamer.
Secondly I think mplayer beats the sh*t out of wmp regarding supported formats. mplayer plays everything wmp does, plays divx/xvid without having to install 3rd party plugins AND plays quicktime/realmedia out of the box. Beat that
(Xine is also very good btw)
I agree with the author that gstreamer has a lot more potential than mplayer, because well.. it’s just a mediaplayer and nothing more *g*
quote: “sometimes, the fragmentation of linux-projects makes one mad (from a users perspective). gstremaer might be a nice framework for future media applications, but i for one would prefer a player that just works and integrates properly for a start! ”
I believe projects like Gstreamer make it possible to reach a level of integration that commercial software can never reach. There are already great gstreamer based players out there (totem for gnome). I personally believe gstreamer is not yet ready for (my|the) desktop, but it’s definately usable and getting better every day. If you want a nice integrated player _today_ that supports all your video and audio formats without hassle, stick with mplayer or xine for now. There are nice frontends available for both that integrate perfectly with the gnome and kde desktops (just take a look at freshmeat).
I love GStreamer. I love Ruby. guys, life’s just so sweet sometime, thanks!
In the article claims that swfdec (swfdec.sf.net) is the only open source decoder of swf. What about GPL Flash (http://www.swift-tools.net/Flash/)?? It’s way better (and older) and support interactive stuff (buttons etc.)
If you actually read what I wrote you would see that I said it is the only swf decoder available under a license as liberal as the LGPL.
what will happen when MAS will become ‘the facto’ in the X world? (will be included by Xouvert R2 and is sponsored by X.org and FreeDesktop.org)
MAS is a competitor of GStreamer?
I *love* the concept behind Gstreamer. I had to implement a server-based multi-client audio/video multimedia-thingie for use in a museum at work. While trying the different opportunities i stumbled across gstreamer and realized that it just was what i needed. however, back then i had problems implementing it due to weird errors and time constraints. in order to at least have something valuable for the customer to see i did a quick & dirty flash/xml clientserver thingie.
great to see gstreamer mature, now if i’ve to work on the before-mentioned project again i’ll definately try gstreamer again.
Well there is some overlap. MAS is however more aimed at being a sound/media server so the overlap doesn’t need to be that big in practice.
That said I don’t think many in the GNOME and KDE community takes MAS serious anymore after their abysmal track record for the last years.
“Secondly I think mplayer beats the sh*t out of wmp regarding supported formats. mplayer plays everything wmp does, plays divx/xvid without having to install 3rd party plugins AND plays quicktime/realmedia out of the box. Beat that
(Xine is also very good btw)”
mplayer doesn’t allow for fullscreen-video on my ati pro 8 mb notebook (at least i don’t know how to achieve it)
besides, the last time i checked the mplayer-plug for mozilla, it rquired a good amount of tweaking to get it work with the (embedded) gui.
as for xine, their mozilla-plug is afaik still in development (since ages) and from my experience, very unstable/unreliable.
also, xine is a resource-hog – with some divx, i can’t get fullscreen-video due to processor (pIII-600). with wmp und xp and the same movie – no problem.
it seems one of the downsides of oss-development that people are eager to code (new) exiting stuff, pursuing new approaches (resulting in functional overlapping projects and rel. thin and shattered manpower), while “filling the gaps” has a very low priority – doesn’t make fun.
this is understandable, but from a users perpective – quite enerving!
Some areas I think should be covered wider in article but were not:
Personally, I consider one of the strongest points of GStreamer to be its total independence of formats supported, and also ability to add / remove them through plugins, and not by recompiling. This means there is nothing like “list of formats supported by GStreamer”, as supporting is just a matter of writing appropriate plugin(s). As of today, it would be even impossible to say exactly what is supported by GStreamer, as every once in while someone on mail list announces plugin we weren’t aware of
. This is unlike how for example MPlayer handles that, and is more similar to what i.e. DirectShow does. Also, core GStreamer is very much independent of what plugins do, as set of features provided by core library is surprisingly small, although generic.
That separation allowed for example to implement aforementioned tagging almost exclusively by means of plugins, with very little extra provisions from core. Also, flexibility of architecture is clearly shown by GStreamer’s first DVD-Nav capable app ever, which was gst-launch – generic commandline tool for constructing pipelines
. Navigation support was achieved solely by work on plugins, without even touching gst-launch itself. Next one, gst-player (GStreamer’s GTK player) gained it by changing only the 40 lines necessary because it catches mouse events.
This abstraction is lifted even higher by libgstplay, hacked on by Julien Moutte, that allows apps to request convenient media playback virtually without knowing anything about the whole process.
All that makes possible what was traditionally weak point ot Linux and Free OS’s in general – powerful multimedia handling without the need to reinvent the wheel everytime someone needs it.
Gstreamer is perhaps the media framework open source Unix needs to turn it into a formidable media platform. Already open source Unix has the best media players available, Mplayer and Xine.
With Gstreamer’s editing capabilities, I see professional media developers flocking to *nix when Gstreamer matures and when media applications begin to take advantage of its robustness and flexibility.
Who says open source doesn’t innovate? That being said, the last time I used Gstreamer via Totem as a front end, I couldn’t play DVDs or CDs or a number of media formats– real player quicktime.
That was a major turnoff. It’s always a good idea to begin supporting basic infrastructures used by one and all, if only to garner the attention and support of users and potential developers, before concentrating on advanced features useful to a niche few.
Based on my experience, I immediately concluded Gstreamer wasn’t as mature and as useful as Mplayer and Xine were, of course until I read this article and revisited Gstreamer’s website. I’m positive several other users, supporters and potential developers made such quick and haste judgments based on their experiences if they were similar to mine.
So to the Gstreamer developers, great work, but if users can’t play their CDs or DVDs using a Gstreamer front end, you encourage misconceptions and false notions about Gstreamer’s usefulness and maturity to them. Now I know better, but I’ve been a victim of such shortsightedness and so are many others.
Well, after reading the comment titled “Additional notes” by Mathrick, perhaps I didn’t have the appropriate plug ins installed to playback CDs and DVDs. So, I ask humbly, are there plug ins available to playbacks DVDs and CDs?
So, out of interest, if I wrote a decoder for, say GIF files, and an encoder for, say PNG files, could the GSTreamer framework allow me to write an image conversion application (which could later be extended simply by adding more plugins)?
If so, this reminds me of the old Amiga datatypes mechanism…
“Datatypes is basically an extension of normal shared libraries (or DLLs) to provide generic data handling capabilities. With this facility, any datatypes-aware program–whether viewer, web browser, or image editor–can be extended after the fact, simply by adding the appropriate datatype for whatever new format comes along.”
Sweet!
yeah, I’ve should have taken a better look at it. sorry, I was so tired when I wrote the post that I didn’t bother. at the time when I considered using gstreamer it didn’t have c++ support afaik. but that was some time ago. good to see that it’s getting along.
It looks cool but its not GPL – its LGPL right ? I would think that a industry standard for linux would need to be truely GPL in order for widespread support from developers out there.
Stop trolling. If it isn’t the 12.*.*.* trolls out is the comcast trolls at it again asking stupid questions regarding licenses. Here is a hint sonny, jump over to gnome.org and count how many libraries use LGPL then make that same comment again.
Please don’t forget that GStreamer exists for over 4 years now. It still has NO stable API and is not very well implemented into GNOME either. I think that after 4 years of hacking on GStreamer it should have reached a point where the API is stable and where it’s integrated correctly into GNOME (and not half). Maybe a hint for the developers to work towards an serious API freeze pretty soon and make it become stable and reliable.
So, out of interest, if I wrote a decoder for, say GIF files, and an encoder for, say PNG files, could the GSTreamer framework allow me to write an image conversion application
Yes, exactly.
In fact you application would be a simple one line command line
gst-launch filesrc location=file.gif ! gifdec ! pngenc ! filesink file.png
perhaps I didn’t have the appropriate plug ins installed to playback CDs and DVDs. So, I ask humbly, are there plug ins available to playbacks DVDs and CDs?
Yes, there are cdplayer and dvd{read,nav}src. cdplayer isn’t anything new, and what’s interesting is dvdnavsrc as it’s what gives navigational capabilities. Everything is basically there, now only playback smoothness needs to be polished (currently navigation events may block playback for a short while)
What about GStreamer and web cameras?
Yes. Web cams show up as video4linux (or v4l2) devices. GStreamer will support these, but the current support is sorely lacking. Encoder plugins or whatever they’re called is one of their goals, though, so this should be improved greatly by 1.0.
I only wish the .net bindings were in better shape (i.e. buildable). That would really make using this framework a pleasure.
“mplayer doesn’t allow for fullscreen-video on my ati pro 8 mb notebook”
…
“also, xine is a resource-hog – with some divx, i can’t get fullscreen-video due to processor (pIII-600). with wmp und xp and the same movie – no problem.”
synergy, that’s not a problem with MPlayer or xine, but with your video driver. You need hardware acceleration for fullscreen video where the scaling is done in hardware, instead of the slow CPU-based scaling. It works well with the NVIDIA cards, ATI now also offers Linux drivers, but their older models might require the GATOS driver for XVideo support (that’s overlay in XFree86 – the hardware video acceleration).
Greate article Christian, what about integrating animated SVG into GStreamer?
The paragraph about the competition isn’t very complete, I think. He says that GStreamer has been doing software audio mixing for some time, so that would make esd (and asd) competition, as well as aRtsd. aRtsd also does MIDI sequencing (I believe), and as that’s a stated goal of GStreamer, that would be competition on that front as well. Plus, ALSA now has its dmix capability which allows sofware mixing to happen at a lower level (either in the kernel or the ALSA library).
He also doesn’t talk at all about network transparency, which is the point of MAS, so I imagine that GStreamer does that as well.
Basically, there’s a lot of overlapping competition in two areas: per-soundcard, software audio mixing and network-transparent audio. Personally, I feel that ALSA’s dmix is the best solution for the former (and I imagine that GStreamer will be able to take advantage of that in its ALSA output plugin), and I also think that network-transparent audio is a solution looking for a problem. I don’t think that any setup that has thin-client workstations won’t be that concerned about having its users play mp3s.
But it seems that GStreamer does other things, too, but as he stated, user playback hasn’t been a priority for them (and it is for most users, which is why we’re all still using XMMS…)
Great article, but all the way through I kept thinking
“this is like virtualdub!” (except better, of course).
How come there isn’t a virtualdub under linux?
The technical reason is that VDub is depending heavily on VfW (Video for Windows) and that technology just isn’t portable
If you meant ‘a VDub like application’ then I suggest you take a look at avidemux2 http://avidemux.tuxfamily.org/
The idea of people working on an NLE for Unix is exciting. I hope they will have a serious look at the only 2 professional NLEs currently available (on Mac/Windows): Avid and Final Cut Pro. And that they will spend a lot of time talking with professional film editors.
The paragraph about the competition isn’t very complete, I think. He says that GStreamer has been doing software audio mixing for some time, so that would make esd (and asd) competition,
No, as much as SDL isn’t competiting with OpenGL
as well as aRtsd. aRtsd also does MIDI sequencing (I believe), and as that’s a stated goal of GStreamer, that would be competition on that front as well.
AFAICT, aRtsd is kind of failed experiment, and also is good suited for audio synthesis, but not so good as general media processing framework, at least not w/o serious hacking. GStreamer already has that.
Plus, ALSA now has its dmix capability which allows sofware mixing to happen at a lower level (either in the kernel or the ALSA library).
See esd comment above
He also doesn’t talk at all about network transparency, which is the point of MAS, so I imagine that GStreamer does that as well.
It is entirely possible (see Ramon’s rt(s)p work), but not essential, and definitely not something worth making central point in design. GStreamer just allows that, but doesn’t impose it itself. Also there is gst MAS plugin, despite MAS being (IMO) vaporware.
Basically, there’s a lot of overlapping competition in two areas: per-soundcard, software audio mixing and network-transparent audio. Personally, I feel that ALSA’s dmix is the best solution for the former (and I imagine that GStreamer will be able to take advantage of that in its ALSA output plugin)
I don’t see any reason not use it
and I also think that network-transparent audio is a solution looking for a problem. I don’t think that any setup that has thin-client workstations won’t be that concerned about having its users play mp3s.
Raw audio streams on network doesn’t sound too good either
. But as w/ everything in GStreamer – you *can* do it, but don’t *have* to. GStreamer can transparently add network transparency ;P
But it seems that GStreamer does other things, too, but as he stated, user playback hasn’t been a priority for them (and it is for most users, which is why we’re all still using XMMS…)
Not really true. Playback *is* important, but it’s not *the* ultimate goal. Having framework that allows for playback, as well as encoding and transformation, is, however. And currently, playback is good enough to be used on daily basis. I myself use gst-launch as my primary video player (no seeking, no pausing, nothing more than video window and sound output
and it works really good.
Well with animation being added to librsvg now I guess we will get a librsvg based plugin as soon as that is done.
Hopefully someone also makes a KSVG plugin since that would enable displaying SVG animations right away.
NMM is really good. I used it. They have lots of plugins, and they are doing big steps foreward. And it’s possible to use it via network.
*Another* framework?
I agree Linux is really lacking a standard audio framework, esd and artsd really sucks. But now we have JACK, MAS — and GStreamer!
I would be much more interested in how these compare for features:
* JACK has been worked on a lot to minimize latency. How does GStreamer compare in this regard? It is probably the most important aspect of media production. Perhaps there are befinits of co-operation with the kernel scheduler here?
* GStreamer sounds like it has a lot going for it in transcoding different codecs. Would an application like VideoLan become extremely simple when built on this? Did you guys look at their code and/or design, which is really proven in practice?
* MAS has the X Consorium’s blessing. This may be more important than ever now when X seems to be undergoing some political changes. When/if Xouvert takes over, will they choose MAS, JACK, GStreamer or something else?
* What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.
In 5-6 years time there would be a fully functional media playing and editing framework for Linux that just works and is well integrated with the desktops and APIs.
Until then, either use Mac OS X or Windows or compromise.
*Another* framework?
I agree Linux is really lacking a standard audio framework, esd and artsd really sucks. But now we have JACK, MAS — and GStreamer!
GStreamer is *media processing* framework, not audio output one. In fact, gstreamer can use any of these for sound manipulation / output. You should think of these project as being complementary, not directly competing
I would be much more interested in how these compare for features:
* JACK has been worked on a lot to minimize latency. How does GStreamer compare in this regard? It is probably the most important aspect of media production. Perhaps there are befinits of co-operation with the kernel scheduler here?
I don’t really know much about RT, but AFAIK there is currently no hard RT support in gst, and adding it would require some (minor, mostly advertising RT capabilities) tweaking on core, and heavy hacking on plugins that want to be RT safe. Note also that GStreamer itself doesn’t introduce any latency and is capable of using JACK. Andy Wingo did some work on RT cleaniness so provided you don’t use any unsafe elems in pipeline, it should preserve RT.
* GStreamer sounds like it has a lot going for it in transcoding different codecs. Would an application like VideoLan become extremely simple when built on this? Did you guys look at their code and/or design, which is really proven in practice?
Yes, it could be much simplified. But then, VLC is bit of framework itself, so switching over to GStreamer would be invalidating much of its code. As a sidenote, GStreamer is also proven in practice
* MAS has the X Consorium’s blessing. This may be more important than ever now when X seems to be undergoing some political changes. When/if Xouvert takes over, will they choose MAS, JACK, GStreamer or something else?
http://www.mediaapplicationserver.net/public-mhonarc/mas-devel/
IMHO project with 30 mails since 2002 is dead and already stinking. AFAICT MAS has been just-around-the-corner ever since its beginning.
* What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.
That’s even better. I’m no KDE guy, but from what I know KDE is currently looking into possible solutions for mmedia. And GStreamer works, is already used in JuK, and will be probably supported in amaroK player. Moreover, choosing GStreamer will definitely be step into direction of greater interoperability, as there is currently no other widely used framework with comparable capabilities. One of the main concerns seemed to be dependency on GLib, but since it’s now required by Qt it shouldn’t matter anymore. So from pragmatic point of view, GStreamer is only natural choice.
What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.
There is no consensus yet among KDE developers. You can read in the mailing lists that there has been a lot of different point of views. We had at one point a lot of stability problems with aRts and very few developers except Stefen Westerfeld knew how to fix the code. My guess is that we will be prudent in dumping aRts and making bold experimentations.
My feeling is that the preferred solution for KDE would be a simple solid audio server for users like me who have basic needs (listening MP3, watching a DVD), more or less what Alsa provides + network transparency which is important for thin clients and a optional complex multimedia server used by content creating applications (music, video, etc.).
Since when did Qt *require* upon GLib? (Nothing against GStreamer – I think it’d be good fro KDE/GNOME to use the same framework)
Since when did Qt *require* upon GLib? (Nothing against GStreamer – I think it’d be good fro KDE/GNOME to use the same framework)
Ever since ATK support was added for accesibility reasons
In your competetion survey you don’t mention the JACK framework. I’m coming from the music production & sound
enginieering domain and the JACK seems to be really very
prominsing. Especially as real-time constraints and latency
problems have been considered very well. And in some way
JACK seems to be a very promising standard for sound
applications in linux.
Admitted, it is currently not about multi media, but may be,
it can be integrated into gstreamer for sound/audio processing?
MIDI-Support is on the way. What do you think?
I think Windows, BeOS, and Mac OS X already have this market covered.
Have anyone taken a look at Mplayer G2? At least their plan seems to be to make a library, that should be easy to use for both a player and for instance a non linear editing program without losing the speed of the old mplayer but keeping compatibility with all formats. The old mplayer doesn’t seem to be a contender to gstreamer, but the new one does. Of course it’s not ready yet.
Just a little advice, and please don’t take it the wrong way. The second to last paragrah has a sentence that does not make sense and it is run-on.
Quote:
“As said it has been a long and at times though journey, but I feel that we have now finally gotten to the point both with the library and with available applications using it, that I feel that we are able to deliver what I originally wanted. Just the continuation of the projects and efforts that are already underway is proof enough that we have succeeded in delivering what I was looking for those years ago.”
There are others please proof read so as not to degrade the quality of your work.