Unlike most (all?) other distributions with built-in Flatpak support, Fedora maintains its own repository of Flatpak applications. Everyone else defaults to using Flathub, where developers of applications themselves tend to publish their Flatpaks. Fedora’s ‘shadow Flathub’ sometimes leads to problems, with Fedora-made Flatpaks containing bugs and brokenness, while presenting themselves as official, developer-made Flatpaks. In turn, users complain to the developers, while the issues they experience are actually caused by Fedora making its own Flatpaks.
One of the applications this happened to is OBS, and over three weeks ago the OBS project requested that either the broken, unofficial Fedora Flatpak be removed, or that it be made clear that the Flatpak was third-party. This request seems entirely reasonable to me, and it would be fairly trivial for Fedora to do this. In fact, I think respecting this request is merely common decency. Sadly, the Fedora project thought differently, and just… Ignored the request.
And so the OBS project escalated the issue.
This is a formal request to remove all of our branding, including but not limited to, our name, our logo, any additional IP belonging to the OBS Project, from your distribution.
Failure to comply may result in further legal action taken. We expect a response within the next 7 business days (By Friday, February 21st, 2025).
↫ Joel Bethke
It seems this caught the attention of the Fedora project, as within less than 24 hours, a formal request was made by the maintainer of Fedora’s OBS RPM package to have the broken OBS Flatpak removed. It seems there’s no official process to follow for making such a request, but I hope it gets through and honoured, if only because, like I said above, it would be common decency to do so.
I do wish to go back to the original OBS complaint, though, as it poses the question most of you are asking yourselves at this point.
I would also like some sort of explanation on why someone thought it was a good idea to take a Flatpak that was working perfectly fine, break it, and publish it at a higher priority to our official builds. We spend an enormous amount of effort on our official Flatpak published to Flathub to ensure everything is working as well as it can be.
↫ Joel Bethke
Why does Fedora maintain its own shadow-Flathub, set at a higher priority than the real Flathub? There’s a few reasons, as detailed in this Fedora Magazine article from 2022. There’s the obvious stuff like Fedora only allowing free and open source software, whereas Flathub also allows proprietary software, meaning that if Fedora ships with the Flathub repository enabled and prioritised, it would violate Fedora’s policies. You can argue back and forth about this, but Fedora’s policy being what it is, I can see where they’re coming from. The article mentions Flathub will split proprietary applications from free and open source ones, but I can’t find any word on if this has happened already.
A second big difference are the sources where the Flatpaks are drawn from. While Flathub allows and all sources, with their packages reusing Debian packages, Ubuntu Snaps, tarballs, AppImages, and more, Fedora exclusively reuses its own RPM packages when creating its Flatpak packages. Furthermore, Fedora Flatpaks use the Docker-like OCI format to publish applications (which ties into the Fedora Registry), while Flathub uses OSTree. Lastly, Fedora Flatpaks use one, single, big underlying runtime, while Flathub has a number of different, smaller runtimes.
The issue here seems to be that the motivations for maintaining a Flatpak repository differ greatly between Flathub and Fedora, but one has to wonder how much of that actually matters to users. Maintaining your own, separate Flatpak repository that effectively duplicates the work developers do when publishing to Flathub is not only wasteful, but also prone to cause bugs, issues, and outdated Flatpaks – which in turn causes strife with the original developers of the applications who have to deal with problems causes not by their own work, but by Fedora – problems that they can’t even fix.
I don’t think this situation makes any sense to perpetuate, and it’s high time Fedora defaults to Flathub for Flatpak applications. It will reduce the workload on package maintainers, prevent needless packaging bugs, improve the experience for users, and make developers happier. It’s a no-brainer at this point.
This is a tough one because I’ve long valued the work put into a distribution’s packages, and I specifically value Fedora’s goals of ensuring things are FOSS and redistributable. That said, I’ve always had RPM Fusion’s free repo installed for patent-encumbered codecs and I absolutely favor Flathub flatpaks on my Fedora systems. I can understand why the Fedora Project would want to ensure users have access to as much software as possible while keeping to their goals and standards, but I feel like that’s incompatible with what flatpaks are good for. Let RPMs be RPMs, and let most flatpaks come from Flathub.
So do I, and installing RPM Fusion is also the first thing I do on a fresh workstation install. LOL
Isn’t the biggest problem that Flathub is filled with anonymous submissions, with just enormous numbers of Flatpaks being uploaded by anonymous users that have no association with the projects and no discernible reason to be publishing software in the first place?
I’ve used the Fedora Flatpak repo before. I’m no Fedora fan, but at least there’s a responsible organization standing behind their flatpaks.
I refuse to use the random, anonymous junk that passes for legitimate software in so many cases on Flathub. For example – the Protonmail flatpak on Flathub – That’s a super intensely important app, you really need to know who’s been involved in building that app and submitting that flatpak. On Flathub, it has the official Protonmail icon, looks legit, has lots of verbiage that looks like it’s from the official Protonmail page, all the links point back to Proton, but in small letters “Unverified” and then one line about it not being a verified app. No information on who the actual flatpak author is. This is literally one of the most security-critical apps you can possibly be running, and Flathub gives you no way to verify who made the flatpak version or why you should trust them in any way, and any problems you are going to be sent back to Proton to sort out, even though Proton didn’t build the flatpak. If you have trouble with the flatpak or it starts stealing your data, who’s going to help you, who’s going to stand behind it? Certainly not “flathub”, whoever that is. Certainly not the anonymous users who put together this unverified version.
That makes me think a good solution might be for Fedora to only offer flatpaks for applications that don’t have a Verified option in Flathub and to maybe have a Verified filter by default when Flathub is enabled. OBS is such a great example of something I 100% want from the OBS Project itself, but I’m sure there are far more like you describe where Fedora’s build is likely to be more trustworthy than some rando’s unverified submission to Flathub (not that they don’t have standards of their own).
That’s a good point. Fedora’s problem, I’m assuming, is that they have to stand behind anything that’s in any of their repos, and there are real, money paying customers of RedHat that will expect some level of organizational support. So I can’t blame them for wanting to build all their own packages.
It’s one problem, and yes. Chicken and the egg though. There has to be software for people to use a distribution mechanism, but there has to be people using the distribution mechanism to get software published to it.
It’s getting better, but companies still don’t officially publish their software in Flatpaks. There are still many appimage only, package only, or tarball only apps out there.
I generally don’t. Their flatpaks don’t work very well. lol
That would be the Gnome Foundation.
“Flathub gives you no way to verify who made the flatpak version or why you should trust them in any way,”
Actually it’s all quite clearly labeled on the Flathub website, and most of that same information is surfaced in the GNOME and KDE software manager interfaces to Flatpak. If you’re running random “flatpak install” or “flatpak run” commands without checking provenance that’s not much different than running “curl | sh” to install software.
Flathub makes possible to filter packages in search results at their website. If I searched “writer”, I got 93 results. In the left there are options like FOSS/nonFOSS and Verified/Not verified. If I select FOSS+Verified, then subset is down to 43 results. Most users would probably be able to manage that.
The page of each package has section Links and therein Manifest leads to GitHub page where you can see who did the packaging. Fore example in the case of Proton Mail the packagers identity rises questions who he/she is and in the case of Butler, it is clearly the developer. However many users probably would find digging into this pretty confusing.
Then there are appstores. If I do the same search in KDE Plasma’s Discover, then:
– there is no way to filter by FOSS+Verified
– there is no way to see manifest and dig into who is behind packaging
I don’t know if Flatpak’s Portal API even exposes this data to be used. And if yes, then why this has not been implemented.
And then there are many other Linux DE appstores with their UI’s and implementations.
So yes… many things are fine, but there is still a long way to go. Especially to make trustability easier and more transparent.
Problems between distro packages and upstream is why Fedora doesn’t like carrying out of tree patches, and has a policy of upstreaming as much as possible. So fairly ironic. lol
Flathub implemented a non-free filter fairly recently, by the way.
I’m not sure why they chose to use OCI instead of ostree.
The article below from 2018 suggests it might been because RH felt it was better for their target market to demo the tech.
https://opencontainers.org/posts/blog/2018-11-07-bringing-oci-images-to-the-desktop-with-flatpak/
From the article:
There are a few RHEL workstations out there, and teams building server containers could add desktop containers too.
The article also points out the OCI container format supports better metadata.
I’m not sure if it’s still true or not, but it’s in the article.
Yeah, I think you nailed it. OCI fits more with other Red hat technologies they were trying to push, it fits more with what servers are used to, single run time, better metadata inside the flatpack itself. It makes a lot of sense. But if the upstreamers have their own flatpack, I wouldn’t invest the resources into building a OCI specific one, without really good reasons. I’d leave the OCI, free source only Fedora repository to only flatpacks that are critical to have for desktop and server use. If you duplicate one already on ostree in flathub, you must have an automated build set up that duplicates the flathub in OSI upon its publish to flathub.
Lets say you want to run TeaPot (the best cli spreadsheet application bar none) how do i install it and no “flatpak” is available on a immutable system? Do i like in blendos have to download docker stuff?.
i dont like where this is going, but i am aluddite, i rather use runit (since is so much faster) than systemd and i prefer pipewire over pulseaudio, i prefer XFS over EXT1/2/3/4 and Btrfs since it is much faster and less damaging to the cache chips on an SSD. If you are still on a mechanical drive, the upgrade makes little sence.
Pipewire is wired in to systemd socket activation. It needs ugly shell script hacks to work otherwise, like the ones Gentoo provides.
Funny, that sort of thing is what snaps are designed for. For the actual question, all immutable systems I know of still allow you to install traditional packages.
> all immutable systems I know of still allow you to install traditional packages.
No, they do not. I suggest that you need to look at more different immutable distros as your sample set is not representative.
Endless OS does not.
Ubuntu Core does not.
I have not done extensive testing of them but the immutable Fedora versions I have tried (e.g. Silverblue and Kinoite) do not.
Every single immutable distro includes Flatpak and leaves /etc+/var writable. For something like Bluefin that ships Homebrew, just use “brew install”. Otherwise spin up a container.
> Every single immutable distro includes Flatpak
Not true. For example, Ubuntu Core does not. First released in 2014 and so one of the oldest immutable distros.
The Fedora immutable systems use ostree to layer packages on to the install.
rpm-ostree install teapotwould install teapot as a layer on the main system.I’m not sure how EndlessOS (Debian) or OpenSuse Aeon accomplish installing packages.
There are also various directories which get mapped to writable directories. Home is always writable and /usr/local is linked to /var/usrlocal which is writeable as well.
I’m still fairly new to immutable systems, and there are still things I’m learning about them.
The issue is not Fedora or Flatpak, Snap, or AppImage.
Hidden behind the trees is the fact that Linux doesn’t exist as an OS, and even “universal” packaging methods have failed to achieve any sort of sanity.
Let’s waste a couple more decades building incompatible software compilations that are on average supported for at most 24 months. It’s worked great so far, right?
Oh, wait, most people here will be dead by then. And many will continue to wait for the year of Linux on the desktop.
Not. Going. To. Happen. Ever.
Artem S. Tashkinov,
You should probably clarify that you are linking to your own article. Anyway, the problem isn’t the linux kernel, but the linux distros that make up the whole OS. The “linux” portion of the OS is actually the least likely to cause problems, in fact I’d say kernel-wise linux reliability and compatibility are extremely high! It’s hard to target linux distros generically because everyone provides a different base install. Not only different dependencies, but different versions of those dependencies. It’s like “DLL hell” on windows. Traditionally linux distros solve this by having a centralized maintainer who takes responsibility for managing dependencies. Without them, DLL hell returns.
There are basically two ways to solve this:
1) Standardize minimum baseline dependencies (more like freebsd).
2) Bundle all needed dependencies (more like android).
#1 is hard to do in practice without coordination and agreements between vastly different operating systems. I don’t think different linux distros can reach the consensus to get on board here.
I’ve seen first hand just how much bloat gets amplified with #2, but letting each project manage their own dependencies does help work around the lack of consensus over what goes into the base.
While Flatpak (etc) don’t completely abstract the entire OS, they really do help. For better or worse some important projects like libav/ffmpeg have a history of breaking ABIs to great frustration for software outside of the repos. Switching to flatpak versions of these applications is so much easier. It’s not ideal to install several different versions of a library, but it’s also unavoidable without a centralized maintainer.
I think what’s controversial is the following question: should linux distros continue the practice of centralized maintainers supporting official FOSS packages. Or do they need to evolve towards decentralized packages like flatpak/snap/appimage in order to better accommodate 3rd party software? Not everyone wants this change to happen.
While I think it’s important for linux to support 3rd party software (meaning not in the distro repos), it does kind of fly in the face of linux repos and their benefits. So I think this is going to continue to be controversial for a long time. But maintaining centralized repos is expensive and I predict that decentralized packages will keep growing in popularity.
The problem is this idea (repos) has never actually worked.
Your particular distro:
* Doesn’t package all the available open source software (not enough maintainers)
* Doesn’t package the open source software that is no longer compatible with your distro libraries
* Doesn’t package software with “source available” if it’s “incompatible” with your distro “ideas”
In essence, Linux as an OS doesn’t even work as it’s been touted to work from every orifice.
Again, a software compilation with zero implied backward or forward compatibility. Time and again, this compilation has been proved to falter. Remember the fiascos related to XScreenSaver, Firefox/Thunderbird in Debian, and now OBS. There have been a lot more examples.
Even Torvalds himself lamented the horrible state of packaging in Linux distros over 10 years ago. Nothing really has changed.
Yeah, I know about stable userspace kernel syscalls. No one writes software using them. It’s akin to using Assembler when e.g. Python exists. For all intents and purposes we may simply forget that they exist. They are proxied by … unstable (in terms of API/ABI) glibc anyway.
Artem S. Tashkinov,
I don’t think that’s a fair statement, repos are a proven way of handling dependencies quite effectively (obviously for software in the repos). Not only did it work, but I’d say it worked exceptionally well, better even than windows.
However I don’t deny that we’re running into the limitations of central maintainers: 3rd party software is excluded and users are left to cope with it on their own, which is painful on any platform especially if the publisher doesn’t specifically support your OS of choice. In this respect I definitely agree that the linux repo model leaves a lot to be desired.
I’m going to be the pedatic guy who says Linux isn’t an OS. It should be obvious from the context but you’ve been pointing the finger at the wrong project entirely. It’s distros like ubuntu/redhat/etc that provide the full operating system and it falls on their shoulders to solve 3rd party installs. Arguably this has been the motivation behind snap/flatpak/appimage.
It’s all about dependencies. Even X11 applications from decades ago can run because it’s part of the base system that’s still well supported today. What breaks is all the libraries and dependencies that aren’t typically bundled with an application. It’s always going to be the same debate: what should be part of the base system and what should be bundled with the application.
Nothing is stopping you from running an old version of phython. Again it’s all about getting the dependencies right. I concede getting dependencies right can be hard, but the need to do it isn’t specific to linux, the same is true on windows and mac. Sometimes I still come across software with missing dependencies on windows. We ought to be able to agree that the solution necessarily involves balancing between what’s part of the standard base versus what’s bundled with the application.
For example The Gimp on windows used to require external GTK dependencies, which was a pain for users to install. However at some point they started bundling their own dependencies with the installer. Same with SDL/FLTK/etc. On windows it has become normal to bundle these libraries, on linux it is normal to use the distro’s version. Obviously there are pros and cons to both approaches.
GNU/Linux distros have never been about packaging the whole world. Their only objective is to make a working operating system. They put up the infrastructure for installing packages, include some libraries and software vendors provide the rest. The only issue with this is that most distros mix vendor and distro packages, with the corresponding PPA issues.
That’s not the point, the point is that you bundle everything your application needs to work. The only thing you can’t bundle is the kernel.
glibc has a stable ABI.
“The issue is not Fedora or Flatpak, Snap, or AppImage.”
We agree on that.
“Hidden behind the trees is the fact that Linux doesn’t exist as an OS, and even “universal” packaging methods have failed to achieve any sort of sanity.”
I mostly agree that Linux in not an OS. It makes more sense to think of the distro as the OS in some ways. However, both POSIX and the Linux kernel achieve a very high degree of compatibility between these.
Flatpak is not perfect but I do think it has basically solved the problem of packing for different distros. Flatpak runs on top of the kernel and bundles in all dependencies that the app needs otherwise. Distrobox does the same in a different way (my preference).
Can you explain how Flatpak has failed to “achieve sanity”?
The problem in this case is not that Flatpak failed to deliver on its promises. The issue is that Fedora is distributing a broken version of OBS even though OBS distributes a working version themselves. Flatpak delivers on its promise that the working version from OBS works as expected on Fedora. The problem is that users are being directed to the wrong software. This issue has nothing to do with technology and could happen on any platform.
“Let’s waste a couple more decades building incompatible software compilations that are on average supported for at most 24 months. It’s worked great so far, right?”
Some distros support their packages for up to 10 years. The support situation is vastly better on Linux than on either Windows or macOS. Windows supports the “OS” well but almost all the software you install on it will be from sources with completely different support Windows and good luck even finding it all. Each package in Flathub will have its own support lifecycle but at least it is all from one place. Again, superior to other ecosystems in the way that you are complaining about.
“Oh, wait, most people here will be dead by then. And many will continue to wait for the year of Linux on the desktop.”
The Linux Desktop may never happen in terms of mainstream adoption. That is because of too much choice and variability, not because of technology. Ask anybody that has to support regular users. Moving people to Linux and away from Windows (as an example) results in fewer support calls and random breakages. My own experience supporting a family member on macOS has been similar to Windows but I don’t have enough data.
“Not. Going. To. Happen. Ever.”
If you mean Desktop Linux then maybe. All the choice and variability that holds Linux back is what attracts me too it though. So, I am ok with it only having millions and millions of users and not 90% market share.
As far as the ability to distribute an application that will run well on a Linux desktop, I think Flakpak has largely delivered on that promise already. Although, as seen in this case, if you instead want to distribute your own version of that app that is broken in the same way everywhere, Flatpak let’s you do that too.
Yeah, not even wrong. Can’t really debate anything with someone that just wants to make up their own definitions of words.
I am sure as a child of one of the two enterprise linux server os fedora has simply higher standards and the manpower to maintain such flatpak copy. Who have no standard are wild maintained etc reliability? Their standards intheir own distinct fundament might also be more hardened etc. it’s logical to make their versions use more from their own system following certain expectations. Surely they can do things like this compared to all these one to five men distros that often differentiate them self’s by theme and wallpaper choice…an information that these packages have been modified and are not as provided by the original devs would be appropriate though.
What are the problems exactly? None of the links I saw describe any specific bug.
It’s interesting you post this today as just this morning I was wondering about this question, why does Fedora even bother maintaining its own Flatpak repository? Now with the explanation, it makes some more sense.
What we need as end users of desktop Linux IMHO is better ability to filter out unverified packages from Flathub, because the quality really does run the gamut, and simultaneously a renewed push to get all desktop software published on Flathub by the original authors, particularly closed/mixed source software, whose authors have a tendency to say “works on Ubuntu, good enough for us”.
Also, fix this issue:
https://github.com/flatpak/flatpak/issues/1258
It should be possible to use CLI apps with Flatpak just like those installed by the package manager, with aliases added to the shell namespace, without end users having to manually create/maintain the aliases to the binaries.
On a slightly related note: As a mostly-happy user of Fedora with KDE Plasma 6/Wayland, I’m cautiously optimistic about the in-development distro planned to be released in Fall, OpenSuse Leap 6. There was previously a lot of debate around whether to get rid of the Leap version of OpenSuse entirely, instead opting for “SlowRoll” which is essentially just snapshotted versions of the rolling-release OpenSuse Tumbleweed, but it seems like they finally decided to put in the work and make a proper community-targeted desktop distro based on the brand-new ALP platform, which was developed as the new base for SUSE Linux Enterprise Server (SLES) and therefore mostly aimed at servers.
https://news.opensuse.org/2024/01/15/clear-course-is-set-for-os-leap/
It’s got Plasma 6, Wayland, they finally switched from AppArmor to SELinux like almost everyone else, and maybe, just maybe, it could give Fedora a run for its money. If it supports my 2022 ASUS ZenBook with AMD APU and 4k touchscreen anywhere as well as Fedora does, I’d be up to switch, especially if it supports hibernation – Fedora doesn’t. Of course, this is all assuming that the software update mechanisms etc. also perform flawlessly, but at this point I’d assume that’s a given for these OSes. Sadly I still need to run Windows now and again simply because I need to use things like a proper WhatsApp app or play games with Skype screensharing, or count on hibernation working when I need to save battery (which is then partially counteracted by Windows being more resource intensive).
Sorry meant Leap 16, not 6.
BTW this is the reason I am not particularly fond of Desktop Linux: It’s the OS that had an App Store (repositories) before the other OSes had an App Store. Oh, and each distro has its own App Store (repositories). And yes, I know you can install stuff from outside the repos but we both know it’s not a good idea save for a couple of apps like Chrome.