There was a discussion on the Fedora devel list late last year about moving to a more rolling release model, and I have seen people continue to suggest this in various places since then, so I thought I would write up my thoughts on rolling releases and how they can/could apply to Fedora.
Note that I am speaking only for myself here, not my employer, FESCo or anyone else.
First, lets talk terminology. What is a “rolling release”? In reference to Linux distributions, it’s where you provide a continual stream of updates for your distro and all work is done in one branch (there’s no seperate point releases, there’s only the distro). (See: http://en.wikipedia.org/wiki/Rolling_release for much more discussion and terms, but I think that the above boils it down some).
Now a bit of Fedora background: Fedora has a devel/master branch, called (for historical reasons) rawhide. Is rawhide a rolling release? I would argue that it is. In addition currently Fedora has at various times 2-3 more branches/releases being maintained. Updates to these are guided by the Updates policy: http://fedoraproject.org/wiki/Updates_Policy. These releases are on a 6month (roughly) cycle. A new branch/release is branched off rawhide, then stablized and released. Old releases are retired 1 month after 2 releases happen. These are not at all rolling releases by the above definition.
So, lets take some updates examples and see how things work now in Fedora vs what they would look like in a rolling distro:
You have foo-1.0-1 installed and foo-1.1-1 comes out. It’s minor bug fixes, possibly docs and translation updates.
- On supported stable Fedora releases: You would possibly get this update if the maintainer felt the bug fixes were worth bothering with.
- On a rolling release/rawhide: You would get this update and not really notice it unless you were hit by a bug in this package or needed the docs or was heavily involved with it.
You have foo-1.0-1 installed and foo-2.0-1 comes out. It’s a major new version. The UI changes along with lots of other things.
- On supported stable Fedora releases: You shouldn’t get this update if at all possible, the exceptions might be if upstream stopped supporting foo-1.0 at all and this fixed security updates that couldn’t otherwise be fixed any other way. If you are a happy foo-1.0 user, I would argue that you don’t want foo-2.0 to suddenly appear and cause you to have to learn how to use it. In a timed release model like Fedora currently uses, you would get foo-2.0 at the next Fedora release, when you have time and desire to sit down and learn all the new changes. You can schedule it on your timeline, not when it appears in the repos.
- On a rolling release/rawhide: You get this update quickly, and must deal with learning the new UI right then. If you have important work in foo-1.0 and are on a deadline, you don’t have much choice when you have to learn 2.0.
You have libfoo-1.0-1 and libfoo-2.0-1 comes out. This is a major new library version, the ABI/API changes. All applications using this must rebuild and probibly get patched to work with the new version.
- On a supported stable Fedora release, you won’t see this update. You will get it at the next Fedora release. This gives you time to port your applications to the new ABI/API, and you can sit down and fix things when you upgrade to the next Fedora release.
- On a rolling release/rawhide: You get this update pretty quickly. The majority of things that need rebuilding are done, but stragglers are sometimes left broken by the upgrade. If you use one of them you are out of luck, or must spend time getting them working as soon as the bundle of packages land in the main repo. There’s some ways around this of course, like compat packages for things, but in general if your application is very slow moving there may be a lag in support.
You have rpm-N and new rpm-N+1 is released with a new package format/compression/payload. (Or you move all / dirs to /usr, or…)
- On a supported stable Fedora release, you never see this update. It’s only on the next Fedora release.
- On rawhide/rolling distro: You announce this in advance and try and get people to make sure they upgrade to the new version, you add docs about the error the old version prints when it can’t deal with the new version. You have a flag day to cut over and anyone who missed the memo has to go figure it out before they can get more updates.
(I could go on with the examples, but I could do that all day. Feel free to suggest other scenerios in comments 😉
As an aside there’s some notable exceptions to the Fedora updates policy worth mentioning:
- The Linux kernel. Currently Fedora keeps pretty close to the most recent upstream stable kernel version for _ALL_ Fedora stable releases. This occasionally causes problems for folks as new bugs make a machine that was working fine not boot, or have issues. There are a number of reasons for this: First, backporting security and bugfixes to old kernel versions is a vast amount of work. Fedora has a crack ninja team of kernel maintainers, but they are a small team. Backporting new hardware enablement would also be very difficult, and new hardware is always appearing. Not allowing new kernels on old releases would result in needing to upgrade to support new hardware, which isn’t very nice. Additionally, kernel is a special package in Fedora, in that it’s one of the few you can install multiples of. So, even in the event of some new bug, you can boot the older working kernel and debug the new problem.
- KDE. KDE upstream unfortunately has releases that are not at all aligned with Fedora. This is of course their choice, but it makes things hard for us. Additionally, older major versions aren’t supported, so, when Fedora N ships KDE 4.x at release time and 4.x+1 comes out in a month, we really have no choice but to update to that as again backporting manpower isn’t there.
So, do I hate rolling releases and want nothing to do with them? No. I think rolling releases are quite valuable, and a good fit for a small group of folks. You may like/want/use rolling releases if:
- You are a early adopter (You WANT foo-2.0 as soon as it’s announced)
- Love troubleshooting issues (You don’t mind when things break and are happy patching something to be fixed up for a new version, or reporting bugs on a new version of something)
- Are an integrator/software developer. (You want the latest libraries and toolchains to make sure your project builds and works there and takes advantage of the latest cool features)
- You are a distro integrator and enjoy making all the parts of things fit together as soon as your upstreams have released them.
However, if you are anyone else, you probibly don’t want to be using a rolling release. With a timed release, YOU can choose when you have time (+ or – 7 months in Fedoras case) to upgrade to the newest collection of software, relearn new applications and ui’s, rebuild/fix local code to new libraries, etc. With a rolling release, you are at the mercy of the upstream projects and your distro as to when you have to accept and adjust to a change.
So, if rawhide is a rolling release and some folks want Fedora to do one, why don’t they just use rawhide? I think there’s a number of reasons for that, many around rawhide being too raw for day to day use. However, I have been working on a number of ideas to improve day to day usefulness of rawhide, as well as using it myself day to day. Look for those ideas soon. Possibly we can address some of those concerns and get more folks using rawhide day to day and have the best of both rolling releases and timed stable releases.
For users who are not intimately familiar with the current state of play of every piece of code that’s on their machine — everyone — the problem with rolling releases is that the bits and pieces come at you, willy nilly. It’s a firehose.
The problem with timed releases is that they exist on someone else’s schedule.
What I would want from a “rolling” release is, first, a strong identity as a distribution. I.e., something that sets it aside from all the others, something that makes me want to use it and stay with it. Does that mean I want every single piece of new code that’s releases upstream? No, of course not. I don’t need every piece of new code.
What I want, though, is someone to exercise some management and judgement, someone who says, our users need “this” but they don’t need “that”. I.e., this makes our product better without changing it in wholesale fashion and it doesn’t break stuff. On the other hand, this other piece of code changes our product in ways we think would make our users unhappy, so we won’t release it.
> The problem with timed releases is that they exist on someone else’s schedule.
Rolling releases are much worse for this. 😉 You get a constant stream of things “when they are ready”. With timed releases you at least know the majority of the changes will be focused at the point releases and you can schedule when you want to do that upgrade on your schedule.
>…What I want, though, is someone to exercise some management and judgement, …
Yeah, but the only one who can 100% know your needs is you. 😉
I’ve been using Fedora for many years, and Red Hat Linux a little before that. I, for one, am happy with the current release model, and would like to see it stay the same. Improving Rawhide to make it more like a “rolling release” sounds great!
I think that Rawhide is rather a bleeding-edge package repo than a rolling release. It all comes from the upgradepath requirement that Rawhide must not contain older packages than Fedora Stable.
Imagine this situation:
You have a very important update for package foo for Fedora Stable. It’s very important that Fedora Stable users receive it as fast as possible. But the same update breaks in Rawhide, rendering foo application completely dysfunctional. What are your choices now?
1. Make Fedora Stable users’ life miserable and wait until Rawhide problems are resolved. That might take days or weeks. Only then push to Fedora Stable.
2. Push the updated foo into Rawhide and Fedora Stable. That makes Rawhide users’ life miserable (breaks foo completely), but helps tremendously Fedora Stable users.
3. Push the update to Fedora Stable, but not to Rawhide. This is a) against the policy b) causes problems for dist-upgrade c) we plan to disallow this by enforcing aforementioned upgradepath policy.
As you see, all three options are bad. We have to decide, either Rawhide is just a bleeding-edge package repo, it receives no QA and it is not intended to be used to run a system from it. Then the upgradepath policy makes sense. Or it is a rolling release, it receives QA (we check whether software works and we don’t push broken updates there), and it is expected to work as a running system. Then the upgradepath policy can’t be there and also we can’t blindly expect that all Fedora Stable releases can be upgraded to Rawhide any time.
I’m not sure I agree that the difference between a rolling release and a devel repo is “formal qa”. I think a number of rolling releases have no formal QA, or all QA is done by the users/consumers.
There are always corner cases. What if your foo update worked for rawhide, did not work/build in Fedora 18, but worked fine in F17/F16?
If rawhide is not intended to be used, what good is it? Just a way to see something builds?
Well I’m not fully sure what Rawhide is intended to be, that’s what my confusion comes from. If it really should be a rolling release (suitable for people to run it and use it), shouldn’t we enable Bodhi for it as well? It would make sense to try to not break it often. Package maintainers should be encouraged to run it and test their package updates on it. But our current guidelines only require them to _build_ the package in Rawhide, not run it. Even for new package submissions.
I’m in the process of getting my first package into Fedora and becoming a package maintainer. I’ve read our wiki documentation lately and I don’t remember clear guidelines and recommendations when it comes to Rawhide. When I asked on #fedora-devel about new package submissions, I was told that “building the package in Rawhide is enough, no one tries to run it anyway”.
That’s why I got the impression that Rawhide is just a package repo useful for solving build problems, nothing more.
If we want Rawhide to be something different, it would be great to have it explained in our new-maintainer wiki guidelines and documentation (and have the important guidelines in multiple documents so that people notice it). For example the update policy could state that broken (in functionality) package updates should not be pushed even to Rawhide. The new package submission guideline could require maintainers to run Rawhide and test the software, same as for stable releases.
But maybe I just understand “rolling release” term differently than others. I imagine a system that undergoes at least some minimal QA (like Bodhi) and that doesn’t receive just about any update, even if it is completely broken. As I say, i don’t really find enough documentation about what we expect it to be.
I completely agree that we should be more clear on what we intend rawhide to be.
I think relegating it to a dumping ground for builds means we have lost a great and vauable resource.
The problem is one of balance. If we say “rawhide is fine, everyone can use it!” we get problems when things break. If we say “Never use rawhide, it eats $whatever” we get 0 testing from it and it’s mostly useless to us. The key would be to have a middle ground and have the right kind of people using it. Not easy.
I’ll note there’s some general rules at:
http://fedoraproject.org/wiki/Updates_Policy#Rawhide_.2F_devel_.2F_master
for maintainers.
I think other packages need to be updated more often too, besides the kernel, the browsers for example. Not necessarily as soon as they appear, but faster than a release cycle.
I will be honest, I think that this happens in the case of the GA releases. I have seen that for a browser like, say, firefox… the delay between Mozilla and Fedora is about 14 days maximum. That said, there have been times where that delay has saved Fedora from a buggy release.
That said, firefox might be one of the few exceptions to this. I am not so sure of it myself.
Firefox is another exception sadly.
It releases upstream much faster than Fedora’s 6 month cycle, and older releases aren’t supported anymore. They are moving to a movel though where they do have a ESL (extended support release) version. Keep in mind that the versioning of firefox has changed. In the past a version 16 to version 17 change might well have been a 16 to 16.1.
Maybe it’s just me but I want something in between your two examples. I’ll call it the Fedora Stable Rolling Release. I want a a stable OS that can still get updates after 13 months. I don’t want foo to go from 1.0 to 2.0, but I do want foo to go from 1.0 to 1.1.
So in X.Y.Z version terms, a stable rolling release should get Y and Z updates, but not X (unless the user approves).
For example, GNOME:
Lets say I grabbed todays snapshot of the Fedora Stable Rolling Release desktop. Presume it comes with GNOME 3.6. Over the years this desktop updates to GNOME 3.8, 3.10, 3.12, etc. Then GNOME 4.0 comes out. At which time it says “GNOME 4 is out! Upgrading this package result in data loss, yadda yadda”. I choose no, because data loss sounds bad. This prevents me from getting all apps that require GNOME 4. But, meanwhile a libfoo 1.1 update comes out. It’s unrelated to GNOME, and it’s just a new copy of the stable package, I want that.
Comments are closed.