After the release of Fedora Linux 35, I conducted a retrospective survey. I wanted to see how contributors felt about the release process and identify possible areas we can improve. There was no particular reason to start this with F35 except for that’s when I got around to doing it. So how did F35 go? Let’s look at the results from the 63 responses.
At Nest, I delivered a talk called “Exploring Our Bugs“. But a single snapshot isn’t very useful. Building on the work there, let’s make this a regular thing. With the recent Fedora Linux 33 end-of-life, I’ve added F33 bugs to the bug exploration notebook. Here’s a few of my key findings.
After a drop in bug reports in F32, F33 had about as many bug reports as F31. This is reflected in both bugs marked as duplicate and non-duplicate bugs.
Bug reports coming from abrt recovered to roughly the historical average after a surprisingly low F32.
We fixed roughly the same amount of F33 bugs as in the last few releases. But with the increase in overall bugs, that means we left more unfixed bugs this time around. The dramatic increase in bugs closed EOL reflects this.
The good news is that we are getting faster at fixing the bug reports that we do fix.
I re-downloaded the historical data to add some additional fields. This allowed me to take a look at a few areas we hadn’t examined previously.
The first area I wanted to look at is the number of bugs tagged as being security-related. Fedora Linux 33 had the highest count of security bugs, with over 1200. Looking at the graph, there’s a big jump between F26 and F27. This suggests a process change. I’ll have to check with Red Hat’s product security team to see if they have an explanation.
The good news is that we’re fixing more security bugs than we’re not. The bad news is that the proportion of security bugs going unfixed is increasing. To be more correct, more bug reports are not marked as fixed. Security fixes often come in upstream releases that aren’t specifically tied to a Bugzilla bug.
Like with other bug reports, we’re fixing security bugs fixed faster than in the past. 50% of security bugs are resolved within about two weeks.
I also wanted to look at how our QA processes are reflected in the bugs. During discussion of an F35 blocker candidate, Adam Williamson commented that it seemed like we were being looser in our interpretation of release criteria lately. In other words, bugs that would not have been blockers in the past are now. The numbers bear this out. While the number of both accepted and rejected blockers is down significantly from F19, there’s a general upward trend in accepted blockers from F30.
We have a big increase in accepted freeze exceptions recently. In fact, it looks exponential. Interestingly, the number of rejected freeze exceptions are roughly the same in that time.
Finally, I was curious to see if our use of the common bugs mechanism has changed over time. It has: we mark far fewer bugs compared to five+ years ago. I will be interested to see if the experiment that uses Ask Fedora to handle common issues changes the trends at all. We’ll have to wait until May 2023.
The graphs are pretty, but what do they mean? We have to be careful to draw too deep of conclusions. What’s in Bugzilla represents bug reports, not necessarily bugs. Some reports aren’t actual bugs and some bugs don’t have reports. And there’s a lot of “why” that we can’t pull from a summary analysis.
That said, it’s clear that we’re getting more bug reports than we can handle. Some of these should properly be filed upstream. How can we improve on the rest? We can’t do it all at once, but perhaps by working on some subset, we can make improvements. The one that jumps out to me is the security bugs. Can we bring more attention to those? I’ll spend the holiday break thinking about how to make them more visible so that they’re fixed or handled more quickly.
In the meantime, I’d love to hear your ideas, too. If you’d like to examine the data for yourself, everything is in the fedora-bug-data repo.
The kernel team is working on final integration for kernel 5.15. This version was released just recently and will arrive soon in Fedora. As a result, the Fedora kernel and QA teams have organized a test week from Sunday, November 14, 2021 through Sunday, November 21, 2021. Refer to the wiki page for links to the test images you’ll need to participate. Read below for details.
It’s time to start thinking about Test Days for Fedora Linux 36. A Test Day is an event aimed getting together interested users and developers to test a specific feature or area of the distribution. You can run a Test Day on just about anything for which it would be useful to do some fairly focused testing in ‘real time’ with a group of testers; it doesn’t have to be code. For instance, we often run Test Days for l10n/i18n topics. For more information on Test Days, see the wiki.
Hello everyone, welcome to the Fedora Linux 35 Final installation of Heroes of Fedora! In this post, we’ll look at the stats concerning the testing of Fedora Linux 35 Final. The purpose of Heroes of Fedora is to provide a summation of testing activity on each milestone release of Fedora. Without community support, Fedora would not exist, so thank you to all who contributed to this release! Without further ado, let’s get started!
Thursday, 2021-10-08 is the Fedora 35 Virtualization Test Day!
Why virt Test Day?
The test day will focus on testing Fedora or your favorite distro inside a bare metal implementation of Fedora running Boxes, KVM, VirtualBox, and whatever you have. The general features of installing the OS and working with it are outlined in the test cases which you will find on the results page.
Hello everyone, welcome to the Fedora Linux 35 Beta installation of Heroes of Fedora! In this post, we’ll look at the stats concerning the testing of Fedora Linux 35 Beta. The purpose of Heroes of Fedora is to provide a summation of testing activity on each milestone release of Fedora. Without community support, Fedora would not exist, so thank you to all who contributed to this release! Without further ado, let’s get started!
My internship of 3 months with Fedora has come to an end. From submitting the form 2 times and finally making it the 3’rd time, the journey has been quite challenging. My project “Improve Fedora QA dashboard” motive was to make the dashboard more impactful so that it will be simplified for the newcomers and they can easily understand and contribute without any complexity.