This installment of the Heroes of Fedora series will focus on work done on Fedora 25 Beta release.
Hello fellow testers, and welcome back to the Heroes of Fedora – F25 Beta edition! In this post we’ll look at who-did-what during the push to F25 – Beta! Before we begin however, something has come to our attention that we need to clear up! Since Fedora 24 Alpha, we have no longer used the TC (Test Candidate) system and have switched to using nightly-test-validation to document testing on branched releases. When this change occurred early in the F24 – Alpha release cycle, the program we have used to generate test statistics was not aware of the change and some of the data it reported back for F24 – Alpha, Beta, Final and F25 – Alpha was erroneous. This has been noted and fixed, so the results from now on should be accurate. We’re sorry for this error and will make sure that this does not affect the stats into the future.
With that out of the way, let’s move on to our Heroes of Fedora F25 – Beta!
Updates Testing
Compared to Fedora 24 Beta, there were 16 less testers and 146 less comments. for the F25 Beta release. See the Heroes below!
Test period: Fedora 25 Beta (2016-08-30 – 2016-10-11)
Testers: 145
Comments1: 1252
1 If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.
Validation Testing
Compared to F24 Beta, there were 9 less tester, 222 less reports, and 3 more unique referenced bugs.
Test period: Fedora 25 Beta (2016-08-30 – 2016-10-11)
Testers: 16
Reports: 404
Unique referenced bugs: 25
| Name | Reports submitted | Referenced bugs1 |
|---|---|---|
| pwhalen | 185 | 1367910 1371338 1374810 1377113 1377640 1379800 1380861 1381309 (8) |
| juliuxpigface | 68 | 1367910 1375721 1382820 1382839 (4) |
| kparal | 49 | |
| pschindl | 15 | 1381996 1382001 1382274 (3) |
| lbrabec | 13 | |
| satellit | 13 | 1363915 1379766 (2) |
| dexterh | 10 | |
| adamwill | 10 | 1379459 1379865 1381996 (3) |
| coremodule | 10 | 1369786 1382808 (2) |
| me2 | 9 | 1374670 1375190 (2) |
| lnie | 5 | |
| puffi | 5 | |
| cmurf | 4 | 1369786 1377825 (2) |
| jagodyn | 4 | |
| jsedlak | 2 | |
| tenk | 2 | 1383686 1383688 (2) |
1 This is a list of bug reports referenced in test results. The bug itself may not be created by the same person.
Bug Reports
Compared to Fedora 24 Beta, there were 94 less reporters and 105 less bug reports this time around.
Test period: Fedora 25 Beta (2016-08-30 – 2016-10-11)
Reporters: 201
New reports: 654
| Name | Reports submitted1 | Excess reports2 | Accepted blockers3 |
|---|---|---|---|
| Mikhail | 57 | 0 (0%) | 1 |
| lnie | 38 | 1 (2%) | 0 |
| Kamil Páral | 37 | 2 (5%) | 0 |
| Joachim Frieben | 35 | 1 (2%) | 1 |
| Christian Stadelmann | 27 | 1 (3%) | 0 |
| Yonatan | 19 | 0 (0%) | 0 |
| Adam Williamson | 18 | 0 (0%) | 2 |
| pavel raur | 12 | 0 (0%) | 2 |
| Karel Volný | 12 | 0 (0%) | 0 |
| Ricardo Ramos | 11 | 0 (0%) | 0 |
| Chris Murphy | 10 | 0 (0%) | 0 |
| Paul Whalen | 9 | 1 (11%) | 1 |
| Giulio ‘juliuxpigface’ | 9 | 0 (0%) | 0 |
| Petr Schindler | 9 | 0 (0%) | 0 |
| RJ | 9 | 0 (0%) | 0 |
| Heiko Adams | 8 | 0 (0%) | 0 |
| Alexander Mayorov | 7 | 0 (0%) | 0 |
| Mustafa Muhammad | 7 | 0 (0%) | 0 |
| Dan Horák | 6 | 0 (0%) | 0 |
| Hans de Goede | 6 | 0 (0%) | 0 |
| Jonathan Briggs | 6 | 1 (16%) | 0 |
| Menanteau Guy | 6 | 4 (66%) | 0 |
| Mikko Tiihonen | 5 | 0 (0%) | 0 |
| Volker Sobek | 5 | 0 (0%) | 0 |
| Alex | 4 | 0 (0%) | 0 |
| F.J. Simon | 4 | 0 (0%) | 0 |
| Fabio Valentini | 4 | 0 (0%) | 0 |
| Mike FABIAN | 4 | 0 (0%) | 0 |
| Mike Simms | 4 | 1 (25%) | 0 |
| Nivag | 4 | 0 (0%) | 0 |
| Peter Robinson | 4 | 0 (0%) | 0 |
| Prasad J Pandit | 4 | 0 (0%) | 0 |
| tenk at fedoraproject.org | 4 | 1 (25%) | 0 |
| yucef sourani | 4 | 0 (0%) | 0 |
| Alexander Kurtakov | 3 | 1 (33%) | 0 |
| Andrey Motoshkov | 3 | 0 (0%) | 0 |
| Bhushan Barve | 3 | 0 (0%) | 0 |
| bob | 3 | 0 (0%) | 0 |
| Chris Smart | 3 | 0 (0%) | 0 |
| Christophe Fergeau | 3 | 0 (0%) | 0 |
| David Hill | 3 | 1 (33%) | 0 |
| Gerardo Rosales | 3 | 0 (0%) | 0 |
| gil cattaneo | 3 | 1 (33%) | 0 |
| Giovanni Campagna | 3 | 0 (0%) | 0 |
| Jacob Godyn | 3 | 0 (0%) | 0 |
| Jan Stodola | 3 | 0 (0%) | 0 |
| Jason Shepherd | 3 | 1 (33%) | 0 |
| Jeremy Cline | 3 | 1 (33%) | 0 |
| Jiri Eischmann | 3 | 0 (0%) | 0 |
| Karl-Heinz Hoetzel | 3 | 0 (0%) | 0 |
| Nemanja | 3 | 0 (0%) | 0 |
| Pavel Alexeev | 3 | 0 (0%) | 0 |
| Petr Pisar | 3 | 0 (0%) | 0 |
| Ralf Corsepius | 3 | 0 (0%) | 0 |
| Stephen Gallagher | 3 | 1 (33%) | 0 |
| Viorel Tabara | 3 | 0 (0%) | 0 |
| Dennis Gilmore | 2 | 0 (0%) | 1 |
| bancfc at openmailbox.org | 2 | 1 (50%) | 0 |
| christian.kirbach at googlemail.com | 2 | 0 (0%) | 0 |
| Colin Walters | 2 | 0 (0%) | 0 |
| cornel panceac | 2 | 0 (0%) | 0 |
| Couret Charles-Antoine | 2 | 0 (0%) | 0 |
| Dietmar Kling | 2 | 0 (0%) | 0 |
| Dmitri A. Sergatskov | 2 | 0 (0%) | 0 |
| eugen.fischer.android at googlemail.com | 2 | 0 (0%) | 0 |
| Frederico Henrique Gonçalves Lima | 2 | 0 (0%) | 0 |
| Geoffrey Marr | 2 | 0 (0%) | 0 |
| Jens Petersen | 2 | 0 (0%) | 0 |
| José Matos | 2 | 0 (0%) | 0 |
| lennart_reuther at web.de | 2 | 0 (0%) | 0 |
| Leslie Satenstein | 2 | 0 (0%) | 0 |
| Lukas Slebodnik | 2 | 1 (50%) | 0 |
| Marius Vollmer | 2 | 0 (0%) | 0 |
| mastaiza | 2 | 0 (0%) | 0 |
| Miro Hrončok | 2 | 0 (0%) | 0 |
| Onuralp SEZER | 2 | 0 (0%) | 0 |
| Paul W. Frields | 2 | 0 (0%) | 0 |
| psmis at protonmail.com | 2 | 0 (0%) | 0 |
| pzeppegno at gmail.com | 2 | 0 (0%) | 0 |
| Radek Vykydal | 2 | 0 (0%) | 0 |
| rh at treblig.org | 2 | 0 (0%) | 0 |
| Sergio Monteiro Basto | 2 | 1 (50%) | 0 |
| Sinny Kumari | 2 | 2 (100%) | 0 |
| stan | 2 | 0 (0%) | 0 |
| Stef Walter | 2 | 0 (0%) | 0 |
| Timo Klerx | 2 | 0 (0%) | 0 |
| Timothy M. Butterworth | 2 | 0 (0%) | 0 |
| Toshio Ernie Kuratomi | 2 | 0 (0%) | 0 |
| Vedran Miletić | 2 | 0 (0%) | 0 |
| William Moreno | 2 | 0 (0%) | 0 |
| Zbigniew Jędrzejewski-Szmek | 2 | 0 (0%) | 0 |
| …and also 110 other reporters who created less than 2 reports each, but 110 reports combined! | |||
1 The total number of new reports (including “excess reports”). Reopened reports or reports with a changed version are not included, because it was not technically easy to retrieve those. This is one of the reasons why you shouldn’t take the numbers too seriously, but just as interesting and fun data.
2 Excess reports are those that were closed as NOTABUG, WONTFIX, WORKSFORME, CANTFIX or INSUFFICIENT_DATA. Excess reports are not necessarily a bad thing, but they make for interesting statistics. Close manual inspection is required to separate valuable excess reports from those which are less valuable.
3 This only includes reports that were created by that particular user and accepted as blockers afterwards. The user might have proposed other people’s reports as blockers, but this is not reflected in this number.
Conclusion
Let the numbers speak for themselves, Fedora has an active community who is involved in multiple aspects of testing! Fedora would not have the quality it does without the community it has, so thank you for your testing efforts and participation; it does not go unnoticed! See you next time around for Heroes of Fedora 25 – Final!
November 3, 2016 — 20:57
Just to provide a bit more info on the accuracy issue…
Prior to Fedora 24, we only had nightly events before the first Alpha TC (‘test compose’, not ‘test candidate’, btw). All validation events after Alpha TC1 were for either a TC or an RC. So the stats tool could generate numbers for ‘all Alpha TC/RC’, ‘all Beta TC/RC’, ‘all Final TC/RC’, and ‘all nightly’, and that was fine – we could print the numbers for all the nightlies, then the numbers for all the Alpha events, then all the Beta events, then all the Final events, and all that worked.
The really important change in the F24 cycle is we now have nightly events throughout the cycle, not just up until the first non-nightly candidate. We still have approximate equivalents of ‘TCs’ and ‘RCs’ – the ‘candidate composes’ – but we make fewer of them, and rely on the nightly events for most of the testing. However, I didn’t think to update the stats program for this. So when Geoff ran the stats script and asked for ‘all the Alpha events’ or ‘all the Beta events’, it would only get numbers from the Alpha or Beta ‘candidate compose’ events, and completely ignore all the nightly events that happened in the appropriate time frames.
So we had to tweak the script a bit so you can now ask for all the events in a specific date range, and to get the ‘Alpha’ numbers, we ask for all the events in the timeframe up to Alpha release, to get the ‘Beta’ numbers we ask for all the events between Alpha release and Beta release, and so on. That way the nightly events go into the right bucket and no events are missed.