Many important packages and software are developed for Fedora every day. One of the most important parts of software development is quality assurance, or testing. For important software collections in Fedora, there are sometimes concentrated testing efforts for pulling large groups of people in who might not always help test. Organizing a Fedora Test Day is a great way to help expose your project and bring more testers to trialing a new update before it goes live.
Most of the time, you will be able to test software updates without help. But for larger software or packages crucial to Fedora, having more eyes and hands to poke around is useful and helpful. This post explains and walks you through the process for organizing your own Fedora Test Day and what work goes into it.
Steps to organize a Test Day
- Decide the change or target feature you want to test for.
- This might involve talking with other developers and packagers to make sure any compatibility issues with other software are included in test cases.
- Create a ticket for organizing your Fedora Test Day. Tickets can be created on the Fedora QA Trac after authenticating against a FAS ID. For filing a ticket, make sure to include the following details.
- Test Day name with the planned date
- Assigning / copying relevant people to the ticket
- Mailing the firstname.lastname@example.org mailing list is helpful for managing and tracking the progress of the ticket for community members
- Do some research and check if any similar test cases were already run. If they were, you can re-use a previous wiki page of test cases. If not, you will need to write new Test Day and test case pages.
- After setting up the wiki, begin considering and working on the meta-app data page. The Test Day meta page helps populate the wiki page. Hence, the Test Day application page is where testers post their results against the test cases they execute.
These are the necessary steps for getting your Fedora Test Day rolling. The results for your test cases appear in the Test Day application. The test day app is a place where all the testers will post their results.
Setting up a Test Day
Before working on putting together your Test Day, you will want to do some homework on the change set for the Fedora release you are targeting for. Are there any changes that might conflict with your software updates? Are there other factors that could be a point of concern? Make sure you have a general sense of what’s new and how that may affect your software. For example, take a look at the Fedora 24 change set.
Your next step would be to share your plans with the rest of the Fedora Quality Assurance team about your plans for a Test Day. To do this, you will need to file a ticket in the Fedora QA Trac. Make sure you share any relevant information with the team like the changes you are trying to target and anything useful to explaining your Test Day.
If you were unable to find any previous test cases on the wiki that you could reuse, you will need to write a new Test Day page. Fortunately, there is a template you can use to help simplify this process. If you are starting from scratch, make sure you use the template to meet the required criteria for organizing your Test Day. In addition to the Test Day page, you will also need to set up a page for test cases. This page will contain the specific instructions for what testing is needed and how to do it. It also details which test cases should yield what results. You can find examples of past test case pages on the wiki.
Next up, the meta page is required for automated result reporting. The Test Day application needs this to work as expected. To help you with writing one of your own, you can find a link to past page for <test day>. Once completed, you will need to submit your Test Day information to the Test Day application in the Fedora Infrastructure.
Running the Test Day
Congratulations! By this point, your Test Day will be live. You can find its own Test Day page publicly once approved. It will look like the screenshot below. You should also consider announcing your Test Day on both the email@example.com and firstname.lastname@example.org mailing lists to help bring some exposure to your event. You can also share the details in other communities, such as the Google+ group.