We want to make sure that every Ubuntu Touch release we ship is better than the previous one. That’s why every release goes through a QA (Quality Assurance) process. And you can help with it.
If you’re interested in helping test our releases, join the UBports QA and testing group on Telegram. This is the fastest way to get to know what you can test.
We also publish a call for testing on this blog before every OTA release. Such a call for testing explains what’s new in the release and how you can test it. For an example, have a look at the Ubuntu Touch OTA-19 Call for Testing. We also publish a companion forum post where we can discuss any questions you may have about anything that comes up during your testing.
Install the release candidate
To be able to test a release candidate of a new OTA release, you have to install it on your Ubuntu Touch device. If you’re already on the Release candidate channel, just open System Settings -> Updates. Ubuntu Touch will check for an available update, and if it finds a new release candidate for the next OTA release, it will download it. Install this after the download is finished. This will reboot your phone into the new release candidate.
If you’re using the default Stable channel for updates, you have to switch your device to the Release candidate channel to be able to test the release candidate. First update all your apps and the system itself in System Settings -> Updates. Make sure there are no pending system updates before switching channels, otherwise no update will be installable anymore. Then click on the settings icon at the top right, choose Channels and then select Release candidate as the channel to get updates from. Return to the Updates screen and install the available update.
Getting an OTA release done
For every OTA release we prepare a GitHub project in the UBports organization. For instance, for OTA-20 (which is currently still in development) this looks like this:
This project board lists the issues we’d like your feedback on. Each issue is listed in one of the following columns:
To do: These issues are known but don’t have a fix or a plan yet, or maybe it’s not known yet where the bug resides exactly.
In progress: These issues are not fixed yet or have a fix with a severe negative side effect.
QA: These issues don’t have complete test results yet.
Done: These issues have been confirmed as fixed.
At the end of the QA process, the goal is to have every issue moved to the Done column, like this (with OTA-19 as an example):
How can you help us test a release candidate?
If you want to help test an OTA release, the QA column in the GitHub project of the release is the one to watch. There are three types of cards in this column:
Issue: Click on the issue. This opens a panel on the right with more information. Then click on Go to issue for full details at the bottom. This will open the full issue page. Follow the instructions there to test this issue.
Pull request: Click on the pull request. This opens a panel on the right with more information. Then click on Go to pull request for full details at the bottom. This will open the full pull request page. This page often shows more developer-oriented comments, but many times you can also follow the instructions to test whether the issue is solved.
Note: This doesn’t have a panel with more information, but often directly links to a merge request (which is GitLab’s name for a pull request) or an issue on a GitLab repository of UBports.
Follow the test instructions
The instructions to test an issue, pull request or merge request will often list:
Steps to reproduce: What you have to do to test whether the issue is solved.
Expected behavior: What you see if the issue is solved.
Actual behavior: What you see if the issue isn’t solved.
If the instructions aren’t clear, just add your questions in a comment. If there are no instructions (for instance for a pull request or merge request with a low-level fix), ask the author how you can test it.
So try the instructions on your device, and then comment on the specific issue, pull request or merge request. We want to know:
Which device are you using?
Which image version are you using? (You can find this in System Settings -> About -> OS)
Does the issue appear fixed in your testing?
Did you notice any side effects of the change as it’s listed in the issue, pull request or merge request?
Did you notice something else that isn’t right?
We’re interested in the result of your test whatever it is. If you can’t reproduce the issue, the fix worked and if enough people report this, we can move the issue to the Done column. If you can still reproduce the issue, the fix obviously didn’t work and we need to research this further.
Do you want to become our test manager?
For testing every person counts, and you can make the difference by testing even one issue. However, testing is a collaborative effort, and this needs some coordination and managing. At the moment this doesn’t happen in a structured way for Ubuntu Touch.
Do you have experience with test management processes? Do you want to coordinate QA people, remind them about tasks, help them go through the process, follow up whether everything happens on time and keep things in order? If you want to become our test manager, join the UBports QA and testing group on Telegram and talk to Florian Leeber (@Flohack).