Test fixes and new features with ubports-qa

A lot of developers are working on fixes and improvements in Ubuntu Touch. But as a user, you can help too. The ubports-qa command makes this quite easy to do.

Find something to test

Developers regularly ask on the UBports forum to test something. For a recent example, have a look at Lionel Duboeuf’s call for testing for MMS users. In that post, Lionel explains what the feature is, for which Ubuntu Touch version, which commands you need to install to run the new code and how you can test it.

Another source of new things to test is the Telegram group UBports QA and testing. You’ll find developers there who ask to quickly test something they’re working on or to verify whether you have an error on a specific Ubuntu Touch version or a specific device.

Check the test requirements

Verify that your device meets the requirements that the developer lists. First, make sure that the change applies to your device. Some changes are specific to one device or a family of devices. Others are general enough that you can test them on any device. Some are meant to be tested on a virtual Focal-based device in the Platform Development Kit.

For instance, generally you need to run the Development or Release candidate channel. If you’re not, change to this channel in your update settings, and make sure to update to the latest available version in this channel before you proceed.

Install the packages to test

After you’re sure your device meets the requirements, install the new version of the package to test. This will generally be done with some ubports-qa commands. For instance, in the example of the MMS test on the forums, the instructions ask you to run the following commands one by one in the Terminal app:

sudo ubports-qa install PR_nuntium_8
sudo ubports-qa install telepathy-ofono 20
sudo ubports-qa install history-service 35
sudo ubports-qa install telephony-service 20
sudo ubports-qa install messaging-app 260

What do these commands do? With ubports-qa install you install a Personal Package Archive (PPA) or a pull request of a GitHub repository. If there’s just one argument after install, as in the first command with PR_nuntium_8, this adds a PPA and upgrades all packages in it. If there are two arguments after install, as in the second command with telepathy-ofono 20, this adds a specific pull request of an application on GitHub as a PPA and upgrades all packages. For the telepathy-ofono 20 example, this installs the packages from pull request 20 of telepathy-ofono.

In this case, you also need to reboot for the changes to take effect. The need for this depends on the package you install.

You can find more information about ubports-qa in the project’s GitHub repository ubports-qa-scripts. Note that the ubports-qa command is also available in the Platform Development Kit.

Test

Now that your (physical or virtual) device is running the right package versions, it’s time to test. Read the developer’s instructions carefully. In the MMS example, Lionel explains a test plan.

Generally you’ll check the feature in normal circumstances, but also in some special cases. For instance, you’ll change some settings and you should also enter wrong settings to investigate the behaviour in these cases.

Report back to the developer

After you have tested the change, please report your findings to the developer, even if it all just works perfectly. For the developer, knowing that something works is as important as knowing that it doesn’t work.

If it doesn’t work as expected, make sure to report as much details as possible to help the developer in finding the root cause. You should list the steps you took and their results. If possible, provide relevant log files (you can find these in the app Logviewer), with events before the test deleted. If you’re not sure which log files are relevant, ask the developer.

Repeat

Testing an operating system like Ubuntu Touch is a huge undertaking, but ubports-qa makes this quite manageable. Don’t forget to remove the PPA you installed for the test with sudo ubports-qa remove and then the argument you provided to the install command. Now you’re ready for your next test.

If you’re interested in helping out more with quality assurance and testing, read the documentation about helping the UBports QA team.