FLOCK 2017 – Testing Testing Testing

I’ve attended Flock for the first time this year. I’ve didn’t know what to expect there. We’ve had prepared workshop about Meta-Test-Family to present it there.

Stef’s presentation: Ansible testing invocation

I’ve attend as many ansible presentations there as possible. First were lead by Stef Walter. It is my colleague also from Cockpit team development. There were some things that I knew, because I’m part of QA team. and some new things because I am not in direct touch with upstream ansible.

Known facts for me

  • Ansible allows us to schedule tests and provide reliable results. link
  • Ansible will contain standard test roles what allows us to create simpler tests, for example for me there is important to have role for beakerlib, because we use it regularly.

What’s new for me

  • There are part of metadata inside these ansible scripts (called tags) what allows us to specify what is proper environment to run commands.
  • There exist inventory, what helps you to generate yaml and prepare machines for you . It is very interesting and promising for me, especially machine preparation, it helps you to debug tests locally inside environment what will be prepared for you also inside some CI

I’ve also invited workshop for this project and we’ve discussed how to use ansible with MTF especially for container test subject. It is very important do do differences what is test subject is. This dictionary sometimes causes lots of confusion (artifact, test, subject, container, docker, image etc 🙂


MTF – meta test family workshop

We’ve had workshop about MTF. MTF began as project what should help you to write tests for modules (and every  final build artifacts) It allows you to write same tests for containers, rpm repositories (module compose) or whatever what it will be (ISO images, openshift)

We’ve started with short presentation about this project and we’ve expected to have there some tests as an output of this workshop. But there happened that attenders was more interested in talk :-), so we’ve changed it to discussion. We’ve heard lots of feedback anc ideas what we’ve should implement there and how to sell it to more people.

Used technologies

  • docker – for container testing
  • systemd-nspawn – for module testing, it is closer to docker, but it allows to use systemd inside
  • Avocado – Primary testing framework, although MTF is not directly dependent to write tests in avocado. It is your choice, it is simplest to use avocado, but in case you can use various frameworks around MTF – unittest, py.test, nosetest, behave

Basics ideas what are already implemented there and were discussed

  • Continuous integration for this project to prove that it is still working. It is connected with another topic and it is to support alternative Linux distribution.
  • Make possible to test each artifacts separately. We are able to use MTF to test just containers, or rpm repositories, and have there specific tests for them. Some artifact tests are too specific, that is not possible to generalize them to more artifacts
  • Previous point also led us to use it for testing common Fedora (no modular one) You can use it for writing tests for your package.
  • MTF helps you to write multi-host tests (container, rpm repository). It is side effect of using lower level of this library. You can have as many instances  of container/nspawn classes as you want.




I was perfect conference and I was proud to be part of Fedora community. I like atmosphere there and talk to many interesting people personally.

I’ll glad to see you all on next Flock 2018