The Fedora infrastructure is working on replacing our current message bus fedmsg by a new library fedora-messaging based on AMQP. This is an update on the work currently in progress.
After deploying a RabbitMQ cluster and bridges to duplicate messages from fedmsg to the fedora-messaging and from fedora-messaging to fedmsg. We are now starting the migration of application to fedora-messaging. After looking at the overall change we have identified a set of critical applications which are needed for the rawhide gating initiative :
- Pagure (src.fp.o)
- CentOS CI
The reliability of the messages sent or received by these applications is critical for this initiative, that’s why moving to fedora-messaging and AMQP is a prerequisite.
To make significant progress in a short time we setup a small team (pingou, abompard and cverna) and organized our work using a GitHub board with the intention to do as much as possible in a week. After a team kickoff call on Monday morning we decided not to focus on more than 2 cards at the time meaning that we would work in pair on one task while the other person could make progress on the other card.
We also tried to focus on what we could achieved in a week and decided to leave the CentOS CI and OpenQA work for later since these 2 applications require more work upfront.
What we achieved
By the end of the week, we had src.fp.o and Koji successfully sending messages using fedora-messaging in staging. While deploying these applications to staging we have also fixed the bridge that replicates the messages from fedora-messaging to fedmsg.
After submitting a Pull Request to migrate resultsDB to fedora-messaging and getting this PR merged, we deployed the application in staging only to find out that resultsDB virtual machine could not connect to the AMQP broker. We finally opened a infrastructure ticket to get the networking modification needed.
WaiverDB, Greenwave and Bodhi are still in progress, they all have a PR either opened or merged and we are still coordinating with the applications maintainer to finish this effort and deploy these applications in staging.
We still have some work to complete this effort but we have made really good progress in only a week. Once the dependencies holding us from completing the card in progress are cleared, we should spend another week focusing on the deployment of these applications.
If you would like to help us in this effort, or would like to help with the migration of another application to fedora-messaging you can reach out on the #fedora-apps IRC channel and or the Fedora Infrastructure mailing list.
Photo by Lewis Ngugi on Unsplash