The Federated Message Bus, or Fedmsg, is used within the Fedora Infrastructure to easily connect services using ZeroMQ publishers and subscribers. This library is now deprecated in favour of Fedora Messaging.

Fedora Messaging provides a framework for declaring message schemas and a set of APIs to publish and consume messages to and from AMQP brokers.

In the project Fedora Happiness Packets, Fedmsg was set up to send messages to the Bus so that Fedora Badges could be awarded to the sender when they send a Happiness Packet, i.e an email worth of appreciation! My piece in this jigsaw was to migrate from Fedmsg to Fedora Messaging in this containerized project.

Breaking Down the Problem

After going through the documentation for Fedora Messaging and understanding its use case in Fedora Happiness Packets, I realized that a publisher needs to be set up in the project. To accomplish the same, I broke down the problem into a set of steps like this:

  1. Install the library Fedora-Messaging in the project 
  2. Setup configurations 
  3. Create a Message Schema 
  4. Make a Docker container for RabbitMQ to listen to the messages.
  5. Set up publishers in the project’s codebase

Before starting, I followed the tutorial given in Fedora Messaging Documentation to get a better grasp of what I was doing. While doing the same, I ran into a couple of minor errors mainly related to configurations of config.toml which are summarized here with their solutions.

Step 1: Install the library

Fedora Messaging needs to be added as a dependency in the project as described here. Fedora Happiness Packets is a containerized application where all the dependencies are installed from a requirements file. I simply added fedora-messaging to the above mentioned file to ensure that the library and all its dependencies are installed in the container.

Apart from the library, the message schema is also needed to be installed in the project to use while publishing messages. I included the name of the python package of the message schema in the requirements file.

Step 2: Setup Configurations

A configuration file config.toml is needed for Fedora Messaging as described here. I started with the configuration file config.toml.example given in the docs and modified it to set up configurations needed only for publishers.

The amqp_url also needs to be configured to connect to the RabbitMQ message broker, that is setup in its docker-container. I configured this while setting up RabbitMQ in the project as described below.

The last step I did in this section was to set the environment variable FEDORA_MESSAGING_CONF to the path of the configuration file created as detailed above. I set this up in the Dockerfile.

Step 3: Create a Message Schema

This step involves a lot of things so again I divided the task into bite sized pieces as follows:

  • Change the basic configurations
  • Create a Message class based on the project’s use case
  • Write tests for the Message Schema
  • Build the distribution and upload the sdist and wheel to PyPI.
  • Submit an RPM package for it to Fedora and EPEL.

Change the basic configurations

While ensuring the crux of this integration worked seamlessly, I mistakenly overlooked some of the small details that need to be edited from the sample message schema. Here is a tiny list of the little things that need to be changed in setup.py:

  • The python namespace and the name of the package. Both of these should be same.
  • Change the version of the package. I used Semantic Versioning for this.
  • Update the URL to the project’s repository.
  • Update the name of message class created in the entry points.
  • Change the README to better describe the package’s use case.

Create a message class 

This is at the crux of the format in which messages will be sent to the Bus as described here. In Fedora Happiness Packets, messages are sent to the bus to ensure the sender and receiver, receive Fedora Badges. Thus the message had to contain their usernames (if permitted by the sender) to award badges as well as the ID of the Happiness Packet send.

I made the message schema best suited to the requirements of Fedora Happiness Packets as discussed here in detail.

Writing tests for the message class

Its always best to have every line of code tested. The same goes here. I created multiple tests for the message class I made in the section above. The tests range from basic message format validation to specific project use cases. For inspiration, refer the test cases given in  the sample message schema.

Upload the distribution to PyPI and submit RPM package to Fedora and EPEL

The message schema needs to be packaged as the consumers of the messages install that package and use it to validate the message and extract various bits of information. I built the distributions and uploaded the sdist and wheel to PyPI for the message schema of Fedora Happiness Packets.

Step 4: Make a Docker container for RabbitMQ

To verify if messages in our Dockerized project are being sent or not in the local development environment, I set up a separate container for RabbitMQ to listen to the messages. This proved to be a challenge due to the added overhead Docker presented.

I was able to create a new container for RabbitMQ by adding this to docker-compose.yml:

rabbitmq:
    image: rabbitmq:alpine
    ports:
      - "5672:5672"
    environment:
      RABBITMQ_DEFAULT_PASS: pass
      RABBITMQ_DEFAULT_USER: user
      RABBITMQ_DEFAULT_VHOST: vhost

I made sure to link the other related containers with this container. 

As mentioned previously, I configured amqp_url in config.toml so that messages could be sent to this message broker like this:

amqp_url = "amqp://user:pass@rabbitMq:5672/vhost"

Step 5: Set up publishers in the project’s code base

Finally, I configured how messages were published on the bus in Fedora Happiness Packet as described here. For this mainly three things need to be kept in mind:

  1. Topic the messages are published to. For this I used the topic that was used in Fedmsg. 
  2. Body of the message. The actual message sent to the bus formatted using the message schema created above and populating it with data using JSON formatting. 
  3. Exception Handling. I made sure to handle the exceptions mentioned here to ensure the end user gets notified/sees an error message when one of the exceptions are raised.

And that’s it! This is how I was able to integrate Fedora Messaging in Fedora Happiness Packets. A big shout out to Justin W. Flory, Jeremy Cline and Aurélien Bompard for helping me out with this at every instance. Thank you so much!

Hope you find something helpful and relevant for your use case in this blog post!