We are excited to announce the general availability of the MetaSource (or MDAPI 4) in both the staging and production Fedora Infrastructure environments. The release includes an architectural rewrite of the MDAPI from Python to Go, making it a performant source of RPM repositories metadata as a REST service with 1:1 API compatibility. More details about the developments and acknowledgements are below.

Rewrite

The project faced a critical challenge due to its dependence on SQLite3-based RPM repositories metadata which were deprecated by the Fedora Linux 41 release. Rather than applying temporary workarounds on the existing codebase, we took this opportunity to fundamentally redesign the project. The improvements included optimized processing of XML-based RPM repositories metadata, swifter response towards HTTP REST operations, interactive documentation for ecosystem experience, stellar coverage across functional testcases among other things. The move from Python to Go programming language allowed us to take advantage of the performance benefits and resource efficiency – all while ensuring that the solution stays simple enough to maintain for a high-throughput HTTP service for a cloud-native first deployment.

Comparison

Sustained querying

MetaSource (or MDAPI v4.0.0) performs roughly 33% faster than MDAPI v3.1.7 while using about 30% lesser memory than that on sustained querying operations. This means that MetaSource would be able to address approx 50% additional requests without furthering resource consumption. Please note that the results may vary depending on unknown variables like network bandwidth and querying nature.

Concurrent querying

MetaSource (or MDAPI v4.0.0) performs roughly 52% faster than MDAPI v3.1.7 while using about 14% lesser memory than that on concurrent querying operations. This means that MetaSource would be able to address approx 110% additional requests without furthering resource consumption. Please note that the results may vary depending on unknown variables like network bandwidth and querying nature.

Appeal

This project would not have been possible without the help of Akashdeep Dhar, Kevin Fenzi, David Kirwan, Michal Konecny, James Antill, Steve Milner, Shounak Dey and countless others who have either contributed to Fedora Infrastructure projects ecosystem or the createrepo_c project. We ask the readers to contribute to the project by developing, maintaining, testing or documenting the project.

As a part of our move from Fedmsg to Fedora Messaging in the last year, we announced the general availability of the Webhook To Fedora Messaging service. While the project was developed to replace the (now decommissioned) GitHub2Fedmsg service, we did not have a user interface for managing webhook binds with Fedora Messaging. The migrating users of the GitHub2Fedmsg service had to hence request for the creation of webhook binds via an issue tracker and the incoming users had to utilize the Swagger UI to create the webhook binds by themselves – which worked just fine but was definitely not ideal.

Given that since then, the project has evolved significantly into an ecosystem of Webhook-based communications to the Fedora Messaging. The support for Forgejo and GitLab repositories was added to support the Fedora Project’s and CentOS Project’s transitions to these platforms respectively. With active discussions around the frontend requests and design architectures, we finally came around to making an interactive user interface available for the Webhook To Fedora Messaging users. Please feel free to give it a try on the production environment and consider helping with the maintenance efforts of the project!