Fedora Infrastructure Archives – Fedora Community Blog

Tag: Fedora Infrastructure (page 1 of 6)

Introducing MetaSource (or MDAPI 4)

September 3, 2025 / t0xic0der / Comments Off

We are excited to announce the general availability of the MetaSource (or MDAPI 4) in both the staging and production Fedora Infrastructure environments. The release includes an architectural rewrite of the MDAPI from Python to Go, making it a performant source of RPM repositories metadata as a REST service with 1:1 API compatibility. More details about the developments and acknowledgements are below.

Rewrite

The project faced a critical challenge due to its dependence on SQLite3-based RPM repositories metadata which were deprecated by the Fedora Linux 41 release. Rather than applying temporary workarounds on the existing codebase, we took this opportunity to fundamentally redesign the project. The improvements included optimized processing of XML-based RPM repositories metadata, swifter response towards HTTP REST operations, interactive documentation for ecosystem experience, stellar coverage across functional testcases among other things. The move from Python to Go programming language allowed us to take advantage of the performance benefits and resource efficiency – all while ensuring that the solution stays simple enough to maintain for a high-throughput HTTP service for a cloud-native first deployment.

Comparison

Sustained querying

Services	Sample Count	Total Duration	Per request Duration
MDAPI MDAPI v3.1.7	1000 requests	63 minutes, 3 seconds	3.7828 seconds
MetaSource MDAPI v4.0.0	1000 requests	42 minutes, 30 seconds	2.5505 seconds

Services	Average Memory	Minimum Memory	Maximum Memory
MDAPI MDAPI v3.1.7	157.09 MiB 160,861 KiB	99.16 MiB 101,544 KiB	204.34 MiB 209,248 KiB
MetaSource MDAPI v4.0.0	109.72 MiB 112,350 KiB	81.66 MiB 83,616 KiB	174.52 MiB 178,712 KiB

MetaSource (or MDAPI v4.0.0) performs roughly 33% faster than MDAPI v3.1.7 while using about 30% lesser memory than that on sustained querying operations. This means that MetaSource would be able to address approx 50% additional requests without furthering resource consumption. Please note that the results may vary depending on unknown variables like network bandwidth and querying nature.

Concurrent querying

Services	Sample Duration	Total Count	Per request Duration
MDAPI MDAPI v3.1.7	500 seconds	148 requests	7.3615 seconds
MetaSource MDAPI v4.0.0	500 seconds	310 requests	3.5530 seconds

Services	Average Memory	Minimum Memory	Maximum Memory
MDAPI MDAPI v3.1.7	217.41 MiB 222,625 KiB	136.73 MiB 140,008 KiB	289.93 MiB 296,888 KiB
MetaSource MDAPI v4.0.0	187.02 MiB 191,510 KiB	130.68 MiB 133,816 KiB	257.44 MiB 263,624 KiB

MetaSource (or MDAPI v4.0.0) performs roughly 52% faster than MDAPI v3.1.7 while using about 14% lesser memory than that on concurrent querying operations. This means that MetaSource would be able to address approx 110% additional requests without furthering resource consumption. Please note that the results may vary depending on unknown variables like network bandwidth and querying nature.

Appeal

This project would not have been possible without the help of Akashdeep Dhar, Kevin Fenz i, David Kirwan, Michal Konecny, James Antill, Steve Milner, Shounak Dey and countless others who have either contributed to Fedora Infrastructure projects ecosystem or the createrepo_c project. We ask the readers to contribute to the project by developing, maintaining, testing or documenting the project.

Introducing User Interface for Webhook To Fedora Messaging

September 1, 2025 / t0xic0der / Comments Off

As a part of our move from Fedmsg to Fedora Messaging in the last year, we announced the general availability of the Webhook To Fedora Messaging service. While the project was developed to replace the (now decommissioned) GitHub2Fedmsg service, we did not have a user interface for managing webhook binds with Fedora Messaging. The migrating users of the GitHub2Fedmsg service had to hence request for the creation of webhook binds via an issue tracker and the incoming users had to utilize the Swagger UI to create the webhook binds by themselves – which worked just fine but was definitely not ideal.

Given that since then, the project has evolved significantly into an ecosystem of Webhook-based communications to the Fedora Messaging. The support for Forgejo and GitLab repositories was added to support the Fedora Project’s and CentOS Project’s transitions to these platforms respectively. With active discussions around the frontend requests and design architectures, we finally came around to making an interactive user interface available for the Webhook To Fedora Messaging users. Please feel free to give it a try on the production environment and consider helping with the maintenance efforts of the project!

Less than 2 weeks until the Datacenter move

June 17, 2025 / kevin / 2 Comments

It’s less than 2 weeks until the switch of fedoraproject to our new datacenter, so I thought I would provide a reminder and status update.

Currently we are still on track to switch to the new datacenter the week of June 30th. As mentioned in previous posts:

End users hopefully will not be affected (mirrorlists, docs, etc should all be up and working all the time)
Contributors should expect for applications and services to be down or not fully working on Monday the 30th and Tuesday the 1st. Contributors are advised to hold their work until later in the week and not report problems for those days as we work to migrate things.
Starting Wednesday the 2nd things should be up in the new datacenter and we will start fixing issues that are reported as we can do so.

We ask for your patience in the next few weeks as we setup to do a smooth transfer of resources.

Another update on the Fedoraproject Datacenter Move

May 29, 2025 / kevin / 2 Comments

Here’s another update on the upcoming fedoraproject Datacenter move.

Summary: there have been some delays, the current target switch week
to the new Datacenter is now the week of 2025-06-30.
( formerly 2025-05-16 ).

2025 Fedora Datacenter move update

April 24, 2025 / kevin / Comments Off

In January this year, we let the community know that we were going to be moving datacenters ( see https://communityblog.fedoraproject.org/fedora-datacenter-move-later-this-year-2025-version/ ). Some things have changed a little, so here’s some updates on the progress of this move and some more detailed plans.

Back in that initial post, we were targeting May to switch to the new datacenter, but due to various issues beyond our control, the current plan is now to switch to the new datacenter the week of June 16th. This is the week after DevConf.cz, and two weeks after Flock.

We already have new hardware in the new datacenter and should be getting access to it in the coming weeks. Then, we will build out a copy of our existing infrastructure there. Anything that can be setup and prepped for the move will be. After that, in May we hope to move a few things that are somewhat isolated from the rest of our infrastructure (OpenQA, Fedora CoreOS, and possibly more).

Then, the week of June 16th we will migrate applications and data to the new datacenter.

Fedora Project users should not notice much during this change. Mirrorlists, mirrors, docs, and other user facing applications should continue working as always. Updates pushes may be delayed a few days while the switch happens. Our goal is to keep any end user impact to a minimum.

For Fedora Contributors, Monday and Tuesday we plan to “move” the bulk of applications and services. Contributors should avoiding doing much on those days as services may be moving around or syncing in various ways. Starting Wednesday, we will make sure everything is switched and fix problems or issues as they are found. Thursday and Friday will continue stabilization work.

It’s worth noting that some things are not included in this move:

pagure.io is not moving now (It will likely move later in the year, details on that later!), so it should be available the entire time of the switch.
discussion.fedoraproject.org is not affected.
mirrorlists and all static content like docs, fedoraproject.org download site, etc.

The following week the newest of the old hardware in our old datacenter will be powered off and shipped to the new datacenter. This hardware will add additional capacity to some systems like openqa and builders.

Finally after that we will rebuild our staging setup. Some of that work may be done in advance, but we will be focusing on the production setup.

This move will get us moved to new (faster!) hardware, increase capacity in various of our systems, add ipv6 connectivity, move to power10, and leave us room to grow and add more as needed later. Thanks in advance for your patience with this move.

End of OpenID authentication in Fedora Account System

March 20, 2025 / zlopez / 4 Comments

On the latest Fedora Infrastructure weekly meeting we decided on a date of OpenID authentication sunset. The date is 20th May 2025.

Why the change?

The OpenID is being replaced by OpenIDConnect (OIDC) in most of the modern web and most of the Fedora infrastructure is already using OIDC as the default authentication method. OIDC offers us better security by handling both authentication and authorization. It also allows us to have more control over services that are using Fedora Account System (FAS) for authentication.

What will change for you?

With the End Of Life of OpenID we will switch to OIDC for everything and no longer support authentication with OpenID. If your web or service is already using OIDC for authentication nothing will change for you. If you are still using OpenID open a ticket on Fedora Infrastructure issue tracker and we will help you with migration to OIDC. For users using FAS as authentication option there should be no change at all.

What will happen now?

We will be reaching to services we identified as using OpenID directly, but as we don’t have control over OpenID authentication we can’t identify everyone.

If you are interested in following this work feel free to watch this ticket.

End of Life of github2fedmsg

January 20, 2025 / zlopez / 3 Comments

As was already announced in Announcing Webhook To Fedora Messaging Fedora Infrastructure team is working on replacement of github2fedmsg service. As the replacement is already deployed and users already had a few months to migrate to webhook2fedmsg we decided to say goodbye to github2fedmsg. The date that github2fedmsg will be decommissioned in fedora infra is 3rd February.

CPE Update Q3 2024

November 15, 2024 / zlopez / Comments Off

This is a summary of the work done on initiatives by the CPE Team. Every quarter, the CPE team works together with CentOS Project and Fedora Project community leaders and representatives to choose projects that will be being worked upon in that quarter. The CPE team is then split into multiple smaller sub-teams that will work on the chosen initiatives + day-to-day work that needs to be done. Some of the sub-teams are dedicated to the continuous efforts in the team whilst some are created only for the initiative purposes.

This update is made from infographics and detailed updates. If you want to just see what’s new, check the infographics. If you want more details, continue reading.

A photograph of a children's playground. There are multiple stairs and slides for someone to climb and slide down. Text is overlaid on the photograph, reading "Communishift is now available."

Photo by Power Lai on Unsplash. Modified by Justin W. Flory. CC BY-SA 4.0.

Communishift is now available

March 7, 2024 / zlopez / Comments Off

Long time after first announcement of this Fedora hosted OpenShift for community we are finally able to say that it’s available. It took us (Fedora Infrastructure Team) a long time to get over all the legal issues of hosting this in Fedora Infrastructure, but we were finally able to solve all those issues.

Communishift is made mainly for hosting containerized projects related to Fedora, but not ready for infrastructure deployment yet. For example you have a neat idea for service that will help Fedora and need to test it something. Or you want to try a testing deployment for something you already have at hand, but it was never deployed in OpenShift. If you don’t meet any of those, feel free to request the communishift project anyway and we will look at the request individually.

A photograph of a dimly-lit server room. Several cabinets are visible with servers and computers inside. Several Ethernet cables are run between servers and switches inside the networking cabinet. Text is overlaid on the image reading, "Fedora Infrastructure & Release Engineering."

Photo by Taylor Vick on Unsplash. Modified by Justin W. Flory. CC BY-SA 4.0.

2023 Year in Review: Infra & Releng

February 29, 2024 / zlopez / Comments Off

This is a summary of the work done by Fedora Infrastructure & Release Engineering teams as of 2023. As these teams are working closely together, we will summarize the work done in one blog post by both teams.