Here’s another update on the upcoming fedoraproject Datacenter move.
Summary: there have been some delays, the current target switch week
to the new Datacenter is now the week of 2025-06-30.
( formerly 2025-05-16 ).
The plans we mentioned last month are all still in our plan, just moved out two weeks.
Why the delay? Well, there were some delays in getting networking
setup in the new datacenter, but thats now been overcome and we are
back on track, just with a delay.
Here’s a rundown of the current plan:
- We now have access to all the new hardware, it’s firmware has been
updated and configured. - We have a small number of servers installed and this week we
are installing OS on more servers as well as building out vm’s for
various services. - Next week is flock, so we will probibly not make too much progress,
but we might do some more installs/configuration if time permits. - The week after flock we hope to get openshift clusters all setup
and configured. - The week after that we will start moving some applications that
aren’t closely tied to the old datacenter. If they don’t have storage
or databases, they are good candidates to move. - The next week will be any other applications we can move
- The week before the switch will be getting things ready for that
(making sure data is synced, plans are reviewed, etc) - Finally the switch week (week of june 30th):
Fedora Project users should not notice much during this change.
Mirrorlists, mirrors, docs, and other user facing applications
should continue working as always. Updates pushes may be delayed
a few days while the switch happens.
Our goal is to keep any end user impact to a minimum. - For Fedora Contributors, Monday and Tuesday we plan to “move” the bulk of applications and services. Contributors should avoiding doing much on those days as services may be moving around or syncing in various ways. Starting Wednesday, we will make sure everything is switched and fix problems or issues as they are found. Thursday and Friday will continue stabilization work.
- The week after the switch, some newer hardware in our old datacenter
will be shipped down to the new one. This hardware will be added
to increase capacity (more builders, more openqa workers, etc).
This move should get us in a nicer place with faster/newer/better hardware.
Are we looking into having more than one datacenter? Having a single datacenter and no geo-redundancy seems to screw us in terms of performance and availability.
Given that a bunch of these things run on OpenShift and the rest run on VMs that can leverage HA technology, it feels like it would behoove us to not take advantage of that.
Yes, we already have multiple datacenters, it’s just that currently IAD2 is our primary/biggest one and that will move to RDU3.
Many things that can be easily HA’ed already are. For example mirrorlist’s and docs and a bunch of other things are served from our proxy network all around the globe.
Other things like koji are very difficult to share accross datacenters due various reasons, mostly storage locality.
I’d love to explore making things more HA after the datacenter move if we can get cycles to do so (aside all the forgejo deployment and usual urgent requests to rework big things).