This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on (

Highlights of the week

Infrastructure & Release Engineering

Goal of this Initiative

Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release
(mirrors, mass branching, new namespaces etc.). The ARC (which is a
subset of the team) investigates possible initiatives that CPE might
take on.


Fedora Infra

  • Got all armv7 builders moved to a stable install, hopefully they will keep this until f36 (and 32bit arm) goes EOL
  • In the middle of a mass update/reboot cycle. Also updating all servers to latest bios and drac firmware and updating passwords. (outage later today! – Wed February 16th)
  • Got working ppc64le builder for FCOS. Had to reinstall a power9 with Fedora 35 instead of RHEL8 as hypervisor.
  • Cleaned up our dns repo (was up to 5GB). Everyone should check out a new copy if you work with it.
  • Bugzilla is forcing everyone to move to API tokens by end of month. All applications MUST move to that or risk their accounts being locked!
  • Fedimg is currently not working

CentOS Infra including CentOS CI

  • Cbs/koji sending notifications to fedora-messaging
  • Still hardware issues on main node in IAD2 (Work In Progress)
  • Fixed deployment (different branch)
  • Ansible-core now pushed to 8-stream so reverting for our own infra ansible hosts
  • Business as usual

Release Engineering

  • Branched (36) and rawhide (37) composes now on track and running fine.

Any other bussiness

  • The-new-hotness is being prepared for the Bugzilla API Authentication change

CentOS Stream

Goal of this Initiative

This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.


CentOS Duffy CI

Goal of this Initiative

Duffy is a system within CentOS CI Infra which allows tenants to provision and
access bare metal resources of multiple architectures for the purposes of
CI testing.
We need to add the ability to checkout VMs in CentOS CI in Duffy. We have
OpenNebula hypervisor available, and have started developing playbooks which
can be used to create VMs using the OpenNebula API, but due to the current state
of how Duffy is deployed, we are blocked with new dev work to add the
VM checkout functionality.


  • Finish provisioning and deprovisioning of nodes in background node pool tasks
  • Tie up loose ends (ongoing)
  • Add some lines to the Q4 community blog post

Image builder for Fedora IoT

Goal of this Initiative

Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra.


  • 2 tasks from the Image Builder team related to our koji. Detailed here:
  • currently looking at pungi and how the fedora IoT team orchestrates their builds
  • 3 blockers on Image Builders side currently:
    • koji service account issue above (in progress?)
    • auth and API changes on their end (completed)
    • they need to add multi-tenancy (in progress)
  • Some small contributions upstream to plugins


Goal of this Initiative

This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management and automate part of the release process.
Read ARC team findings in detail at:


  • Move Bodhi from python-openid to OIDC (ongoing)
  • Build beta / staging RPMs automatically in Koji (ongoing)
  • Dependency management (ongoing)
  • Fix the specfiles and RPMs since the split of packages (ongoing)


Goal of this initiative

Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL).

EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more.


  • EPEL9 up to 1907 source packages (increase of 132 from last week)
    • At current pace will surpass C9S in three weeks
    • At current pace will surpass EPEL8 by June
  • Fixing various fail-to-install bugs
  • centpkg added to epel9 testing so maintainers can more easily dogfood c9s
  • Presented State of EPEL to Fedora Council

Kindest regards,
CPE Team