This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. We provide you both infographic and text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in depth details look below the infographic.

Week: 04 Aug – 08 Aug 2025

Infrastructure & Release Engineering

The purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.).
List of planned/in-progress issues

Fedora Infra

• In progress:
  • [FMTS] TLS certificate for distrobuildsync-cs service is about to expire in 30 days
  • pagure.io spammer: joe53
  • Move ibiblio machines to new vlan
  • Update/Remove Old Wiki Information
  • The cloud-noc-os01 machine seems to hang on SSH
  • Planned Outage – update/reboots – 2025-08-05 21:00 UTC
  • reconfigure power10 machines
  • Increase vCPUs on Libravatar.org
  • Forgejo: Owner access to @jflory7 for @CommOps
  • Move copr_hypervisor group from iptables to nftables
  • tmpwatch removed from ansible
  • Link in staging distgit instance leads to prod auth
  • Fix logrotate on kojipkgs01/02
  • Broken link for STG IPA CA certificate, needed for staging CentOS Koji cert
  • [CommOps] Open Data Hub on Communishift
  • retire easyfix
  • Deploy Element Server Suite operator in staging
  • Pagure returns error 500 trying to open a PR on https://src.fedoraproject.org/rpms/python-setuptools-gettext
  • maubot-meetings bot multi line paste is cut
  • Move OpenShift apps from deploymentconfig to deployment
  • Support allocation dedicated hosts for Testing Farm
  • EPEL minor version archive repos in MirrorManager
  • Add fedora-l10n pagure group as an admin to the fedora-l10n-docs namespace projects
  • vmhost-x86-copr01.rdu-cc.fedoraproject.org DOWN
  • Add yselkowitz to list to notify when ELN builds fail
  • Recurring issue that Bugzilla stops sending messages to fedora-messaging
  • Cleaning script for communishift
  • Move from iptables to firewalld
  • Port apps to OIDC
  • Help me move my discourse bots to production?
  • Migration of registry.fedoraproject.org to quay.io
• Done:
  • Failing to authenticate against Fedora (oidc and kerberos)
  • src.fedoraproject.org is down
  • Flatpak remote errors
  • Beberapa WhatsApp Tokopedia
  • Reinstantiate my ML membership
  • EPERM in flatpak-indexer
  • Unable to file pull requests in Pagure due to 403 errors
  • STG : unable to get TLS .csr signed by ipa through fasjson.stg.fedoraproject.org
  • flatpak builds non functional since datacenter move
  • netapp storage for Forgejo stg/prod
  • [FMTS] TLS certificate for git-hooks service is about to expire in 30 days
  • [FMTS] TLS certificate for batcave service is about to expire in 30 days
  • [FMTS] TLS certificate for fmc service is about to expire in 30 days

CentOS Infra including CentOS CI

• In progress:
  • [SN#1648] Figure out a way to help Duffy cleanup failed provisions
  • [SN#1654] git.centos.org migration to GitLab
  • [SN#1580] [spike] : investigating options/alternatives for the upcoming DC move
  • Upgrade our monitoring stack from Zabbix 6.0.x LTS to 7.0.x LTS
  • Release Improvements
• Done:
  • [SN#760] publish rpm-specs-latest tarball for CentOS Stream
  • Add dns A record for mirrors.stg.centos.org
  • new ansible role for lookaside cache
  • Prepare new host for lookaside cache server

Release Engineering

• In progress:
  • Allow resigning f43 tagged packages with f44 key
  • Process approved non-responsive maintainer ticket for pwalter
  • f42 Games iso missing from torrents
  • Create games-sig group for dist-git
  • Create rawhide+1+1 (F45) keys
  • Please update koji builders to use koji-image-builder 8.
  • readjust the ‘heavy-builder’ channel builders
  • pybind11 PR cannot be merged
  • Improve the mass-tag script so it notify if something is broken and verify permissions
  • Packages containg a “+” do not have a f42 branch
  • Transfer/orphan packages owned by rdieter
  • `–no-git-branch` option to fedpkg request-branch and creating the branch manually doesn’t work as expected
  • Turn EPEL minor branching scripts into playbooks
  • Mass retirement of packages with uninitialized rawhide branch
  • 300+ F42FTBFS bugzillas block the F41FTBFS tracker
  • Packages that have not been rebuilt in a while or ever
  • Send compose reports to a to-be-created separate ML
  • Could we have fedoraproject-updates-archive.fedoraproject.org for Rawhide?
  • Investigate and untag packages that failed gating but were merged in via mass rebuild
  • Update pungi filters
  • Implement checks on package retirements
  • Untag containers-common-0.57.1-6.fc40
  • Provide stable names for images
  • Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
  • When orphaning packages, keep the original owner as co-maintainer
  • Create an ansible playbook to do the mass-branching
  • RFE: Integration of Anitya to Packager Workflow
  • Fix tokens for ftbfs_weekly_reminder. script
  • Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
• Done:
  • Fedora flatpak remote is broken (returning 404)
  • Stalled EPEL package: ykpers
  • update tag2distrepo.keys for coreos-pool to remove F41 key
  • Filing FTBFS Bugs after F43 Mass rebuild
  • Please block ivtv-firmware in koji
  • Unretire notification-daemon
  • Unretire mozjs115
  • pyparted PR #4 is stuck
  • Fedora 43 Mass Rebuild Tracker
  • When processing stalled EPEL package requests, make the requester EPEL bugzilla override
  • Remove spam-o-matic or turn it into purely an email wrapper for repoclosure

If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.