This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. It also contains updates for the CPE (Community Platform Engineering) Team as the CPE initiatives are in most cases tied to I&R work.

We provide you both an infographic and a text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in-depth details look at the infographic.

Week: 20 May – 24 May 2024

Infrastructure & Release Engineering

The purpose of this team is to take care of day-to-day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces, etc.).
List of planned/in-progress issues

Fedora Infra

• In progress:
  • Open ticket with Dell to replace hard disk
  • rhel7 EOL
  • Migration of registry.fedoraproject.org to quay.io
  • add monitoring for dnf countme
  • Replace Nagios with Zabbix in Fedora Infrastructure
  • notifications do not notify
  • DNF countme minor changes post migration
  • PDC retirement
  • Move from iptables to firewalld
  • fedoraplanet.org: Upgrade Venus to Pluto
  • Move mailman to newer release of Fedora or CentOS
  • Setup RISC-V builder(s) VM in Fedora Infrastructure
  • Update compose hosts to get latest pungi release (4.6.0)
  • Deploy new sign hardware/software
  • Commits don’t end up on the scm-commits list
  • Create monitoring tool for rabbitmq certificates
  • Request for new FAS group: miracle-sig
• Done:
  • New discourse spammer
  • Spam on Fedora Users list
  • Issue with Fedora 40 i686 koji builder’s repository
  • Planned Outage – updates / reboots – 2024-05-15 21:00UTC
  • Inactive packagers to be removed for the F40 cycle
  • epel-packagers-sig sponsors
  • Election Candidate Name is not being displayed in Elections App
  • Staging wiki is not available
  • another chatgpt spammer
  • 400 – Bad Request on fmn.apps.ocp.fedoraproject.org
  • Another discourse spammer to suspend
  • EPEL9 repodata created with newer createrepo_c breaking compat for el8
  • Zabbix: putting missing configuration under source control.
  • Zabbix: template for handling external hosts outside iad2.

CentOS Infra including CentOS CI

• In progress:
  • c8s/c7 EOL infrastructure planning
  • Review all our ansible infra roles pointing to mirror.centos.org
  • Set up compose testing for c10s
  • koji set-build-volume throws errors for some builds 
  • Prepare CBS c10s koji tags
  • Stream 10 mass signing (infra part)
  • Ipsilon DB clean-up
• Done:
  • What’s the process to update a CentOS CI Tenant / Jenkins?
  • Remove legacy mirror
  • CBS: Repositories not synchronized

Release Engineering

• In progress:
  • Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
  • Silverblue aarch64 installer image compose always fails on F38 and Rawhide
  • i686 builders need to use 32-bit inode numbers
  • Fixes for release-candidate.sh in pungi-fedora
  • When orphaning packages, keep the original owner as co-maintainer
  • Use an automated script to control checksums of compose images
  • Create an ansible playbook to do the mass-branching
  • Package retirements are broken in rawhide
  • Implement checks on package retirements
  • Support for self-service package un-retirement
  • F38 End of Life
  • Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
• Done:
  • Error 500 when creating repository
  • Unretire unison
  • Unretire rpms/php-psr-http-client
  • Cleanup post F40 Release
  • golang-github-prometheus-node-exporter was not properly retired
  • Publish “latest” tag to quay.io/fedora/fedora
  • python-dask-expr not enabled for f41 after repo creation
  • Lenovo: F40 official iso for P1 G7 preload
  • Repo request failed with HTTP 500 error, still owned by releng-bot
  • New repo failed with 500 – ticket 62362
  • Repo request failed with HTTP 500 error, still owned by releng-bot
  • New repo requests are not adding components to koji
  • Orphaning pgRouting
  • Builds should not restart if they hit OOM
  • spam-o-matic (depcheck) is wildly, uselessly wrong

CPE Initiatives

EPEL

Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS, Scientific Linux (SL) and Oracle Linux (OL).

Updates

• Updated cacti in EPEL 7, EPEL 8, EPEL 9, and Fedora 39 to address multiple CVEs, including a critical RCE
• Worked with RelEng to resolve issues with centos-10-sync script on batcave

If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.