The Modularity Team was able to hold a session at Flock 2019 to gather feedback and discuss a few issues. The session was well attended and there was a bunch of great discussion.

We started out by spending 15 minutes, measured on a stopwatch(!) gathering the most pressing issues. The audience and the panel came up with quite a few ideas.

list of issues

We then asked people to vote on what they felt were the most important items that we should focus on. As you can see in the picture, which is a little cryptic, “offline local builds,” “OBS/COPR Compat,” “upgrade path,” and “default streams in BR”. As we elaborate on the outcomes of each topic we will also explain the somewhat cryptic shorthand we used for each topic.

Offline Local Builds

First up, “offline local builds.” Much of the discussion centered around what does that term actually mean. In other words, what are the user stories we are attempting to satisfy? While we didn’t actually write them as user stories, we did establish the following requirements. We need to build modules which:

  • without access to Red Hat Internal Infrastructure
  • without access to the internet (some preparation steps may be required)
  • for arches different from the one doing the building
  • for target OS versions different from the running one

We had a few useful outcomes. First, much of this already exists in the Module Build Service (MBS) which is available in Fedora as the module-build-service rpm. A user can pass the build_module_locally parameter and --offline flag to get the main part of this functionality since v2.20.

However, the “optional” features – like a different architecture or OS release – are not implemented, limiting the usefulness of the tool. As a result, our next step(s) is to submit these enhancement requests to the development team and ensure they are prioritized.

Second, the meeting revealed a general lack of awareness of these features in MBS. As a team, we need to do a better job of documenting and promoting this functionality to the community.

We did a nice job of capturing the outcome on a flipchart.

offline local builds notes

OBS & COPR compatibility

The next topic we took on was “OBS/COPR compat” or, compatibility with alternative build systems, in particular Open Build Service (OBS) and COmmunity PRojects (COPR). Similar to the Offline Local Builds discussion, the core problem here is lack of documentation. Fedora does not document anywhere the exact actions that MBS is taking to convert a modulemd file from the input version (stored in Fedora’s dist-git) and the final version after the builds have been completed. Without this documentation, it’s extremely difficult for a third-party to replicate the MBS behavior. The Modularity and MBS teams committed to producing this documentation.

OBS & COPR compatibility notes

Upgrade Path when Streams are Default

The next topic of discussion focused on what the user expects to happen when they upgrade from one Fedora release to another and how can we ensure that Fedora’s behavior matches that expectation.

The debate covered a couple different variants of the problem. First, what does “default stream” mean in terms of upgrade? The conclusion was that if the user did not explicitly select anything about the stream of a module they are on then, upgrades (including stream switches), should happen automatically per the “recommendation of the release”. In other words, the module owner should be defining what happens when a stream default changes from release to release.

However, if the user explicitly selected the stream they are on, even if that stream happened to be the same as the default one, stream switches should only occur by explicit user action. If the stream is not available in the release the user is trying to upgrade to, the user must make an explicit choice about what to do (switch streams or delay upgrading).

The dnf tool does not currently support automatic stream switching and must be enhanced to support this function. Design documentation will be provided by the Modularity Team to describe the functionality that dnf should implement.

Upgrades when default stream selected notes

Making Module Streams Available in the Buildroot

The last topic we covered was how and when RPMs built in module streams should be available to other RPMs for building. In other words, as RPMs move in to modules, we must make the RPMs in those modules available to be built upon by non-modular RPMs. The most notable example of this problem has been the Java stack in the last few releases of Fedora.

What we would like to do is have any RPMs provided by a default module stream present in the buildroot for all non-modular RPMs. It is worth mentioning that we don’t need to worry about modules because modules explicitly declare any dependencies rather than relying on expected RPMs/binaries in the buildroot.

However, with our present tools certain aspects of this are “hard”. First, we don’t have a way to mash multiple composes together, the module stream(s) build(s) and the non-modular buildroot. Next, we need Bodhi integration so that as RPMs are updated they are added to future incantations of the buildroot. Finally, we need to way to make RPMs that are filtered out of the released module streams still available to the buildroot.

No new requirements came up during the discussion. As a result, the present design that the Modularity and Factory 2.0 teams have developed seems sufficient to meet requirements. However, the lack of publicity of the design seems to indicate that further review, while not gating, would be very much appreciated.

notes regarding requirements for default streams in the buildroot


Overall, I felt that this session was very productive. The format we used, in particular, seemed especially successful. In fact, I think we should try to do one of these at each of the major conferences (DevConf.CZ, Flock, DevConf.US?) using the same format.

I would also like to point out the great attendance and participation we got for this session. I think I can speak for the whole team saying we really appreciate the engagement and collaboration on working out the kinks in Fedora Modularity.

Please comment here or file issues with the Modularity Team if you have any comments or questions.