The New Fedora Documentation Website, The Rest Of The Story

Today, a lot of hard work and effort from a multi-year process pays off. Our new docs site is live at https://docs.fedoraproject.org/. Thanks to Adam Šamalík for a converting everything to the Antora publication engine, and to many members of the docs team for significant work in converting from DocBook to AsciiDoc format.  This piece picks up where the announcement in the Fedora Magazine stopped.  It has more details that may be of interest to contributors.

Why did we move to Antora?

Antora is a static site generator by the same folks behind the AsciiDoctor project. The engine converts a collection of version controlled documentation written in AsciiDoc into an html site that is easy to scale and deliver via our web infrastructure. The project, while new, is active and responsive and has already made several solid beta releases, and is being embraced by static site documentarians.

Antora is designed to be the premier AsciiDoc rendering engine, and Dan Allen, one of the principals of the project, has spoken at a FUDCon about AsciiDoc and was thinking specifically about our documentation set when doing the design work. Adam has designed a container-based build that should be easily run on both individual workstations for local preview and in the CentOS CI build system for production.

We had originally selected AsciiBinder as our AsciiDoc tool, but found we continually needed to add additional features and functionality. We rolled out AsciiBinder about a year ago, but did not announce it as we still wanted to get it working fully. This tool, built by OpenShift, wasn’t designed for sites as large and complicated as ours. It has several projects using it, but is in maintenance mode until more technical contributors are found. An additional motivation was that our DocBook tool, publican, is no longer supported and no longer has an effective upstream.

Why did we move to AsciiDoc?

We moved to AsciiDoc from DocBook to decrease the friction for contribution. DocBook is a fantastic markup for technical writing and probably provides the most expressive possible writing markup. However, it is XML and hard to write. There is a lot of opportunity to “over-markup” documents, yet we weren’t getting full value out of the markup. We were not doing special checks or anything that leveraged DocBook’s power.

In Fedora docs we wanted something easier to use. We debated many ideas, including wikis and something as simple as Markdown. Wikis were discounted because they don’t really provide information architecture and quickly turn into large piles outdated mess than isn’t easily audited. Markdown was seriously considered as it is very simple to use and learn, however it isn’t quite expressive enough to capture the things we wanted to markup. In particular, like a wiki, it marks things up by how they should appear, not by what they are. AsciiDoc can be used this way as well, but gives us flexibility to mark things up based on their function as well. We are still writing our AsciiDoc style guide, but we believe the markup is simple enough to learn that you will quickly be writing in it.

A bonus is that our primary downstream consumer and contributor, Red Hat, has independently decided to move to AsciiDoc as well. We were glad to see this as for the first few months after we made our decision we were worried that we may be closing off their avenue for contribution. Red Hat is also working to on a new concept in writing called Modular Docs (I know … modular) that allows for more realistic content re-use. You can see some of that work going on in the Quick Docs Repository.

What happened to translations?

One challenge with moving to AsciiDoc has been localization. The non-XML markups are just not well supported by good localization tools. We’ve been working with po4a to try and bridge this gap, however, we have needed improve the upstream AsciiDoc parser to get us to a good place. The upstream is very small, but has been active for over a decade. In the middle of us starting to work with them, their hosting from the Debian project was shut down. I was able to help them migrate to Red Hat provided webspace managed by the Open Source and Standards community cage team. They have also moved their code to GitHub. These two moves have resulted in a resurgence in contribution.

We were part-way to a translation solution when we decided to move away from the then-new AsciiBinder tool to Antora.

Now that we have po4a ready and we are on Antora, we need to get it all working together. We have some test projects in Zanata and need to work on translating the Antora provided structure and the documentation. This has been a slow process as there are not a lot of tooling people working on Fedora Docs. This is an opportunity for tooling contribution as well as language testing if you’re interested.

How does all of this actually work?

There will be a Fedora Classroom with Adam on the 6th of August where he will talk about how the tooling works and how to contribute. This will be a mechanics of contribution, not content of contribution, focused conversation.

The short version is that the files are organized into Modules by Antora (there is that word again!). These modules are assembled into the final site. This separates the files on disk from their urls and from their their information architecture (menus, hierarchy, findability). I encourage you to read the Antora docs if you want the details, they are, as you would expect, well written. If you’re just interested in contribution, the key thing to know is that each module has a nav.adoc file that defines the left-hand menu and the actual content is located in the pages directory.

On the backend, the Antora Playbook’s site.yml lists all of the repositories with the documentation sources. These are assembled into static html and pushed to a publishing repository. Fedora Infrastructure pulls the contents of this repo and pushes to stage every 5 minutes and to production once an hour.

What is Quick Docs and is it out to kill the wiki?

Quick Docs is Matthew Miller’s idea for trying to get our users out of the wiki. The Fedora wiki can be very confusing and hard to find information in. There are lots of outdated pages as people have been, correctly, using it as a whiteboard, but then they forget to erase their old work. There are many areas of valuable content, but it’s easy to make one click and be in a wilderness of incomplete, incorrect, or obsolete pages with no indication that you’ve gone astray.

With Quick Docs, we started by creating stubs for the most commonly used user documentation. We have been slowly getting that content migrated to the docs site and putting in redirects from the Wiki. Right now the information architecture of Quick Docs needs some help, but with the new tooling we are on our way to being able to solve this problem.

To be very clear, this is not an attempt to kill the wiki. The Fedora wiki serves an important need as a planning space for the Project. It is a great place to “whiteboard” ideas and share drafts. Where it tends to fall down is when it is used as a durable documentation store or when people are left trying to sort through it to find current information. A version controlled documentation site is better for this. No team or doc is being forced to move, however, we hope you will see the value (and ask if you don’t) and choose to move to this more visible and findable location.

How do I contribute to the docs?

There are several ways to contribute to the docs:

I encourage you to contact the Documentation Mailing List or for content issues, Petr Bokoč, or for tooling issues, Adam Šamalík.

Categories: Fedora Project Community

2 Comments

  1. Leslie Satenstein

    August 3, 2018 — 3:19 am

    Hi Brian, Adam
    I’ve been running asciidoctor against each …/en-US/*.adoc, for 3 documents.
    One frustration, perhaps it is me, is that the asciidoctor includes do not work.
    I end up with many {XXXX} unsatisfied substitutions. Am I the only one experiencing this problem?

    Since asciidoctor is not in the rpmfusion repositories, I was/am not able to raise a bug report.
    Could a mini project be created to have asciidoctor migrated to rpmfusion?

    And yes, asciidoctor is easy to use and the quality of output is verygood to excellent.

    • I think there are two issues here:

      1. asciidoctor is packaged for Fedora, so consider installing it from the distribution repositories with `dnf install rubygem-asciidoctor`

      2. When you run asciidoctor against a file destined to be built with Antora (or asciibinder), the entity definitions may not get included as the include pathing is for the builder, not asciidoctor. With the launch of Antora powered-docs we have a `build.sh` and a `preview.sh` that use containers to give you the proper preview experience.

Leave a Reply

Your email address will not be published.

*

Copyright © 2018 Fedora Community Blog

Theme by Anders NorenUp ↑

%d bloggers like this: