Using Docker Buildkit for OS Agnostic Builds

One of the many things I’m working on for wolfSSL right now is a Markdown based documentation build system which has to massage some of the input sources quite a bit. The build dependencies for this are getting increasingly difficult to set up. I therefore decided to make it easier for everyone by using Docker Buildkit.

Docker Buildkit basically gets everything you need to build (OS and dependencies) and caches it for subsequent runs, builds the code and allows you to copy the assets out of the build. The following is the file I created, the file format should be similar to anyone who has used a Dockerfile before:

FROM texlive/texlive:latest AS builder

WORKDIR /deps

RUN apt-get -y update
# Pandoc and mkdocs are the tools we use
# Doxygen to convert the Doxygen docs
# build-essential and cmake to build the dependencies
# libfmt-dev nlohmann-json3-dev libspdlog-dev and libcxxopts-dev are needed by Doxybook2
# fonts-noto is needed by our PDF generator
RUN apt-get -y install pandoc mkdocs doxygen git build-essential cmake libfmt-dev nlohmann-json3-dev libspdlog-dev libcxxopts-dev fonts-noto

# Inja is a dep for Doxybook2
RUN git clone --depth=1 https://github.com/pantor/inja
RUN cd inja && cmake . -DBUILD_TESTING=OFF -DBUILD_BENCHMARK=OFF && make install

RUN git clone --depth=1 https://github.com/matusnovak/doxybook2
RUN cd doxybook2 && cmake . && make install

WORKDIR /src/wolfssl
# Cache getting wolfSSL source for Doxygen docs
RUN git clone --depth 1 https://github.com/wolfSSL/wolfssl
# Copy the source files into Docker, this and any subsequent steps won't be cached
COPY . .

# Parallel build PDF
FROM builder AS stage1
WORKDIR /src/wolfssl
RUN make pdf

# Parallel build HTML
FROM builder AS stage2
WORKDIR /src/wolfssl
RUN make html

FROM scratch AS export-stage
COPY --from=stage1 /src/wolfssl/wolfssl.pdf .
COPY --from=stage2 /src/wolfssl/html ./html

Everything up to the COPY . . will be cached as the output is expected to be the same every time, this copy step will copy the sources over to the Docker image. It is slow to run the first time as the “texlive” Docker image is over 1GB in size and there are a few hundred MB in “apt” dependencies that need to be installed, so having all this cached is very useful.

I’m not too concerned that the wolfSSL git tree clone may be an old cache because the Makefile that is run in subsequent steps will update it. The key thing that makes Docker Buildkit different from just using docker build is that the FROM builder stages are executed in parallel, the progress output is also a lot more friendly:

You can see from the title bar here that this is running from a Makefile. The make docker step simply looks like this:

.PHONY: docker
docker:
 @DOCKER_BUILDKIT=1 docker build --target=export-stage -t doc_build --output=build .

Breaking this down the @ just hides from the user the command being run. DOCKER_BUILDKIT=1 enabled the Docker Buildkit mode. The export-stage as seen in the Dockerfile is what we want to run, that then pulls in stage1 and stage2 which in-turn pulls in builder. Basically if you read the FROM steps in the Dockerfile from bottom to top you can see the step dependency tree. The output parameter is where the file stage will write to in the local filesystem.

The final stage just copies the output files from the Docker build to the host filesystem, so that we actually have the required build output. Once the build is complete it looks like this:

The rest of this documentation build system will likely be opened up soon. We are working on cleaning it up and putting some finishing touches to it. But it uses a Markdown base for the chapters and combines it with Doxygen -> Markdown converted API reference. It then does some post-processing so that the Markdown is massaged correctly for Pandoc and mkdocs, which both use slightly different Markdown syntax.

Whilst this is just building documentation, this could be used to build anything where you want people to be able to compile no matter what OS they choose to develop on. It could also be incorporated as part of a CI/CD system.