Continuous Integration Practices with Docker— Part 2

This is part 2 of my previous blog post on practical software development principles.

There are so many docker file examples online but they don’t go into detail or don’t give all best practices at once. Many developers use default tools made by the language’s build tool or generic docker images. Most of the developers just build their code and copy into a generic image and unfortunately, they think it is the right way since there is no quality control.

Use Docker, not just for distributing your application but also for standardizing your continuous delivery pipeline.

In my previous article, I mentioned that every developer should be able to build, test, and debug locally. This is also true for your Continuous Delivery platform: a developer should be able to run the same test and build infrastructure locally. “Tests pass on my computer but not on CI” is another version of “it works on my computer”. Often CI platform has a different architecture, less CPU/RAM. It is very common that tests depend on timeouts and it fails on CI because it runs slower. Sometimes there is a dependency you had on your computer but forgot about it. It is a good practice to agree that your tests and builds should be able to run inside a docker container and a developer who joins the team should be able to run it with just reading README and having docker-desktop installed. This will decrease the onboarding of a new developer significantly.

I will explain this article over an example Golang application since it is a compiled language and more standardized.

Before reading this you may also want to check Dockerfile Best Practices on the official docker page. I will stress on specific parts and how to achieve it.

These are the best practices that I will stress:

• Use Multistage builds to split build and runtime image.
• Make your final image as small as possible.
• Run only one process in one docker container.
• Run as a non-root user.

Use Multistage builds to split build and runtime image.

It is a very common and good practice to split your build and runtime image.

This is a multistage example from official docker page:

FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html  
COPY app.go    .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .

FROM alpine:latest  
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]

The first stage named “builder” has the Golang compiler, installs dependencies, and based on Debian. It includes common tools like bash, curl that should not be included in a production image.

Let’s concentrate on the builder:

Our main idea is to have a standard build image with best practices that everyone can run.

We are going to build our first stage image in a totally different repository where we install commonly used libraries and tools.

# golang debian buster 1.14.1 linux/amd64
# https://github.com/docker-library/golang/blob/master/1.14/buster/Dockerfile
FROM golang@sha256:eee8c0a92bc950ecb20d2dffe46546da12147e3146f1b4ed55072c10cacf4f4c as builder
# Ensure ca-certficates are up to date
RUN update-ca-certificates
WORKDIR /tmp/app
# use modules
COPY go.mod tools.go /tmp/app/
ENV GO111MODULE=on
RUN go mod tidy
RUN go mod download
RUN go mod verify
RUN go get github.com/mattn/goveralls
RUN rm -rf /tmp/app
ONBUILD RUN update-ca-certificates
ONBUILD COPY . .
ONBUILD RUN go test -v -cover -race -coverprofile=./coverage.out ./...
# Enforce code coverage check
ARG CI_SERVICE
ARG COVERALLS_TOKEN
ONBUILD RUN goveralls -coverprofile=./coverage.out -service=$CI_SERVICE -repotoken=$COVERALLS_TOKEN
ONBUILD RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
      -ldflags='-w -s -extldflags "-static"' -a \
      -o /app/app .

This Dockerfile is based on the secure Dockerfile example of C Hemidy and his blog post about it is here.

• We start with the official Golang 1.14.1 image and use it by sha256 hash for security.
• Always update certificate authority certificates since it is required for TLS enabled calls.
• We used tools.go file to store common dependencies and install and validate these dependencies.
• Install extra tools you use, I installed goveralls a tool to update coveralls code coverage results.

Use ONBUILD calls to define commands you want to run when another image is build based on this image. This is important to decide and follow a common build and validation process among the services. Large companies have tooling departments that build these or you can have someone with practical knowledge about these tasks. Unfortunately, it is hard to find these people.

Here I added test, code coverage check, and standard production build with ONBUILD commands. These will run when the individual developers build their image.

You can find the source code of this base image here:

bgokden/golang_base_builder_image_template

I have updated one of my weekend projects to use this base image and its Dockerfile turned into this:

# FROM berkgokden/go_builder_base:v0.0.1 AS builder
FROM berkgokden/go_builder_base@sha256:d3098e69bd256a6d93fc9fa81c3256796b3512ae4d724451d3e5d1644f009063 AS builder
FROM gcr.io/distroless/static@sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4
COPY --from=builder /app/app /app/app
ENTRYPOINT ["/app/app"]
EXPOSE 9090 9091

As you can see it is just referencing the layers, copying the application binary, and exposing the ports. If you are going to write many microservices, you need to be able to write short easy to write Dockerfiles. I can even write this by hearth except for the sha256 hashes and it is a secure and tiny image. Only 7.45MB. A practical image for a microservice would be around 10MB.

Source code and Dockerfile can be found here:

bgokden/go-http-redirector

I would like to mention google distroless image here. This image has an image built from scratch that has the bare minimum required to run a binary image without external dependencies. Most of the known vulnerabilities are in these libraries so it is better if we exclude them. Also, it runs the application as a non-root user.

You can build your own image from scratch but you should definitely check Google Distroless image:

GoogleContainerTools/distroless

Sum it up:

Docker is a great tool to standardize the development, build, and production environment. It can improve and simplify the CI pipeline with some simple steps. Follow these steps and enjoy small and secure docker images.

Follow me on Linkedin:

Berk Gökden - Senior Software Engineer - Caspar.AI | LinkedIn

Or Github:

bgokden - Overview