Let’s say you are using Bazel to build a C program which links against
a system-provided version of libcurl, the multiprotocol file transfer
library. What is the best way to link your program against this library
within Bazel? This blog post provides an answer to that question.
This post describes a pattern for implementing a continuous integration
(CI) pipeline using Bazel. This pattern is my starting point whenever
I set up a new Bazel-based project in CI, after which I add any
project-specific pipeline customizations.
This pipeline is purely about the CI (build to release) stages of a
pipeline. A full continuous delivery (CD) pipeline, which includes
deployment, will be discussed in a later post.
In Bazel, a successful build should be a quiet build. While build failures
should, of course, print ample information to stderr to aide in troubleshooting,
any custom Bazel code you write should not output progress information to stdout
or stderr. Let Bazel be responsible for overall build progress reporting.
If you are passing secrets via environment variables that are retrieved
by command-line programs, there’s an even easier way to do it – use the
command rule from Atlassian’s bazel-tools repo and its
raw_environment attribute.
An executable rule which can be executed via bazel run is the natural
way to model interactions with external systems in Bazel such as uploading
build artifacts to a remote artifact repository. For example, imagine
a rules_artifactory ruleset which includes a rule artifactory_push()
executable rule which uploads a compiled .dpkg to an Artifactory
apt repository, or a rules_docker ruleset which has a rule
docker_push() which pushes a Docker image to a remote image repository.
When writing custom rules, you often need to invoke executables with
argument lists. For example, let’s say you are writing a custom rule
that executes gcc to compile a set of input source files. You
could write:
Bazel started on Linux and Mac OS, and most people use Bazel on these
platforms exclusively, but Bazel can execute on Windows as well. However,
Windows has enough idiosynchatic differences that writing a single,
operating-system agnostic rule that executes on both Windows and Linux/Mac
is quite hard. Often it is easiest to have the rule detect whether
it is running on Windows and execute different behavior.
When writing a custom rule that generates files, be sure to add
prefixes to all filenames so that multiple instances of your rule
can be instantiated within the same Bazel package.
One tends to write a lot of Bash scripts when using Bazel. In
order to make these scripts more robust, enable
Bash’s unofficial strict mode by starting all scripts
like this:
Bazel requires all files to end with LF (not CR-LF) in order to work correctly.
If you are performing cross-platform Bazel development with Windows users, you
can force this setting for all text files in your repo by creating a
.gitattributes file with the following content:
Quick Bazel tip for today: If you want to build everything except for a specific subtree, you can prefix the subtree you want to exclude with a -.
For example, to build everything except for //client_access_library/...:
bazel build -- //... -//client_access_library/...
Bazel’s philosophy strongly encourages binding to exact, specific versions of all third-party dependencies to help ensure reproducible builds. As Bazel users, we must remember to extend this philosophy to Bazel itself.
When setting up a Bazel-based build system, you should choose a specific version of Bazel and require all developers and the build system to use it. This can be done in a few ways:
Use Bazelisk and a .
Read more...
When writing custom Bazel rules, you spend a lot of time either reading or writing Bazel File objects.
File objects have two properties for accessing the underlying file path: File.path and File.short_path. When writing custom rules, I often chose one of the two properties at random, and switched to the other if it didn’t work right.
I wrote some simple custom rules to test the various combination of rule types and file types to determine when I should use path or short_path.
Read more...
Bazel is a powerful yet complicated system, and it can be intimidating to newcomers.
While the Bazel user guide and user manual preach the benefits of giving Bazel full control over your build process by rewriting all build processes using Bazel-native rulesets (as Google reportedly does internally), this is an immense amount of work. Specifically, if you are integrating third-party software into your Bazel-based build process, reverse engineering and rewriting the third-party project’s build system into Bazel can easily take days – and then you need to maintain it.
Read more...
In 2020, I led the redesign and re-implementation of the object storage system behind RelativityOne.
As part of this project we reengineered the continuous delivery pipeline of the service to embrace the philosophy of a service-wide monorepo with a Bazel-based build system. We chose Bazel because we wanted a build system that could support many different languages (the service has code written in C, C#, Python, Go, Terraform, Packer, and other languages…) while remaining fast and correct.
Read more...