Practical Bazel: Retrieving Secrets from Netrc in Custom Rules

Bazel developers are currently working on adding the ability to retrieve secrets using a credential-helper executable, similar to how other tools like Docker and Git handle managing secrets. Until then, the recommended approach is to store secrets in ~/.netrc. This blog post explains how to write a custom Bazel rule which reads secrets from ~/.netrc.

A number of built-in Bazel rules, such as http_archive(), natively support retrieving credentials from ~/.netrc. To use this feature, you:

Add the appropriate secret to ~/.netrc using the typical netrc format:

machine storage.cloudprovider.com
    password RANDOM-TOKEN

Use the auth_patterns option in http_archive() to specify how you want the secrets used:

1
2
3
4
5
6


http_archive(
    ...,
    auth_patterns = {
        "storage.cloudprovider.com": "Bearer <password>"
    }
)

The above causes http_archive() to add the HTTP header Authorization: Bearer RANDOM-TOKEN to all outbound requests to storage.cloudprovider.com. This mechanism is generic enough to handle nearly all authentication use cases.

However, let’s say you are writing your own custom rule which downloads data over HTTP. How would you retrieve the secrets from ~/.netrc? You could start by reading the source code to the http_archive() rule, or you could just use the information below.

The key functionality is provided by the functions read_netrc(), read_user_netrc(), and use_netrc() in the file @bazel_tools//tools/build_defs/repo:utils.bzl. These functions parse .netrc files and produce the data structures necesasry to support authentication.

A simple usage of these methods looks something like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


# my_http_archive: The start of a custom implementation of http_archive()
load("@bazel_tools//tools/build_defs/repo:utils.bzl", "read_netrc", "read_user_netrc", "use_netrc")

def _get_auth(ctx, urls):
    """Given the list of URLs obtain the correct auth dict."""
    if "NETRC" in ctx.os.environ:
        netrc = read_netrc(ctx, ctx.os.environ["NETRC"])
    else:
        netrc = read_user_netrc(ctx)
    return use_netrc(netrc, urls, ctx.attr.auth_patterns)

def _impl(ctx):
    url = ctx.attr.url
    auth = _get_auth(ctx, [url])
    output = ctx.attr.output_file if ctx.attr.output_file != "" else ctx.attr.name

    ctx.download(
        output = output,
        url = url,
        sha256 = ctx.attr.sha256,
        auth = auth,
    )

    # ...

my_http_archive = repository_rule(
    attrs = {
        "url": attr.string(
            mandatory = True,
            doc = "The URL of the artifact to pull",
        ),
        "sha256": attr.string(
            default = "",
            doc = "The expected SHA-256 of the downloaded artifact",
        ),
        "output_file": attr.string(
            default = "",
            doc = "The output filename of the artifact (defaults to the rule's name)",
        ),
        "auth_patterns": attr.string_dict(
        ),
    },
    implementation = _impl,
)

However, you are not limited to using ~/.netrc solely for HTTP credentials – you can use it to store basically any type of secret. I’ve also used it to store things like GitHub auth tokens with the function below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


def _get_github_auth_token(ctx):
    """Gets the github auth token from ~/.netrc and return it"""
    if ctx.attr.netrc:
        netrc = read_netrc(ctx, ctx.attr.netrc)
    elif "NETRC" in ctx.os.environ:
        netrc = read_netrc(ctx, ctx.os.environ["NETRC"])
    else:
        netrc = read_user_netrc(ctx)

    for host in ["github.com", "api.github.com"]:
        url = "https://{host}/".format(host = host)
        auth_dict = use_netrc(netrc, [url], {host: "Bearer <password>"})
        if auth_dict:
            return auth_dict[url]["password"]

    fail("""Could not find GitHub auth token.  Add the following line to your ~/.netrc:

machine github.com password ghp_...

Where ghp_... is a GitHub personal access token with appropriate permissions.
""")