diff --git a/content/api/api-container-builds-tutorial.mdx b/content/api/api-container-builds-tutorial.mdx new file mode 100644 index 0000000..efd507e --- /dev/null +++ b/content/api/api-container-builds-tutorial.mdx @@ -0,0 +1,539 @@ +--- +title: 'Container builds API tutorial' +--- + +This tutorial walks you through using Depot API to build Docker images programmatically. The container builds API allows you to build Docker images on behalf of your users without managing build infrastructure. + +Depot provides two SDKs for building images via the API: + +**Node.js SDK + Depot CLI** + +The Node.js SDK handles project management and build registration, then delegates the actual build to the Depot CLI. This approach is simpler and requires less code. + +**Go SDK + BuildKit** + +The Go SDK provides direct access to BuildKit, giving you full control over the build process. You manage the connection, configuration, and build steps yourself. + +--- + +## Choose your approach + +Select the SDK that best fits your use case: + +
+ + Node.js SDK + Depot CLI + + + + +
+ +## Prerequisites + +- A Depot account with an organization +- Node.js installed locally +- [Depot CLI](/docs/cli/installation) installed + +## Setup + +This tutorial uses code from our [example repository](https://github.com/depot/examples/tree/main/build-api). Clone it to follow along: + +```shell +git clone https://github.com/depot/examples.git +cd examples/build-api +``` + +The example repository contains the following Node.js examples under (`nodejs/`): + +- [`list-projects.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/list-projects.js) - List all projects +- [`create-project.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/create-project.js) - Create a new project +- [`delete-project.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/delete-project.js) - Delete a project +- [`create-build.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/create-build.js) - Build image with options (load/save/push) + +To get started, install Node.js dependencies: + +```bash +cd nodejs +npm install +``` + +## Step 1: Create an organization token + +1. Navigate to your organization settings in the Depot dashboard +2. Scroll to **API Tokens** section +3. Enter a description (e.g., `test-token`) and click **Create token** +4. Copy the token and save it securely (you won't see it again) + +Set the token as an environment variable: + +```shell +export DEPOT_TOKEN= +``` + +## Step 2: Install Depot CLI + +Install via curl: + +```shell +curl -L https://depot.dev/install-cli.sh | sh +``` + +Or via Homebrew (macOS): + +```shell +brew install depot/tap/depot +``` + +## Step 3: Create a project + +Projects in Depot provide isolated builder infrastructure and cache storage. We recommend creating a separate project for each customer organization to maximize cache effectiveness and prevent cache poisoning. + +To create a project, use the `ProjectService.createProject` method with your organization token: + +```javascript +const {depot} = require('@depot/sdk-node') + +const headers = { + Authorization: `Bearer ${process.env.DEPOT_TOKEN}`, +} + +const result = await depot.core.v1.ProjectService.createProject( + { + name: 'my-project', + regionId: 'us-east-1', + cachePolicy: {keepBytes: 50 * 1024 * 1024 * 1024, keepDays: 14}, // 50GB, 14 days + }, + {headers}, +) + +console.log(result.project.projectId) +``` + +Try it with the example: `node nodejs/src/create-project.js my-project` + +Save the `projectId` from the output, you'll need it for builds. + +Example output: + +```text +_Project { + projectId: 'krt0wtn195', + organizationId: '3d1h48dqlh', + name: 'my-project', + regionId: 'us-east-1', + createdAt: Timestamp { seconds: 1708021346n, nanos: 83000000 }, + cachePolicy: _CachePolicy { keepBytes: 53687091200n, keepDays: 14 } +} +``` + +## Step 4: Build a Docker image + +To build an image, first register a build with the Build API using `BuildService.createBuild`. This returns a build ID and one-time build token that you pass to the Depot CLI: + +```javascript +const {depot} = require('@depot/sdk-node') +const {exec} = require('child_process') + +const headers = { + Authorization: `Bearer ${process.env.DEPOT_TOKEN}`, +} + +// Register the build +const result = await depot.build.v1.BuildService.createBuild({projectId: ''}, {headers}) + +// Execute build with Depot CLI +exec( + 'depot build --load .', + { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, + }, + (error, stdout, stderr) => { + if (error) { + console.error(`Error: ${error}`) + return + } + console.log(stdout) + }, +) +``` + +Try it with the example: `node nodejs/src/create-build.js ` + +The `--load` flag downloads the built image to your local Docker daemon. + +## Step 5: Run the container + +List your local Docker images: + +```shell +docker image ls +``` + +Run the built container: + +```shell +docker run +``` + +You should see "Hello World" output from the Node.js application. + +## Step 6: Save to a registry + +### Push to Depot Registry + +Instead of loading locally with `--load`, you can save the image to Depot Registry using the `--save` flag: + +```javascript +exec('depot build --save .', { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, +}) +``` + +Try it: `node nodejs/src/create-build.js save` + +The build output shows how to pull or push the saved image: + +```text +Saved target: + To pull: depot pull --project + To push: depot push --project --tag +``` + +### Push to external registries + +To push directly to Docker Hub, GHCR, ECR, or other registries during the build, use the `--push` flag with `--tag`: + +```javascript +exec('depot build --push --tag docker.io/myuser/myapp:latest .', { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, +}) +``` + +First authenticate with `docker login`, then pushing to other registries simply requires setting the proper image name: + +```shell +# Docker Hub +node nodejs/src/create-build.js push docker.io/myuser/myapp:latest + +# GitHub Container Registry +node nodejs/src/create-build.js push ghcr.io/myorg/myapp:latest + +# AWS ECR +node nodejs/src/create-build.js push 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp:latest +``` + +
+
+ +
+ + Go SDK + BuildKit + + + + +
+ +## Prerequisites + +- A Depot account with an organization +- Go 1.21+ installed locally + +## Setup + +This tutorial uses code from our [example repository](https://github.com/depot/examples/tree/main/build-api). Clone it to follow along: + +```shell +git clone https://github.com/depot/examples.git +cd examples/build-api +``` + +The Go examples use two packages: + +- **Buf Connect API** (`buf.build/gen/go/depot/api`) - For project management +- **Depot Go SDK** (`github.com/depot/depot-go`) - For builds + +Available examples: + +- [`list-projects/main.go`](https://github.com/depot/examples/blob/main/build-api/go/list-projects/main.go) - List all projects +- [`create-project/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-project/main.go) - Create a new project +- [`delete-project/main.go`](https://github.com/depot/examples/blob/main/build-api/go/delete-project/main.go) - Delete a project +- [`create-build/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-build/main.go) - Build image (saved to Depot) +- [`build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go) - Build and push to external registry + +Install dependencies: + +```bash +cd go +go mod download +``` + +## Build flow overview + +Building with the Go SDK involves three steps: + +1. **Register a build** - Request a build from the Depot API +2. **Acquire a builder machine** - Get an ephemeral BuildKit machine with your project cache +3. **Build and push** - Connect to BuildKit and execute the build + +See the complete implementation in [`build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go). + +## Step 1: Create an organization token + +1. Navigate to your organization settings in the Depot dashboard +2. Scroll to **API Tokens** section +3. Enter a description (e.g., `test-token`) and click **Create token** +4. Copy the token and save it securely (you won't see it again) + +Set the token as an environment variable: + +```shell +export DEPOT_TOKEN= +``` + +## Step 2: Create a project + +Projects in Depot provide isolated builder infrastructure and cache storage. To create a project, use the Buf Connect API client with `ProjectService.CreateProject`: + +```go +import ( + "net/http" + corev1 "buf.build/gen/go/depot/api/protocolbuffers/go/depot/core/v1" + "buf.build/gen/go/depot/api/connectrpc/go/depot/core/v1/corev1connect" + "connectrpc.com/connect" +) + +token := os.Getenv("DEPOT_TOKEN") + +// Create the Project Service client +client := corev1connect.NewProjectServiceClient( + http.DefaultClient, + "https://api.depot.dev", +) + +// Create a new project +req := connect.NewRequest(&corev1.CreateProjectRequest{ + Name: "my-project", + RegionId: "us-east-1", + CachePolicy: &corev1.CachePolicy{ + KeepGb: 50, // 50GB + KeepDays: 14, // 14 days + }, +}) + +// Add authentication header +req.Header().Set("Authorization", fmt.Sprintf("Bearer %s", token)) + +resp, err := client.CreateProject(ctx, req) +if err != nil { + log.Fatal(err) +} + +log.Printf("Project ID: %s", resp.Msg.Project.ProjectId) +``` + +Try it with the example: `go run ./create-project/main.go my-project` + +Save the project ID, you'll need it for builds. + +## Step 3: Register a build + +To start a build, register it with the Build API using `build.NewBuild`. This returns a build ID and one-time build token: + +```go +import ( + "github.com/depot/depot-go/build" + cliv1 "github.com/depot/depot-go/proto/depot/cli/v1" +) + +token := os.Getenv("DEPOT_TOKEN") +projectID := os.Getenv("DEPOT_PROJECT_ID") + +build, err := build.NewBuild(ctx, &cliv1.CreateBuildRequest{ + ProjectId: projectID, +}, token) +if err != nil { + log.Fatal(err) +} + +// Report build result when finished +var buildErr error +defer build.Finish(buildErr) +``` + +The `build.Finish()` call reports success or failure back to Depot when your build completes. + +## Step 4: Acquire a builder machine + +With your build registered, acquire an ephemeral BuildKit machine using `machine.Acquire`. The machine comes pre-configured with your project's cache: + +```go +import "github.com/depot/depot-go/machine" + +buildkit, buildErr := machine.Acquire(ctx, build.ID, build.Token, "arm64") +if buildErr != nil { + return +} +defer buildkit.Release() +``` + +Specify `"arm64"` or `"amd64"` for your target platform. Released machines stay alive for 2 minutes to serve subsequent builds. + +## Step 5: Connect to BuildKit + +Connect to your BuildKit machine using `buildkit.Connect`: + +```go +import "github.com/moby/buildkit/client" + +buildkitClient, buildErr := buildkit.Connect(ctx) +if buildErr != nil { + return +} +``` + +This establishes a secure mTLS connection to the BuildKit endpoint. + +## Step 6: Configure the build + +Configure your build by creating a `SolveOpt` with your Dockerfile path, build context, and export settings: + +```go +import ( + "github.com/docker/cli/cli/config" + "github.com/moby/buildkit/session" + "github.com/moby/buildkit/session/auth/authprovider" +) + +solverOptions := client.SolveOpt{ + Frontend: "dockerfile.v0", + FrontendAttrs: map[string]string{ + "filename": "Dockerfile", + "platform": "linux/arm64", + }, + LocalDirs: map[string]string{ + "dockerfile": ".", + "context": ".", + }, + Exports: []client.ExportEntry{ + { + Type: "image", + Attrs: map[string]string{ + "name": "myuser/myapp:latest", + "oci-mediatypes": "true", + "push": "true", + }, + }, + }, + Session: []session.Attachable{ + authprovider.NewDockerAuthProvider(config.LoadDefaultConfigFile(os.Stderr), nil), + }, +} +``` + +The `Session` uses your Docker credentials from `docker login` to authenticate registry pushes. + +## Step 7: Stream build output (optional) + +To monitor build progress, create a status channel and process BuildKit status messages: + +```go +import "encoding/json" + +buildStatusCh := make(chan *client.SolveStatus, 10) +go func() { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + for status := range buildStatusCh { + _ = enc.Encode(status) + } +}() +``` + +This streams build progress in real-time as JSON. + +## Step 8: Build and push + +Execute the build with `buildkitClient.Solve`. BuildKit automatically reuses cached layers from your project: + +```go +_, buildErr = buildkitClient.Solve(ctx, nil, solverOptions, buildStatusCh) +if buildErr != nil { + return +} +``` + +When complete, your image is pushed to the registry specified in the `Exports` configuration. + +Try the complete example: `DEPOT_PROJECT_ID= go run ./build-and-push/main.go` + +### Push to third-party registries + +To push to external registries, configure the full registry path in your image name and provide authentication. + +#### Set the full registry path: + +```go +Exports: []client.ExportEntry{ + { + Type: "image", + Attrs: map[string]string{ + "name": "docker.io/myuser/myapp:latest", // or ghcr.io, ECR, etc. + "oci-mediatypes": "true", + "push": "true", + }, + }, +}, +``` + +The `build-and-push` example supports two options for authentication: + +#### Option 1: Docker login credentials (default) + +After running `docker login`, BuildKit automatically uses credentials from `~/.docker/config.json`: + +```bash +docker login docker.io +DEPOT_PROJECT_ID= go run ./build-and-push/main.go docker.io/user/app:latest +``` + +#### Option 2: Programmatic credentials (for CI/CD) + +Provide credentials via environment variables: + +```bash +DEPOT_PROJECT_ID= \ +REGISTRY_USERNAME=myuser \ +REGISTRY_PASSWORD=mytoken \ +REGISTRY_URL=https://index.docker.io/v1/ \ +go run ./build-and-push/main.go docker.io/user/app:latest +``` + +The example automatically detects which method to use based on the presence of `REGISTRY_USERNAME` and `REGISTRY_PASSWORD`. + +See the complete working examples in the repository: [`go/create-build/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-build/main.go) and [`go/build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go). + +
+
+ +--- + +## Next steps + +- Review the [API reference](/docs/api/overview) for complete API documentation +- Explore the [Node.js SDK on GitHub](https://github.com/depot/sdk-node) +- Explore the [Go SDK on GitHub](https://github.com/depot/depot-go) +- Learn about [BuildKit in depth](/blog/buildkit-in-depth) diff --git a/content/container-builds/reference/api-authentication.mdx b/content/api/authentication.mdx similarity index 100% rename from content/container-builds/reference/api-authentication.mdx rename to content/api/authentication.mdx diff --git a/content/container-builds/reference/api-overview.mdx b/content/api/overview.mdx similarity index 99% rename from content/container-builds/reference/api-overview.mdx rename to content/api/overview.mdx index 86b92ff..3f3fc3d 100644 --- a/content/container-builds/reference/api-overview.mdx +++ b/content/api/overview.mdx @@ -13,7 +13,7 @@ Our API is built with Connect, offering [multiprotocol support](https://connectr ## Authentication -Authentication to the API is handled via an `Authorization` header with the value being an Organization Token that you generate inside of your Organization Settings. See the [Authentication docs](/docs/container-builds/reference/api-authentication) for more details. +Authentication to the API is handled via an `Authorization` header with the value being an Organization Token that you generate inside of your Organization Settings. See the [Authentication docs](/docs/api/authentication) for more details. ## Security diff --git a/content/cache/reference/bazel.mdx b/content/cache/reference/bazel.mdx index c8039bd..4bfbbae 100644 --- a/content/cache/reference/bazel.mdx +++ b/content/cache/reference/bazel.mdx @@ -10,30 +10,13 @@ description: Learn how to use Depot remote caching for Bazel builds ## Configuring Bazel to use Depot Cache -Depot Cache can be used with Bazel from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Bazel from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Bazel - each runner is launched with a `$HOME/.bazelrc` file that is pre-populated with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Bazel to use Depot Cache as described below. - -### From your local machine or any CI/CD system - -To manually configure Bazel to use Depot Cache, you will need to set two build flags in your `.bazelrc` file. Configure Bazel to use the Depot Cache service endpoint and set API token as the `authorization` header: - -```bash -build --remote_cache=https://cache.depot.dev -build --remote_header=authorization=DEPOT_TOKEN -``` - -If you are a member of multiple organizations, and you are authenticating with a user token, you must additionally specify which organization to use for cache storage with the `x-depot-org` header: - -```bash -build --remote_header=x-depot-org=DEPOT_ORG_ID -``` - -Once Bazel is configured to use Depot Cache, you can then run your builds as you normally would. Bazel will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to override the `$HOME/.bazelrc` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Bazel to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with Bazel in `depot/build-push-action` @@ -74,3 +57,94 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure Bazel to use Depot Cache, you will need to set two build flags in your `.bazelrc` file. Configure Bazel to use the Depot Cache service endpoint and set API token as the `authorization` header: + +```bash +build --remote_cache=https://cache.depot.dev +build --remote_header=authorization=DEPOT_TOKEN +``` + +If you are a member of multiple organizations, and you are authenticating with a user token, you must additionally specify which organization to use for cache storage with the `x-depot-org` header: + +```bash +build --remote_header=x-depot-org=DEPOT_ORG_ID +``` + +After Bazel is configured to use Depot Cache, you can then run your builds as you normally would. Bazel will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with Bazel in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to mount the secret and configure Bazel: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Create .bazelrc with cache configuration +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + echo "build --remote_cache=https://cache.depot.dev" >> ~/.bazelrc && \ + echo "build --remote_header=authorization=${DEPOT_TOKEN}" >> ~/.bazelrc && \ + bazel build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Bazel in Bake files + +When using Bake files to build Docker images containing Bazel workspaces, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to mount the secret and configure Bazel: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Create .bazelrc with cache configuration +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + echo "build --remote_cache=https://cache.depot.dev" >> ~/.bazelrc && \ + echo "build --remote_header=authorization=${DEPOT_TOKEN}" >> ~/.bazelrc && \ + bazel build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/gocache.mdx b/content/cache/reference/gocache.mdx index eaed62e..46bebdd 100644 --- a/content/cache/reference/gocache.mdx +++ b/content/cache/reference/gocache.mdx @@ -6,15 +6,58 @@ description: Learn how to use Depot remote caching for Go ## Configuring Go to use Depot Cache -Depot Cache can be used with Go from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Go from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Go - each runner is launched with the `GOCACHEPROG` environment variable pre-populated with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure `GOCACHEPROG` to use Depot Cache as described below. +If you don't want Depot to set up the `GOCACHEPROG` environment variable on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure `GOCACHEPROG` to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Go in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Go projects, your build needs access to Go's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Go cache credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +3. Update your Dockerfile to install the Depot CLI and configure Go cache: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Install Depot CLI +RUN curl -L https://depot.dev/install-cli.sh | sh + +# Mount secret and set GOCACHEPROG +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + PATH="/root/.depot/bin:$PATH" \ + GOCACHEPROG="depot gocache" \ + go build -v ./ +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Go to use Depot Cache, set the `GOCACHEPROG` in your environment: @@ -42,33 +85,64 @@ To set verbose output, add the --verbose option: export GOCACHEPROG='depot gocache --verbose' ``` -Once Go is configured to use Depot Cache, you can then run your builds as you normally would. Go will automatically communicate with `GOCACHEPROG` to fetch from Depot Cache and reuse any stored build artifacts from your previous builds. +After Go is configured to use Depot Cache, you can then run your builds as you normally would. Go will automatically communicate with `GOCACHEPROG` to fetch from Depot Cache and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Go in `depot/build-push-action` +### Using Depot Cache with Go in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Go projects, your build needs access to Go's remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Update your Dockerfile to install the Depot CLI and configure Go cache: -Follow these steps to securely pass your Go cache credentials into your Docker build: +```dockerfile +# syntax=docker/dockerfile:1 -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +# ... other Dockerfile instructions -2. Configure your GitHub Action to pass secrets to the container build: +# Install Depot CLI +RUN curl -L https://depot.dev/install-cli.sh | sh -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +# Mount secret and set GOCACHEPROG +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + PATH="/root/.depot/bin:$PATH" \ + GOCACHEPROG="depot gocache" \ + go build -v ./ ``` -3. Update your Dockerfile to install the Depot CLI and configure Go cache: +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Go in Bake files + +When using Bake files to build Docker images containing Go projects, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to install the Depot CLI and configure Go cache: ```dockerfile # syntax=docker/dockerfile:1 @@ -86,3 +160,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/gradle.mdx b/content/cache/reference/gradle.mdx index 591d092..91ea9c4 100644 --- a/content/cache/reference/gradle.mdx +++ b/content/cache/reference/gradle.mdx @@ -10,7 +10,7 @@ description: Learn how to use Depot remote caching for Gradle builds ## Configuring Gradle to use Depot Cache -Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners @@ -20,11 +20,78 @@ Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, org.gradle.caching=true ``` -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Gradle to use Depot Cache as described below. +If you don't want Depot to override the `init.gradle` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Gradle to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Gradle in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Gradle projects, your build needs access to Gradle's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Gradle credentials into your Docker build: + +1. Verify that caching is enabled in your `gradle.properties` file: -To manually configure Gradle to use Depot Cache, you will need to configure remote caching in your `settings.gradle` file. Configure Gradle to use the Depot Cache service endpoints and set your API token as the `password` credential: +```properties +org.gradle.caching=true +``` + +2. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +3. Update your `settings.gradle` to read the Depot token from an environment variable: + +```groovy +buildCache { + remote(HttpBuildCache) { + url = 'https://cache.depot.dev' + enabled = true + push = true + credentials { + username = '' + password = System.getenv('DEPOT_TOKEN') + } + } +} +``` + +4. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +5. Update your Dockerfile to mount the secret and run the build: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + ./gradlew build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure Gradle to use Depot Cache, you will need to configure remote caching in your `settings.gradle` file. First, verify that caching is enabled in your `gradle.properties` file: + +```properties +org.gradle.caching=true +``` + +Then, configure Gradle to use the Depot Cache service endpoints and set your API token as the `password` credential: `settings.gradle`: @@ -58,17 +125,17 @@ buildCache { } ``` -Once Gradle is configured to use Depot Cache, you can then run your builds as you normally would. Gradle will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. - -### Using Depot Cache with Gradle in `depot/build-push-action` +After Gradle is configured to use Depot Cache, you can then run your builds as you normally would. Gradle will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -When using `depot/build-push-action` to build Docker images that contain Gradle projects, your build needs access to Gradle's remote cache credentials to benefit from caching. +### Using Depot Cache with Gradle in Depot CLI -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +When building directly with Depot CLI, follow these steps: -Follow these steps to securely pass your Gradle credentials into your Docker build: +1. Verify that caching is enabled in your `gradle.properties` file: -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +```properties +org.gradle.caching=true +``` 2. Update your `settings.gradle` to read the Depot token from an environment variable: @@ -86,31 +153,92 @@ buildCache { } ``` -3. Configure your GitHub Action to pass secrets to the container build: +3. Update your Dockerfile to copy Gradle configuration files and mount the secret: -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + ./gradlew build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . ``` -4. Update your Dockerfile to mount the secret and run the build: +### Using Depot Cache with Gradle in Bake files + +When using Bake files to build Docker images containing Gradle projects, you can pass secrets through the `target.secret` attribute: + +1. Verify that caching is enabled in your `gradle.properties` file: + +```properties +org.gradle.caching=true +``` + +2. Update your `settings.gradle` to read the Depot token from an environment variable: + +```groovy +buildCache { + remote(HttpBuildCache) { + url = 'https://cache.depot.dev' + enabled = true + push = true + credentials { + username = '' + password = System.getenv('DEPOT_TOKEN') + } + } +} +``` + +3. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +4. Update your Dockerfile to copy Gradle configuration files and mount the secret: ```dockerfile # syntax=docker/dockerfile:1 # ... other Dockerfile instructions -# Copy settings.gradle and run build with mounted secret -COPY settings.gradle . +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ./gradlew build ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +5. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/maven.mdx b/content/cache/reference/maven.mdx index 9ed734e..2fdb369 100644 --- a/content/cache/reference/maven.mdx +++ b/content/cache/reference/maven.mdx @@ -10,7 +10,7 @@ description: Learn how to use Depot remote caching for Maven builds ## Configuring Maven to use Depot Cache -Depot Cache can be used with Maven from Depot's managed GitHub Actions runners, your local machine, or any CI/CD system. +Depot Cache can be used with Maven from Depot's managed GitHub Actions runners, your local machine, any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners @@ -48,9 +48,92 @@ You should also verify that you have registered the Build Cache extension in you ``` -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Maven to use Depot Cache, as described below. +If you don't want Depot to override the Maven configuration files on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Maven to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Maven in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Maven projects, your build needs access to Maven's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Maven credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: + +```xml + + + true + SHA-256 + true + + https://cache.depot.dev + + + + +``` + +3. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: + +```xml + + + + + depot-cache + + + + Authorization + Bearer ${env.DEPOT_TOKEN} + + + + + + +``` + +4. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +5. Update your Dockerfile to copy configuration files and run the build with mounted secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Maven configuration files +COPY .mvn/maven-build-cache-config.xml .mvn/maven-build-cache-config.xml +COPY .m2/settings.xml /root/.m2/settings.xml + +# Run build with mounted secret +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + mvn clean install +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Maven to use Depot Cache, you will need to configure remote caching in your `~/.m2/settings.xml` file. Configure Maven to use the Depot Cache service endpoints and set your API token where there is the `DEPOT_TOKEN` below: @@ -77,19 +160,88 @@ To manually configure Maven to use Depot Cache, you will need to configure remot **Note: Maven support currently only supports Depot Organization API tokens, not user tokens.** -Once Maven is configured to use Depot Cache, you can run your builds as usual. Maven will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +After Maven is configured to use Depot Cache, you can run your builds as usual. Maven will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Maven in `depot/build-push-action` +### Using Depot Cache with Maven in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Maven projects, your build needs access to Maven's remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: -Follow these steps to securely pass your Maven credentials into your Docker build: +```xml + + + true + SHA-256 + true + + https://cache.depot.dev + + + + +``` -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +2. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: -2. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: +```xml + + + + + depot-cache + + + + Authorization + Bearer ${env.DEPOT_TOKEN} + + + + + + +``` + +3. Update your Dockerfile to copy configuration files and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Maven configuration files +COPY .mvn/maven-build-cache-config.xml .mvn/maven-build-cache-config.xml +COPY .m2/settings.xml /root/.m2/settings.xml + +# Run build with mounted secret +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + mvn clean install +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Maven in Bake files + +When using Bake files to build Docker images containing Maven projects, you can pass secrets through the `target.secret` attribute: + +1. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: ```xml ``` -3. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: +2. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: ```xml @@ -130,21 +282,23 @@ Follow these steps to securely pass your Maven credentials into your Docker buil ``` -4. Configure your GitHub Action to pass secrets to the container build: - -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +3. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} ``` -5. Update your Dockerfile to copy configuration files and run the build with mounted secret: +4. Update your Dockerfile to copy configuration files and run the build with mounted secret: ```dockerfile # syntax=docker/dockerfile:1 @@ -161,3 +315,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +5. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/moonrepo.mdx b/content/cache/reference/moonrepo.mdx index e13f150..f2bab5e 100644 --- a/content/cache/reference/moonrepo.mdx +++ b/content/cache/reference/moonrepo.mdx @@ -10,31 +10,13 @@ description: Learn how to use Depot remote caching for moonrepo builds ## Configuring moonrepo to use Depot Cache -Depot Cache can be used with moonrepo from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with moonrepo from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. -To configure `moon` to use Depot Cache, you will need to set a `DEPOT_TOKEN` environment variable with an organization or user token and add the following to your `.moon/workspace.yml` file: +### From Depot-managed Actions runners -```yaml -unstable_remote: - host: 'grpcs://cache.depot.dev' - auth: - token: 'DEPOT_TOKEN' -``` +[Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with moonrepo - each runner is launched with the necessary environment variables for accessing Depot Cache. -If you are using a user token and are a member of more than one organization, you will additionally need to set an `X-Depot-Org` header to your Depot organization ID in `.moon/workspace.yml`: - -```yaml -unstable_remote: - host: 'grpcs://cache.depot.dev' - auth: - token: 'DEPOT_TOKEN' - headers: - 'X-Depot-Org': '' -``` - -See [moonrepo's remote cache documentation](https://moonrepo.dev/docs/guides/remote-cache#cloud-hosted-depot) for more details. - -Once moonrepo is configured to use Depot Cache, you can then run your builds as you normally would. moonrepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to override the moonrepo workspace configuration on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure moonrepo to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with moonrepo in `depot/build-push-action` @@ -85,3 +67,123 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure `moon` to use Depot Cache, you will need to set a `DEPOT_TOKEN` environment variable with an organization or user token and add the following to your `.moon/workspace.yml` file: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +If you are using a user token and are a member of more than one organization, you will additionally need to set an `X-Depot-Org` header to your Depot organization ID in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' + headers: + 'X-Depot-Org': '' +``` + +See [moonrepo's remote cache documentation](https://moonrepo.dev/docs/guides/remote-cache#cloud-hosted-depot) for more details. + +After moonrepo is configured to use Depot Cache, you can then run your builds as you normally would. moonrepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with moonrepo in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Configure moonrepo to read the Depot token from an environment variable in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +2. Update your Dockerfile to copy the workspace configuration and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy moonrepo workspace configuration +COPY .moon/workspace.yml .moon/workspace.yml + +# Mount secret as environment variable and run build +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + moon run build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with moonrepo in Bake files + +When using Bake files to build Docker images containing moonrepo workspaces, you can pass secrets through the `target.secret` attribute: + +1. Configure moonrepo to read the Depot token from an environment variable in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +2. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +3. Update your Dockerfile to copy the workspace configuration and run the build with mounted secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy moonrepo workspace configuration +COPY .moon/workspace.yml .moon/workspace.yml + +# Mount secret as environment variable and run build +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + moon run build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/pants.mdx b/content/cache/reference/pants.mdx index 755a0c7..8efc918 100644 --- a/content/cache/reference/pants.mdx +++ b/content/cache/reference/pants.mdx @@ -10,15 +10,66 @@ description: Learn how to use Depot remote caching for Pants builds ## Configuring Pants to use Depot Cache -Depot Cache can be used with Pants from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Pants from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Pants - each runner is launched with a `pants.toml` file that is pre-configured with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Pants to use Depot Cache as described below. +If you don't want Depot to override the `pants.toml` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Pants to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Pants in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Pants projects, your build needs access to Pants' remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Pants credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Update your `pants.toml` to read the Depot token from an environment variable: + +```toml +[GLOBAL] +remote_cache_read = true +remote_cache_write = true +remote_store_address = "grpcs://cache.depot.dev" + +[GLOBAL.remote_store_headers] +Authorization = "%(env.DEPOT_TOKEN)s" +``` + +3. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +4. Update your Dockerfile to mount the secret and run the build: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy pants.toml and run build with mounted secret +COPY pants.toml . +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + pants package :: +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Pants to use Depot Cache, you will need to enable remote caching in your `pants.toml`. Configure Pants to use the Depot Cache service endpoints and set your API token in the `Authorization` header: @@ -41,19 +92,56 @@ If you are a member of multiple organizations, and you are authenticating with a remote_store_headers = { "x-depot-org" = "DEPOT_ORG_ID" } ``` -Once Pants is configured to use Depot Cache, you can then run your builds as you normally would. Pants will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +After Pants is configured to use Depot Cache, you can then run your builds as you normally would. Pants will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Pants in `depot/build-push-action` +### Using Depot Cache with Pants in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Pants projects, your build needs access to Pants' remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Update your `pants.toml` to read the Depot token from an environment variable: -Follow these steps to securely pass your Pants credentials into your Docker build: +```toml +[GLOBAL] +remote_cache_read = true +remote_cache_write = true +remote_store_address = "grpcs://cache.depot.dev" -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +[GLOBAL.remote_store_headers] +Authorization = "%(env.DEPOT_TOKEN)s" +``` -2. Update your `pants.toml` to read the Depot token from an environment variable: +2. Update your Dockerfile to copy the configuration and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy pants.toml and run build with mounted secret +COPY pants.toml . +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + pants package :: +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Pants in Bake files + +When using Bake files to build Docker images containing Pants projects, you can pass secrets through the `target.secret` attribute: + +1. Update your `pants.toml` to read the Depot token from an environment variable: ```toml [GLOBAL] @@ -65,21 +153,23 @@ remote_store_address = "grpcs://cache.depot.dev" Authorization = "%(env.DEPOT_TOKEN)s" ``` -3. Configure your GitHub Action to pass secrets to the container build: - -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +2. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} ``` -4. Update your Dockerfile to mount the secret and run the build: +3. Update your Dockerfile to copy the configuration and mount the secret: ```dockerfile # syntax=docker/dockerfile:1 @@ -93,3 +183,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/sccache.mdx b/content/cache/reference/sccache.mdx index e9fd0ea..f4e40bb 100644 --- a/content/cache/reference/sccache.mdx +++ b/content/cache/reference/sccache.mdx @@ -10,32 +10,13 @@ description: Learn how to use Depot remote caching for sccache builds ## Configuring sccache to use Depot Cache -Depot Cache can be used with sccache from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with sccache from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with sccache - each runner is launched with a `SCCACHE_WEBDAV_ENDPOINT` environment variable and is pre-configured with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure sccache to use Depot Cache as described below. - -### From your local machine or any CI/CD system - -To manually configure sccache to use Depot Cache, you will need to set two environment variables in your environment, representing the Depot Cache service endpoint and your API token: - -```shell -export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev -export SCCACHE_WEBDAV_TOKEN=DEPOT_TOKEN -``` - -If you are a member of multiple organizations, and you are authenticating with a user token, you must instead specify a password along with which organization should be used for cache storage as follows: - -```shell -export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev -export SCCACHE_WEBDAV_USERNAME=DEPOT_ORG_ID -export SCCACHE_WEBDAV_PASSWORD=DEPOT_TOKEN -``` - -Once sccache is configured to use Depot Cache, you can then run your builds as you normally would. sccache will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to set up the `SCCACHE_WEBDAV_ENDPOINT` environment variable on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure sccache to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with sccache in `depot/build-push-action` @@ -75,3 +56,94 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure sccache to use Depot Cache, you will need to set two environment variables in your environment, representing the Depot Cache service endpoint and your API token: + +```shell +export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev +export SCCACHE_WEBDAV_TOKEN=DEPOT_TOKEN +``` + +If you are a member of multiple organizations, and you are authenticating with a user token, you must instead specify a password along with which organization should be used for cache storage as follows: + +```shell +export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev +export SCCACHE_WEBDAV_USERNAME=DEPOT_ORG_ID +export SCCACHE_WEBDAV_PASSWORD=DEPOT_TOKEN +``` + +After sccache is configured to use Depot Cache, you can then run your builds as you normally would. sccache will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with sccache in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to mount the secret as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ + SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev sccache --start-server && \ + cargo build --release +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with sccache in Bake files + +When using Bake files to build Docker images containing Rust projects with sccache, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to mount the secret as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ + SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev sccache --start-server && \ + cargo build --release +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/turbo.mdx b/content/cache/reference/turbo.mdx index 4256621..61ab896 100644 --- a/content/cache/reference/turbo.mdx +++ b/content/cache/reference/turbo.mdx @@ -10,25 +10,13 @@ description: Learn how to use Depot remote caching for Turborepo builds ## Configuring Turborepo to use Depot Cache -Depot Cache can be used with Turborepo from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Turborepo from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Turborepo - each runner is launched with a `TURBO_API` environment variable and is pre-configured with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Turborepo to use Depot Cache as described below. - -### From your local machine or any CI/CD system - -To manually configure Turborepo to use Depot Cache, you will need to set three environment variables in your environment. These represent the Depot Cache service endpoint, your API token, and your Depot organization id: - -```shell -export TURBO_API=https://cache.depot.dev -export TURBO_TOKEN=DEPOT_TOKEN -export TURBO_TEAM=DEPOT_ORG_ID -``` - -Once Turborepo is configured to use Depot Cache, you can then run your builds as you normally would. Turborepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to set up the Turborepo environment variables on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Turborepo to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with Turborepo in `depot/build-push-action` @@ -74,3 +62,97 @@ RUN --mount=type=secret,id=TURBO_API,env=TURBO_API \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure Turborepo to use Depot Cache, you will need to set three environment variables in your environment. These represent the Depot Cache service endpoint, your API token, and your Depot organization id: + +```shell +export TURBO_API=https://cache.depot.dev +export TURBO_TOKEN=DEPOT_TOKEN +export TURBO_TEAM=DEPOT_ORG_ID +``` + +After Turborepo is configured to use Depot Cache, you can then run your builds as you normally would. Turborepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with Turborepo in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to mount the secrets as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=TURBO_API,env=TURBO_API \ + --mount=type=secret,id=TURBO_TOKEN,env=TURBO_TOKEN \ + --mount=type=secret,id=TURBO_TEAM,env=TURBO_TEAM \ + turbo build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=TURBO_API,env=TURBO_API --secret id=TURBO_TOKEN,env=TURBO_TOKEN --secret id=TURBO_TEAM,env=TURBO_TEAM -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=TURBO_API,env=TURBO_API --secret id=TURBO_TOKEN,env=TURBO_TOKEN --secret id=TURBO_TEAM,env=TURBO_TEAM -t your-image:tag . +``` + +### Using Depot Cache with Turborepo in Bake files + +When using Bake files to build Docker images containing Turborepo workspaces, you can pass secrets through the `target.secret` attribute: + +1. Define the secrets in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "TURBO_API" + }, + { + type = "env" + id = "TURBO_TOKEN" + }, + { + type = "env" + id = "TURBO_TEAM" + } + ] +} +``` + +2. Update your Dockerfile to mount the secrets as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=TURBO_API,env=TURBO_API \ + --mount=type=secret,id=TURBO_TOKEN,env=TURBO_TOKEN \ + --mount=type=secret,id=TURBO_TEAM,env=TURBO_TEAM \ + turbo build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +TURBO_API=https://cache.depot.dev TURBO_TOKEN=your_token TURBO_TEAM=your_org_id depot bake +``` diff --git a/content/container-builds/how-to-guides/docker-bake.mdx b/content/container-builds/how-to-guides/docker-bake.mdx index 39c5bdd..be639fe 100644 --- a/content/container-builds/how-to-guides/docker-bake.mdx +++ b/content/container-builds/how-to-guides/docker-bake.mdx @@ -247,6 +247,8 @@ target "worker" { } ``` +**Note:** When you use the `depot/bake-action` in a GitHub Actions workflow, the `x-depot.project-id` in your Docker Compose file or `project_id` in your HCL bake file take precedence over the `project` input in the action configuration. + ### Loading images locally Load specific targets to your local Docker daemon by including the target name after the load flag: diff --git a/content/container-builds/how-to-guides/docker-compose.mdx b/content/container-builds/how-to-guides/docker-compose.mdx index ffe1ab1..cada566 100644 --- a/content/container-builds/how-to-guides/docker-compose.mdx +++ b/content/container-builds/how-to-guides/docker-compose.mdx @@ -66,6 +66,8 @@ services: With the above configuration, the `app` service will be built in the `abc123456` Depot project and the `backend` service will be built in the `xyz123456` Depot project when running `depot bake`. +**Note:** When you use the `depot/bake-action` in a GitHub Actions workflow, the `x-depot.project-id` in your Docker Compose file takes precedence over the `project` input in the action configuration. + ## Building images with `docker compose build` If you are unable to use `depot bake --load` and need to use `docker compose build` directly, you can still use Depot to accelerate your builds. Docker Compose can use Docker Buildx to build the requested images in the `docker-compose.yml` file, and Depot can be installed as a Buildx driver serve those build requests. diff --git a/content/overview/index.mdx b/content/overview/index.mdx index 147662b..e708730 100644 --- a/content/overview/index.mdx +++ b/content/overview/index.mdx @@ -8,6 +8,7 @@ hideToc: true import {DocsCard, DocsCardGrid} from '~/components/docs/DocsCard' import {DocsCTA, DocsCTASecondary} from '~/components/blog/CTA' import {TrackedLink} from '~/components/TrackedLink' +import {CodeIcon, CpuIcon, DatabaseIcon, GitHubLogoIcon, RobotIcon, ShippingContainerIcon} from '~/components/icons' Depot accelerates your most important development workflows. @@ -19,6 +20,7 @@ Depot accelerates your most important development workflows. } links={[ {text: 'Quickstart: Build Docker images faster', href: '/docs/container-builds/quickstart'}, {text: 'Learn more about Depot container builds', href: '/docs/container-builds/overview'}, @@ -27,16 +29,22 @@ Depot accelerates your most important development workflows. } links={[ {text: 'Quickstart: Use fast runners for your GitHub Actions', href: '/docs/github-actions/quickstart'}, {text: 'Learn more about Depot GitHub Actions runners', href: '/docs/github-actions/overview'}, ]} /> - +} + links={[{text: 'Learn more about Depot Cache', href: '/docs/cache/overview'}]} +/> } links={[ {text: 'Quickstart: Run Claude Code in a sandbox', href: '/docs/agents/claude-code/quickstart'}, {text: 'Learn more about Depot remote agent sandboxes', href: '/docs/agents/overview'}, @@ -45,6 +53,7 @@ Depot accelerates your most important development workflows. } links={[ {text: 'Quickstart: Use our container image registry', href: '/docs/registry/quickstart'}, {text: 'Learn more about the Depot container registry', href: '/docs/registry/overview'}, @@ -53,10 +62,11 @@ Depot accelerates your most important development workflows. } links={[ { text: "Reference: Access Depot's underlying architecture programmatically", - href: '/docs/container-builds/reference/api-overview', + href: '/docs/api/overview', }, ]} /> diff --git a/content/registry/quickstart.mdx b/content/registry/quickstart.mdx index 6f02e96..ca079a5 100644 --- a/content/registry/quickstart.mdx +++ b/content/registry/quickstart.mdx @@ -99,7 +99,7 @@ depot pull --project my-image To pull a build from the Depot Registry in a Kubernetes cluster, you can use the `kubectl` command to [create a secret with the Docker registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/), then create a Kubernetes deployment that uses the secret to pull the image. ```shell -kubectl create secret depot-registry regcred \ +kubectl create secret docker-registry regcred \ --docker-server=registry.depot.dev \ --docker-username=x-token \ --docker-password=