Docker - Offline Installation

You can run CloudQuery in a container with integrations pre-installed. This is useful for isolated deployments where you don't want to download integrations from the internet.

Downloading integrations requires users to be authenticated, normally this means running cloudquery login but that is not doable in a CI environment or inside of a docker build process. The recommended way to handle this is to use an API key. More information on generating an API Key can be found here

To download the integrations based on your configuration file, use the cloudquery plugin install command. Below is an example Dockerfile based on the CloudQuery container. It uses a build.spec.yaml with the minimum configuration required to download the integrations.

# build.spec.yaml
kind: source
spec:
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v30.1.0"
  tables: ["aws_ec2_instances"]
  destinations: ["postgresql"]
---
kind: destination
spec:
  name: "postgresql"
  path: "cloudquery/postgresql"
  registry: "cloudquery"
  version: "v8.7.5"
  spec:

# Dockerfile
FROM ghcr.io/cloudquery/cloudquery:latest as build
WORKDIR /app
COPY ./build.spec.yaml /app/build.spec.yaml
ARG CLOUDQUERY_API_KEY

RUN /app/cloudquery plugin install build.spec.yaml

FROM ghcr.io/cloudquery/cloudquery:latest

WORKDIR /app

# Copy the .cq directory which contains the integrations
COPY --from=build /app/.cq /app/.cq

Build this container as you would normally do:

docker build --build-arg CLOUDQUERY_API_KEY=<your-api-key> ./ -t my-cq-container:latest

Run the Container

Run the container as you would run the default CloudQuery container. Here is an example:

docker run \
  # you can mount a different config file that uses the same integrations as in the build.spec
  -v <ABSOLUTE_PATH_TO_CONFIG_FILE>:/config.yml \
  # set any env variable with -e <ENV_VAR_NAME>=<ENV_VAR_VALUE>
  my-cq-container:latest \
  sync /config.yml

Related docs

Read more about the plugin install command in the CLI Documentation.

Docker Generate an API Key