Skip to main content

Getting Started

Download and Install#

You can download the precompiled binary from releases, or using CLI:

brew install cloudquery/tap/cloudquery# After initial install you can upgrade the version via:brew upgrade cloudquery

Running#

Init Command#

The first step is to generate a config.hcl file that will describe which cloud provider you want to use and which resources you want CloudQuery to ETL:

cloudquery init aws# cloudquery init gcp aws # This will generate a config containing gcp and aws providers# cloudquery init --help # Show all possible auto generated configs and flags

All official and approved community providers are listed at CloudQuery Hub with their respective documentation.

Spawn or Connect to a Database#

CloudQuery needs a PostgreSQL database (>11). You can either spawn a local one (usually good for development and local testing) or connect to an existing one.

For local, you can use the following docker command:

docker run -p 5432:5432 -e POSTGRES_PASSWORD=pass -d postgres

If you are using an existing database, you will have to update the connection section in config.hcl:

cloudquery {  plugin_directory = "./cq/providers"  policy_directory = "./cq/policies"
  provider "aws" {    source  = ""    version = "latest"  }
  connection {    dsn = "host=localhost user=postgres password=pass database=postgres port=5432"  }}

Fetch Command#

Once config.hcl is generated, run the following command to fetch the resources. (You need to be authenticated โ€” see relevant section under each provider):

cloudquery fetch# cloudquery fetch --help # Show all possible fetch flags

Once your provider has fetched resources, you can run the following example queries.

Exploring and Running Queries#

Schema and tables for each provider are available in CloudQuery Hub

A few examples for AWS:

List ec2_images:#

SELECT * FROM aws_ec2_images;

Find all public-facing AWS load balancers:#

SELECT * FROM aws_elbv2_load_balancers WHERE scheme = 'internet-facing';

Policy Command#

CloudQuery Policies allow users to write security, governance, cost, and compliance rules, using SQL as the query layer and HCL as the logical layer.

All official and approved community policies are listed on CloudQuery Hub.

Execute a policy#

All official policies are hosted at https://github.com/cloudquery-policies.

cloudquery policy run aws-cis-1.2.0