Quickstart

Follow the steps below to start syncing data from CloudQuery source plugins (opens in a new tab) to destination plugins (opens in a new tab).

Download CloudQuery CLI

You can download the precompiled binary from releases (opens in a new tab), or using CLI:

curl -L https://github.com/cloudquery/cloudquery/releases/download/cli-v6.10.0/cloudquery_linux_amd64 -o cloudquery
chmod a+x cloudquery

Sign in with CloudQuery

To sign in from the CLI, run the following command:

./cloudquery login

A new browser window will open where you will complete the sign-in process.

You will be able to create a new account when you sign in with the CLI for the first time. Alternatively, head to CloudQuery Cloud (opens in a new tab) to create a new account.

For a non-interactive sign-in process and sync, you can generate an API key. This is useful for production deployments.

Create a Sync Configuration

CloudQuery supports many sources (opens in a new tab) and destinations (opens in a new tab).

In this guide we'll create a configuration to sync from the AWS source (opens in a new tab) to the SQLite destination (opens in a new tab) using the init command.

Run the following command to create a configuration file:

./cloudquery init --source aws --destination sqlite --spec-path aws_to_sqlite.yaml

To pick from a list of available sources and destinations, run the init command without any arguments.

By default, when you run the script, it will sync the aws_ec2_instances table. You can find all the available resources you can sync on the AWS source plugin page (opens in a new tab).

Authenticate to AWS

The CloudQuery AWS plugin requires read only permissions in order to sync information from your AWS cloud environment.

There are several different methods for authenticating to your AWS cloud environment (opens in a new tab). Be sure you authenticate before you run your first sync.

Start Syncing

Run the following command to start syncing:

./cloudquery sync aws_to_sqlite.yaml

You should see a spinner with number of resources synced and the time it took to sync.

To learn about the available options for the sync command, run cloudquery sync --help.

Examine the Data

Once the sync is done you can query the data via the database directly via the sqlite3 (opens in a new tab) command line tool or any other SQLite client.

Run the following command to get your oldest 10 EC2 instances:

./sqlite3 db.sql "SELECT account_id, instance_id, region, launch_time FROM aws_ec2_instances ORDER BY launch_time ASC LIMIT 10"

The output should look like this:

account_id    instance_id          region          launch_time
------------  -------------------  --------------  -------------------
012345678901  i-0jklm0123nopq4567  us-west-1       2021-06-10 07:30:00
012345678901  i-0ijkl9012mnop3456  ca-central-1    2021-07-25 12:40:00
012345678901  i-0hijk8901lmno2345  sa-east-1       2021-08-15 15:10:00
012345678901  i-0ghij7890klmn1234  eu-central-1    2021-08-30 11:50:00
012345678901  i-0fghi6789jklm0123  us-east-2       2021-09-05 17:20:00
012345678901  i-0efgh5678ijkl9012  ap-northeast-1  2021-10-25 08:00:00
012345678901  i-0defg4567hijk8901  ap-southeast-1  2021-11-10 13:15:00
012345678901  i-0cdef3456ghij7890  eu-west-1       2021-11-20 09:45:00
012345678901  i-0bcde2345fghi6789  us-west-2       2021-12-15 14:30:00
012345678901  i-0abcd1234efgh5678  us-east-1       2022-01-01 10:00:00

Next Steps

Now that you have successfully synced data from the AWS source (opens in a new tab) to the SQLite destination (opens in a new tab), you can explore more sources and destinations:

macOS Windows