Skip to main content

Generating resources

Adding resources to a provider can sometimes be a tedious task, some resources can have more than hundreds of fields and relations, and adding them all can take a long time. To remedy this issue, the cq-gen project was created. cq-gen allows to easily generate more of the boilerplate code for resources from common specs such as go code, openapi specs, protobuf, and graphql.

Set up cq-gen in your provider project

If you haven't created a provider use this command to create a project or alternatively use our template repository as a base.

mkdir cq-my-provider
cd cq-my-provider

go mod init github.com/[username]/[cq-my-provider]

# install the `cq-gen` binary under your go path
go install github.com/cloudquery/cq-gen

Next, create a tools.go file and add cq-gen as a tool dependency for your module.

//go:build tools

package tools

import (
_ "github.com/cloudquery/cq-gen"
)

To automatically add the dependency to your go.mod run:

go mod tidy

Running cq-gen for the first time

To run cq-gen you must first create a resource hcl config. As an example we will create the AWS CloudFormation stacks

service = "aws"
output_directory = "."
add_generate = true

resource "aws" "cloudformation" "stacks" {
path = "github.com/aws/aws-sdk-go-v2/service/cloudformation/types.Stack"
ignoreError "IgnoreAccessDenied" {
path = "github.com/cloudquery/cq-provider-aws/client.IgnoreAccessDeniedServiceDisabled"
}
deleteFilter "AccountRegionFilter" {
path = "github.com/cloudquery/cq-provider-aws/client.DeleteAccountRegionFilter"
}

options {
primary_keys = ["id"]
}

userDefinedColumn "account_id" {
description = "The AWS Account ID of the resource."
type = "string"
resolver "resolveAWSAccount" {
path = "github.com/cloudquery/cq-provider-aws/client.ResolveAWSAccount"
}
}
userDefinedColumn "region" {
type = "string"
description = "The AWS Region of the resource."
resolver "resolveAWSRegion" {
path = "github.com/cloudquery/cq-provider-aws/client.ResolveAWSRegion"
}
}

}

*Note: we will call this file stacks.hcl

To execute this configuration we will run our cq-gen tool with the following command:

# cq-gen has to run from the directory of the `gen.hcl` file
cd resources/services/<service-name>
cq-gen --resource stacks --domain cloudformation --config gen.hcl

The command above will generate the following code:

package cloudformation

import (
"context"

"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/cloudformation"
"github.com/aws/aws-sdk-go-v2/service/cloudformation/types"
"github.com/cloudquery/cq-provider-aws/client"
"github.com/cloudquery/cq-provider-sdk/provider/schema"
)

//go:generate cq-gen --resource stacks --config gen.hcl --output .
func Stacks() *schema.Table {
return &schema.Table{
Name: "aws_cloudformation_stacks",
Description: "The Stack data type.",
Resolver: fetchCloudformationStacks,
Multiplex: client.ServiceAccountRegionMultiplexer("cloudformation"),
IgnoreError: client.IgnoreAccessDeniedServiceDisabled,
DeleteFilter: client.DeleteAccountRegionFilter,
Options: schema.TableCreationOptions{PrimaryKeys: []string{"id"}},
Columns: []schema.Column{
{
Name: "account_id",
Description: "The AWS Account ID of the resource.",
Type: schema.TypeString,
Resolver: client.ResolveAWSAccount,
},
{
Name: "region",
Description: "The AWS Region of the resource.",
Type: schema.TypeString,
Resolver: client.ResolveAWSRegion,
},
... # shortened for sake for readability

As you can see above the function definition, cq-gen, added a //go:generate command to help regenerate easily with go generate.

Flags

The cq-gen command is used to generate schema.Table from given source (go, protobuf, openapi, graphql, etc') It supports the following flags:

  • -output: which directory to write the resulting source code.
  • -resource: the name of the resource to generate as defined in the hcl.
  • -domain: the domain of the resource to generate as defined in the hcl.
  • -config: the path to the configuration file of cq-gen on how to generate the resource.

Configuration

For full documentation on how to use cq-gen and all the available configuration options please check the repository.