CloudQuery has a pluggable architecture, with plugins communicating over gRPC (opens in a new tab). A plugin can be implemented to be a source, destination or both.
- Source plugin - Responsible for extracting and transforming configuration from cloud-providers, SaaS apps, and other APIs (Available source plugins (opens in a new tab)).
- Destination plugin - Responsible for writing the data from the source plugins to various destinations such as databases, message queues and storage (Available destination plugins (opens in a new tab)).
All plugins are split to official (maintained by CloudQuery) and community (maintained by members of the community in their own repositories).
The core responsibilities of a source plugin:
- Define the schema (tables).
- Authenticate with the supported APIs, SaaS services and/or cloud providers.
- Extracting data from the supported APIs and transform them into the defined schema.
- Send the data via protobuf (opens in a new tab) to the CLI for further processing and storage at the defined destination plugins.
The core responsibilities of a destination plugin:
- Authenticate with the destination (such as database, message queue, storage).
- Auto-migrate the schemas defined by the source plugins.
- Save each incoming object in the appropriate table.