Configure dbt core collector with Elementary for BigQuery

in this tutorial, we'll guide you through the essential steps to effectively set up and configure the dbt collector with elementary data for bigquery this involves establishing connections to the bigquery dataset used by elementary to log the dbt artifacts, and configuring the collector, to ensure you can monitor, collect, and analyze the operations and impacts of your dbt models you will need to go through 2 steps 1 set up bigquery connection to the elementary table (estimated task time 3 minutes) 2 configuration of the dbt core collector (estimated task time 1 minute) prerequisites before you begin configuring the collector for dbt core collector with elementary for bigquery , ensure you have the following a google cloud service account (for instance a json keyfile) with specific roles granted the necessary read permissions to access elementary artifact table in the bigquery project the minimum roles needed are bigquery data viewer bigquery job user configuration setting up the connection the first step consist in creating a google cloud connection with the account credentials mentioned in the prerequistes 1\ navigate to your kensu hub url https //xxxxx hub kensuapp com/ https //demo hub kensuapp com/kensu hub components plugin/#/connections 2\ within the data applications collectors, locate the dbt core one and click on configure 3\ under the dependencies section, click on the configure button in the dbt core elementary bigquery cell 4\ look for the + icon clicking on this icon will open a form where you can enter the details for your new bigquery connection 5\ now, you'll need to fill out the form with specific information about your bigquery account here's what you'll need conn id a unique id for your connection keyfile json the credentials json to connect the service account project id your google cloud project containing the elementary dataset 6\ after double checking the information you've entered, click on the submit button to finalize the connection kensu hub will process your details and establish a connection to your google cloud account configure the collector 1\ come back to the collector page 2\ navigate to the configuration panel within the kensu hub components collector dbt core cell, click on configure 3\ click on the + icon this action will generate a new configuration file 4\ you can now edit the configuration, by adding in the specified section, choose the appropriate kensu and dbt core elementary bigquery connections for this configuration you can select the default kensu connection or create your own do not select any other connection you can identify the collector by adding a specific identifier in the id field schema must refer to the google bigquery dataset id which contains the elementary data all the other fields can be left blank 5\ click on submit all congratulations! you've successfully set up the collector it will be activated shortly, and you will begin monitoring your dbt models