BigQuery is a fully-managed, serverless data warehouse that enables scalable analysis over petabytes of data. It is a Platform as a Service that supports querying using ANSI SQL.
Two connectors are available for Google BigQuery:
- Databases: Google BigQuery – Use this connector for ETL workflows and other data warehouse needs, such as sending or retrieving data.
- Connectors: Google BigQuery (REST API) – Review the Google BigQuery APIs and libraries documentation for your use case, and configure this connector as needed.
Contents
Set up a database connection to Google BigQuery
Rate this feature
Included in the May 2022 release Google BigQuery links: Service Accounts, Authentication
A. Create a Google BigQuery database connection
Start establishing a connection to Google BigQuery in either of the following ways:
- From the Resources menu, select Connections. Then, click + Create connection at the top right.
–or– - While working in a new or existing integration, you can add an application to a flow simply by clicking Add source or Add destination.
In the resulting Application list, under Databases, click Google BigQuery.
B. Edit Google BigQuery database connection application details
On the Create Connection page, set the following options:
Name (required): Provide a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection, and a unique identifier will prove helpful later when selecting among a list of connections that you’ve created.
Application (required, non-editable): A reminder of the app you’re editing.
Project: Provide the unique identifier used for the applicable BigQuery project in your Google Cloud account. To get this value you can either go to your Project list in Google Cloud Platform and look at the ID field, or read it from the service account key file.
Client email: Enter the email address for the Google Cloud service account that should be used to authorize this connection. See the Google Cloud service account documentation for help finding the Client email and Private key.
Private key: First, copy the private key from the Google portal for the service account that you want to use to authenticate the connection. Before you add it to integrator.io you must replace all newline characters (\n) throughout the private key:
- Paste the private key into a text editor.
- Find \n.
- With your cursor in that location, delete the \n characters and press Enter or Return.
- Repeat this for each instance of \n.
- Ensure -----BEGIN PRIVATE KEY----- appears before the key, and
-----END PRIVATE KEY----- appears after the key. For example:
- Copy and paste the reformatted private key (including the begin and end declarations) into integrator.io.
Dataset: Enter the name of the dataset containing the tables and views integrator.io should be able to access.
C. Edit advanced Google BigQuery database connection settings
Before continuing, you have the opportunity to provide additional configuration information, if needed, for the Google BigQuery connection.
Borrow concurrency from (optional): Select another connection from the list....
Concurrency level (optional): Up to 25 allowed....
D. Save and authorize the Google BigQuery database connection
Set up a REST API connection to Google BigQuery
Google BigQuery links: API guide, Authentication
A. Set up a Google BigQuery (REST API) connection
Start establishing a connection to Google BigQuery in either of the following ways:
- From the Resources menu, select Connections. Then, click + Create connection at the top right.
– or – - While working in a new or existing integration, you can add an application to a flow simply by clicking Add source or Add destination.
In the resulting Application list, click Google BigQuery (REST API).
The Create connection pane opens with required and advanced settings.
B. Edit Google BigQuery (REST API) application details
At this point, you’re presented with a series of options for providing Google BigQuery (REST API) authentication.
Name (required): Provide a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection, and a unique identifier will prove helpful later when selecting among a list of connections that you’ve created.
Application (required, non-editable): A reminder of the app you’re editing.
Configure scopes (required): Scopes are permissions that the Google BigQuery API defines to limit access to your account.
- Click the Configure scopes button.
- To access all of the scopes in the API, click the » button.
– or –
Check the scopes you want to add and click the > button. - Click Save when you’re ready to leave the editor and finish setting up your connection.
Redirect (callback) URL (read-only): Use the callback URL to exchange secure messages with the authorization server after authentication. You must whitelist this URL with your authorization server.
Configure your client id and secret (required): Check this box to reveal iClient options.
iClient (required): Select the iClient pair that stores the client ID and client secret provided to you by Google BigQuery. To add an iClient and configure your credentials, click the plus (+) button. Click the edit ( 
) button to modify a selected iClient. Be sure to give the iClient a recognizable name for use in any other connections.
Sign in to your Google BigQuery account at console.cloud.google.com/bigquery, and select Credentials from the APIs & services menu.
Click Configure consent screen.
Determine whether the application will be Public or Internal. We recommend setting the application to Internal, so it's only available to your organization.
Proceed to the OAuth consent screen page. Provide an application name and add integrator.io to the list of Authorized domains. Click Save and continue.
Add or remove the scopes you selected in integrator.io, and select Save and continue.
To create an OAuth client ID, click Create credentials → OAuth client ID at the top.
Click Web application from the dropdown menu and provide a unique name. Paste the Callback URL provided in integrator.io into the application’s Authorized redirect URIs and click Create.
Your Client Id and secret are available after you click Create.
Copy and paste your Client Id and secret into integrator.io to complete the connection.
Important: Google BigQuery has standard rate quotas and limits for tables.
C. Edit advanced Google BigQuery (REST API) settings
Before continuing, you have the opportunity to provide additional configuration information, if needed, for the Google BigQuery (REST API) connection.
Borrow concurrency from (optional): Select another connection from the list....
Concurrency level (optional): Up to 25 allowed....
D. Save and authorize the Google BigQuery (REST API) connection
Once you have configured the Google BigQuery (REST API) connection, you have two options for continuing:
- Save & authorize – click this button to test the connection, commit the new connection so that it will be available to all integrations for your account (and applied to the current source or destination app, if you created it within a flow)
- Cancel – click to exit without saving any new changes
When you select Save & authorize, you are presented with a Google BigQuery form in a new browser window.
After signing in, review the permissions for the integration and click Allow.
The browser window closes, and the new connection is now successfully added to your account. It will be applied to the current source or destination app, if you created it within a flow. Otherwise, you may proceed to register the connection with an integration.
Comments
0 comments
Please sign in to leave a comment.