BigQuery is a fully managed, serverless data warehouse that enables scalable analysis over petabytes of data. It is a Platform as a Service that supports querying using ANSI SQL.
Two connectors are available for Google BigQuery:
-
Databases: Google BigQuery – Choose this connector for ETL workflows and other data warehouse needs, such as sending or retrieving data.
-
Connectors: Google BigQuery (REST API) – Review the Google BigQuery APIs and libraries documentation for your requirements, and configure this connector as needed.
You'll need to enable the following permissions to use the full functionality of the BigQuery connector. These permissions need to be enabled in the BigQuery account that will be used in your connection:
-
bigquery.jobs.create(required) -
bigquery.datasets.get -
bigquery.routines.get -
bigquery.routines.list -
bigquery.tables.get -
bigquery.tables.list
Google BigQuery links: Service Accounts , Authentication
Start establishing a connection to Google BigQuery in either of the following ways:
-
From the Resources menu, select Connections. Then, click + Create connection at the top right.
– or –
-
While working in a new or existing integration, you can add an application to a flow simply by clicking Add source or Add destination.
In the resulting Application list, under Databases, click Google BigQuery.
On the Create Connection page, set the following options:
Name (required): Provide a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection, and a unique identifier will prove helpful later when selecting among a list of connections that you’ve created.
Application (required, non-editable): A reminder of the app you’re editing.
Default project ID: Provide the unique identifier used for the applicable BigQuery project in your Google Cloud account. To get this value you can either go to your Project list in Google Cloud Platform and look at the ID field, or read it from the service account key file.
Client email: Enter the email address for the Google Cloud service account that should be used to authorize this connection. See the Google Cloud service account documentation for help finding the client email and private key.
Private key: First, copy the private key from the Google portal for the service account that you want to use to authenticate the connection. Before you add it to integrator.io you must replace all newline characters (\n) throughout the private key:
-
Paste the private key into a text editor.
-
Find \n.
-
With your cursor in that location, delete the \n characters and press Enter or Return.
-
Repeat this for each instance of \n.
-
Ensure -----BEGIN PRIVATE KEY----- appears before the key, and
-----END PRIVATE KEY----- appears after the key. For example:
-
Copy and paste the reformatted private key (including the begin and end declarations) into integrator.io.
Default dataset: Enter the name of the dataset containing the tables and views integrator.io should be able to access.
Tip
If you specify only a default project and dataset here (instead of a project, dataset, and table), you can use this connection for multiple exports and imports later. The project and dataset will still need to be specified in this connection, but you can update it to use any table in an export or import.
Before continuing, you have the opportunity to provide additional configuration information, if needed, for the Google BigQuery connection.
Once you have configured the Google BigQuery database connection, you have two options for continuing:
-
Save & authorize – click this button to test the connection, commit the new connection so that it will be available to all integrations for your account (and applied to the current source or destination app, if you created it within a flow)
-
Cancel – click to exit without saving any new changes
After you start the connection , configure it in the Create connection panel and complete all of the required * authentication settings:
|
Setting |
Instructions |
|---|---|
|
Name your connection * |
Enter a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection. A unique identifier will prove helpful later when selecting it from a list of the connections in your account. |
|
OAuth 2.0 client * |
Select the OAuth 2.0 client that stores the client ID and client secret provided to you by Google. To add an OAuth 2.0 client and configure your credentials, click the plus ( + ) button. Click the edit (
|
|
Scopes * |
Scopes are OAuth 2.0 permissions that Google BigQuery defines to limit access to your account.
|
Tip
This connector documentation describes only the settings shown for the Simple view. For the corresponding HTTP settings, see OAuth 2.0 auth universal connector documentation.
-
Sign in to your Google BigQuery account.
-
Navigate to Credentials > APIs & services.
-
Click Configure consent screen.
-
Determine whether the application will be Public or Internal. It is recommended to set the application to Internal , to be available only to your organization.
-
Navigate to the OAuth consent screen page.
-
Provide an application name and click Save and continue .
-
Add or remove scopes and click Save and continue.
-
Navigate to Create credentials > OAuth client ID.
-
Click Web application from the dropdown menu and provide a unique name.
-
In the Redirect URI field, copy and paste the URI mentioned below depending on your instance:
-
For NA: paste
https://integrator.io/connection/oauth2callback -
For EU: paste
https://eu․integrator.io/connection/oauth2callback
-
-
Click Create.
-
Copy Client ID.
-
Copy Client secret.
Warning
Depending on your browser history and the permissions you’ve granted, you may see the following screen when attempting to log in:
If so, click Advanced , then Go to integrator.io (unsafe) to allow the exception and continue.