BigQuery is a fully-managed, serverless data warehouse that enables scalable analysis over petabytes of data. It is a Platform as a Service that supports querying using ANSI SQL.
Two connectors are available for Google BigQuery:
- Databases: Google BigQuery – Choose this connector for ETL workflows and other data warehouse needs, such as sending or retrieving data.
- Connectors: Google BigQuery (REST API) – Review the Google BigQuery APIs and libraries documentation for your requirements, and configure this connector as needed.
Set up a database connection to Google BigQuery
Google BigQuery links: Service Accounts, Authentication
A. Create a Google BigQuery database connection
Start establishing a connection to Google BigQuery in either of the following ways:
- From the Resources menu, select Connections. Then, click + Create connection at the top right.
– or – - While working in a new or existing integration, you can add an application to a flow simply by clicking Add source or Add destination.
In the resulting Application list, under Databases, click Google BigQuery.
B. Edit Google BigQuery database connection application details
On the Create Connection page, set the following options:
Name (required): Provide a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection, and a unique identifier will prove helpful later when selecting among a list of connections that you’ve created.
Application (required, non-editable): A reminder of the app you’re editing.
Default project ID: Provide the unique identifier used for the applicable BigQuery project in your Google Cloud account. To get this value you can either go to your Project list in Google Cloud Platform and look at the ID field, or read it from the service account key file.
Client email: Enter the email address for the Google Cloud service account that should be used to authorize this connection. See the Google Cloud service account documentation for help finding the client email and private key.
Private key: First, copy the private key from the Google portal for the service account that you want to use to authenticate the connection. Before you add it to integrator.io you must replace all newline characters (\n) throughout the private key:
- Paste the private key into a text editor.
- Find \n.
- With your cursor in that location, delete the \n characters and press Enter or Return.
- Repeat this for each instance of \n.
- Ensure -----BEGIN PRIVATE KEY----- appears before the key, and
-----END PRIVATE KEY----- appears after the key. For example: - Copy and paste the reformatted private key (including the begin and end declarations) into integrator.io.
Default dataset: Enter the name of the dataset containing the tables and views integrator.io should be able to access.
Tip: If you specify only a default project and dataset here (instead of a project, dataset, and table), you can use this connection for multiple exports and imports later. The project and dataset will still need to be specified in this connection, but you can update it to use any table in an export or import.
C. Edit advanced Google BigQuery database connection settings
Before continuing, you have the opportunity to provide additional configuration information, if needed, for the Google BigQuery connection.
Borrow concurrency from (optional): Select another connection from the list....
Concurrency level (optional): Up to 25 allowed....
D. Save and authorize the Google BigQuery database connection
Set up a REST API connection to Google BigQuery
Set up a connection
After you start the connection, configure it in the Create connection panel and complete all of the required * authentication settings:
| Setting | Instructions |
|---|---|
| Name your connection * | Enter a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection. A unique identifier will prove helpful later when selecting it from a list of the connections in your account. |
| OAuth 2.0 client * |
Select the OAuth 2.0 client that stores the client ID and client secret provided to you by Google. To add an OAuth 2.0 client and configure your credentials, click the plus (+) button. Click the edit ( |
|
Scopes * |
Scopes are OAuth 2.0 permissions that Google BigQuery defines to limit access to your account. |
How to retrieve client ID and client secretTip: This connector documentation describes only the settings shown for the Simple view. For the corresponding HTTP settings, see OAuth 2.0 auth universal connector documentation.
- Sign in to your Google BigQuery account.
- Navigate to Credentials > APIs & services.
- Click Configure consent screen.
- Determine whether the application will be Public or Internal. It is recommended to set the application to Internal, to be available only to your organization.
- Navigate to the OAuth consent screen page.
- Provide an application name and click Save and continue.
- Add or remove scopes and click Save and continue.
- Navigate to Create credentials > OAuth client ID.
- Click Web application from the dropdown menu and provide a unique name.
- In the Redirect URI field, copy and paste the URI mentioned below depending on your instance:
- For NA: paste
https://integrator.io/connection/oauth2callback - For EU: paste
https://eu․integrator.io/connection/oauth2callback
- For NA: paste
-
Click Create.
- Copy Client ID.
- Copy Client secret.
If so, click Advanced, and then click Go to integrator.io (unsafe) to allow the exception and continue.
Comments
Please sign in to leave a comment.