Set up an OAuth 2.0 HTTP connection to Azure Data Lake Storage Gen2

ADLS2 documentation: API guide, Authentication

Contents

• A. Register an Active Directory app
• B. Configure access to the ADLS2 Server
• C. Set up an HTTP connection
• D. Edit HTTP application details
• E. Configure auth settings
• F. Save, test, and authorize
• G. Integrate with ADLS2

A. Register an Active Directory app

You must first create an endpoint in Active Directory to sign in via OAuth 2.0:

1. Navigate to Azure App registrations, and click + New registration.

2. Register the new application:
   1. Give it whatever name you’d like.
   2. Choose the Supported account types for your setup.
   3. Provide the Redirect URI for the integrator.io OAuth 2.0 callback URL: https://integrator.io/connection/oauth2callback or https://eu.integrator.io/connection/oauth2callback.

3. Click Register. On the resulting Overview page, record your Application (client) ID and Directory (tenant) ID for use below in the integrator.io connection. 

4. On the Overview page, click Manage > Certificates & secrets. On the resulting Certificates & secrets page, click + New client secret; give the secret a name and choose an expiration period. Then, click Save.

5. Record the Value (you can ignore the Secret ID). The token will not be revealed again when you sign back in.

6. Finally, navigate to Manage > API permissions, and click + Add a permission:
   1. Choose the Azure Data Lake API.
   2. Check user_impersonation.
   3. Click Add permissions.

B. Configure access to the ADLS2 Server

1. Navigate to your ADLS2 storage account in the Azure portal, and proceed to the Access control (IAM) page.
2. Click + Add and then Add role assignment.

3. Set the Role to Storage blob data owner, and select the account that you will be signing in as.

4. Click Save.

C. Set up an HTTP connection

Start establishing an HTTP connection to Azure Data Lake Storage Gen2 in either of the following ways:

• From the Resources menu, select Connections. Then, click + Create connection at the top right.
      – or –
• While working in a new or existing integration, you can add an application to a flow simply by clicking Add source or Add destination/lookup.

In the resulting Application list, select HTTP.

The Create connection pane opens with required and advanced settings.

D. Edit HTTP application details

Note: This article is concerned only with the settings needed for ADLS2 connections. For complete field documentation, see Set up an OAuth 2.0 HTTP connection.

Configure HTTP headers: Add a header for x-ms-version, and provide any version value after 2017, such as 2020-08-04.

Base URI: Paste the Primary endpoint – Blob service value from Azure.

Media type: Accept the default value, JSON.

Success media type: Choose the media type that applies to this connection.

Error media type: Select XML.

E. Configure auth settings

Continuing in the Create connection pane, select OAuth 2.0 for the Authentication type. The settings then become specific to OAuth 2.0 connections:

Grant type: Select Authorization code.

iClient: Click the + button to create an iClient (token ID-secret pair). Enter an identifiable Name along with the following values you copied earlier:

• Client ID: Paste in the Application (client) ID from the Azure App registration page.
• Client secret: Paste in the token from the Certificates & secrets page. 

Click Save & close to add the iClient to your integrator.io account and select it for this connection.

Continuing in the Configure auth section, scroll to the Authentication URL setting:

Authentication URL: Enter the following URL, filling in your Directory (tenant) ID from the App Registration page:
https://login.microsoftonline.com/<tenant_id>/oauth2/v2.0/authorize

Scopes: Copy the Base URI from above and append /user_impersonation,offline_access.

Access token URL: Enter this URL in the same format as the Authentication URL, replacing /authorize with /token.

Client authentication: Select Send client credentials in body.

Access token headers: Add a header for Content-Type, and give it a value of application/x-www-form-urlencoded.

Continuing in the Configure auth section, scroll to the Location setting:

Location: Select Header.

Header name: Accept the default value, Authorization.

Scheme: Select Bearer.

Authentication fail status code: Enter 403.

F. Save, test, and authorize

Once you have configured the HTTP connection, click Save & authorize to follow the Microsoft prompts and authorize this connection:

G. Integrate with ADLS2

Then, when creating a flow step (such as an export, import, or lookup) to Azure Data Lake Storage Gen2, start by selecting an HTTP application, and then choose your new connection. 

Here’s an example export that simply fetches a JSON file named test.json from a container named container1: