Articles in this section

Create an AWS IAM iClient for Amazon S3 and Amazon Redshift

Creating an Amazon Web Services (AWS) iClient allows you to securely store IAM role details that Celigo uses to access supported AWS services.

You can use an Amazon AWS iClient with supported connections that use AWS IAM role authentication, including:

  • Amazon S3
  • Amazon Redshift

You can create an iClient from Resources > iClients or directly from a supported connection when the connection supports IAM role authentication. See Amazon S3 connection and Amazon Redshift connection.

Important: The AWS IAM iClient setup is common, but the IAM role name prefix and permissions policy are connector-specific. For example, Amazon S3 and Amazon Redshift require different IAM role prefixes and different AWS permissions.

Before you begin

Open your AWS Console in a separate window so that you can copy values from Celigo, create or update the IAM role in AWS, and paste the role ARN back into Celigo.

Make sure your AWS account permissions allow you to:

  • Create and modify IAM roles
  • Create and modify IAM permissions policies

Create the AWS iClient in Celigo

  1. In Celigo, navigate to Resources > iClients.
  2. Click Create iClient.
  3. In Name, enter a clear and distinguishable name.
  4. Confirm that Type is set to AWS IAM.
  5. Copy the Celigo AWS account ID.
  6. Generate and copy the External ID.
  7. Create the IAM role in AWS.
  8. Paste the IAM role ARN into the Role ARN setting in Celigo.
  9. Save the iClient.

iClient settings

You can either create a new iClient or reuse an existing one. However, we recommend creating a separate AWS IAM iClient for each connector or use case when the IAM roles require different permissions.

Setting Description
Name Enter a clear and distinguishable name. For example, s3-prod-iam-iclient or redshift-prod-iam-iclient.
Type The iClient type. For AWS IAM authentication, this is AWS IAM.
Celigo AWS account ID (read-only)

Copy Celigo’s AWS account ID. You need this value when creating the IAM role trust relationship in AWS. 

Create an IAM role.

Role ARN

After creating the IAM role in AWS, copy the role ARN and paste it here. The required IAM role name prefix depends on the connector. For Amazon S3, the role name must begin with celigo-file-adaptor. For Amazon Redshift, the role name must begin with celigo-data-warehouse-adaptor

Amazon Resource Names (ARNs) uniquely identify AWS resources. ARNs should be handled carefully, but they are not secret values.

Retrieve your role's ARN.

External ID

Generate and copy the External ID unique to this iClient. Add this value to the AWS IAM role trust relationship.

Create an IAM role.

Create an IAM role in AWS

In AWS, create an IAM role that Celigo can assume.

  1. In AWS IAM, create a role.
  2. For Trusted entity type, select AWS account.
  3. Select Another AWS account.
  4. Paste the Celigo AWS account ID from the Celigo iClient.
  5. Under Options, select Require external ID.
  6. Paste the External ID generated in Celigo.
  7. Enter a role name.

See Create an IAM role

Use the connector-specific role prefix:

Connector Role name must begin with Example
Amazon S3 celigo-file-adaptor celigo-file-adaptor-s3-prod
Amazon Redshift celigo-data-warehouse-adaptor celigo-data-warehouse-adaptor-redshift-prod

Add a permissions policy

Create inline policies to add to the role. The trust relationship allows Celigo to assume the IAM role. It does not grant access to Amazon S3, Amazon Redshift, or any other AWS service. Add the permissions policy required for the connector.

Amazon S3 permissions policy

Use the following minimum policy for an Amazon S3 connection that needs to list the bucket and read/write objects. Replace <your-bucket> with your S3 bucket name. Add any additional permissions required by your specific flow.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:HeadBucket",
      "Resource": "arn:aws:s3:::<your-bucket>"
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<your-bucket>"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::<your-bucket>/*"
    }
  ]
}

Amazon Redshift permissions policy

Use the following policy for an Amazon Redshift connection that uses AWS IAM authentication.

Celigo uses GetClusterCredentialsWithIAM to obtain temporary database credentials based on the IAM identity. You don’t need to provide a separate named database user.

Replace <region>, <aws-account-id>, <cluster-name>, and <database-name> with your AWS Region, AWS account ID, Redshift cluster name, and database name.

GetClusterCredentialsWithIAM

Replace the placeholders with the customer’s AWS Region, AWS account ID, Redshift cluster name, and database name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowRedshiftIAMCredentials",
      "Effect": "Allow",
      "Action": "redshift:GetClusterCredentialsWithIAM",
      "Resource": "arn:aws:redshift:<region>:<aws-account-id>:dbname:<cluster-name>/<database-name>"
    },
    {
      "Sid": "RedshiftDataApi",
      "Effect": "Allow",
      "Action": [
        "redshift-data:ExecuteStatement",
        "redshift-data:DescribeStatement",
        "redshift-data:GetStatementResult"
      ],
      "Resource": "arn:aws:redshift:<region>:<aws-account-id>:dbname:<cluster-name>/<database-name>"
    }
  ]
}

AWS’s sample for GetClusterCredentialsWithIAM scopes the action to a Redshift dbname resource for a specific cluster and database.

Retrieve your IAM role’s ARN

After you create the IAM role and add the required permissions policy:

  1. In AWS, navigate to IAM > Roles.
  2. Open the role you created for Celigo.
  3. Copy the role ARN.
  4. Return to Celigo.
  5. Paste the ARN into the Role ARN field in the iClient.
  6. Save the iClient.

See Find your IAM role's ARN.

Use the AWS iClient in an Amazon S3 connection

  1. Create or edit an Amazon S3 connection.
  2. Set Auth type to IAM role.
  3. Select the AWS iClient.
  4. Enter the required S3 connection details, such as the ping bucket.
  5. Test and save the connection.

Use an IAM role whose name begins with: celigo-file-adaptor

Use the AWS iClient in an Amazon Redshift connection

  1. Create or edit an Amazon Redshift connection.
  2. Set Auth type to IAM role.
  3. Select the AWS iClient.
  4. Select the AWS Region.
  5. Enter the Default database name.
  6. Enter the Default cluster name as shown in your Amazon Redshift cluster details.
  7. Test and save the connection.

Use an IAM role whose name begins with: celigo-data-warehouse-adaptor

Note: Existing Amazon Redshift connections are not automatically converted to IAM role authentication. To use AWS IAM authentication, edit the connection, change Auth type to IAM role, select the AWS iClient, and test the connection.