Articles in this section

Set up a connection to Celigo file storage

A Celigo File storage connection lets your flows read, write, and manage files in Celigo Storage the same way they use any other file provider, such as Amazon S3, FTP, or Google Drive. There are no credentials to enter: the connection grants access to your account's own storage, so you only decide which folders it can reach. Use it to stage files between flows, look up files per record, and automate folder and file management without configuring external storage.

Celigo File storage offers the same storage file steps and record-based operations as the Celigo APIs connector, and only those. If you also need to invoke a custom API, send data to a listener, or call other Celigo standard resource APIs, use a Celigo APIs connection instead. Existing Celigo APIs file transfer steps continue to work; you don't need to migrate them.

Feature availability:

  • Celigo Storage is available only on the multiple-environment structure.
  • Account owners and administrators can create a connection with any scope. Users with the Manage all role, or a custom role that manages an integration, can create a connection with Integration access scope limited to the integrations they manage.

Set up a connection

After you start the connection and select Celigo File storage (search for file storage), configure it in the Create connection panel:

Setting Instructions
Name your connection (required) Enter a clear and distinguishable name. You'll choose this connection by name when you configure exports, imports, and lookups.
Description Describe what this connection is for, such as the folders it's limited to.
Scope (required) Choose how much of Celigo Storage the connection can reach. Full access grants every folder in the environment. Custom limits the connection to the folders you list in File storage paths. Integration access limits the connection to the system folders of the integrations you select.
File storage paths Appears when Scope is Custom. Enter a comma-separated list of folder paths. Matching is prefix-based with no wildcards, so each path also grants access to everything beneath it. Enter / to allow all paths, or leave empty for no path restriction.
Integrations Appears when Scope is Integration access. Select the integrations whose system folders this connection can access. Non-admin users can select only the integrations they manage. If an integration is later deleted, it's removed from the connection's scope automatically.

PGP encryption

Celigo File storage supports PGP (Pretty Good Privacy) encryption for files. When you enable PGP on the connection, imports can encrypt files as they're written and exports and lookups can decrypt files as they're read. Configure PGP under Advanced, and then turn on Encrypt files or Decrypt files on each step that needs it.

  • Enable PGP cryptographic — Turns PGP on for the connection. When off, the encrypt and decrypt settings on flow steps are unavailable.
  • PGP public key — The public key used to encrypt files on import.
  • PGP private key — The private key used to decrypt files on export and lookup.
  • PGP passphrase — The passphrase that protects the private key.
  • Compression algorithm — The algorithm used to compress files before encryption.
  • ASCII armor — Turns ASCII-armored output on or off.

PGP keys and the passphrase are encrypted at rest, must be re-entered when you edit the connection, and are never returned in API responses, written to logs, or displayed after you save them.

Concurrency

  • Borrow concurrency from (optional) — Select another connection to share its concurrency level. Data flowing through both connections is submitted together under the shared limit.
  • Auto-recover rate limit errors — This setting is enabled by default with a predefined Target concurrency level. To change it, you must have admin or manage permissions.
  • Concurrency level — When Auto-recover rate limit errors is enabled, the concurrency level matches the value set by the runtime and can't be changed. When it's disabled, enter the maximum number of concurrent requests the connection allows. See Configure connections to optimize throughput and governance.

Use the connection in flows

Select Celigo File storage as the application on a source, destination, or lookup step, and then choose the step type. The fields and behavior are the same as the Celigo Storage steps on the Celigo APIs connector.

Step type Flow role Use it to Field reference
Transfer files out of source application Export Read files from a storage folder into the flow, parsed into records or transferred as-is Transfer Celigo Storage files
Transfer files into destination application Import Write files from the flow into a storage folder Transfer Celigo Storage files
Look up additional files (per record) Lookup Retrieve files from a storage folder for each record, using the same fields as the export Transfer Celigo Storage files
Export records, Look up additional records (per record), and Import records Export, lookup, import Manage files and folders without reading their contents: list, search, create folders, move, rename, copy, replace, and delete Manage Celigo Storage files with record-based steps

On the record-based steps, Invoke Celigo standard resource APIs is the only option and File storage is the only resource. To upload or download file contents, use the transfer files steps.

Steps reference folders by a stable internal ID, so renaming or moving a folder in Celigo Storage doesn't break a flow that uses it. For how folder references behave when a folder is deleted, see Use Celigo Storage as a file source or destination in flows.

Related articles