Articles in this section

Celigo Storage query API reference

The Celigo Storage query API accepts one SQL statement and runs it against files in Celigo Storage as an asynchronous job: you submit the query, poll the job's status, and fetch the result one page at a time. This reference covers the three endpoints, their request and response fields, the limits that apply, and every error code. For a step-by-step walkthrough, see Query files in Celigo Storage.

Prerequisites

You need a token whose scope covers the files you intend to query — a Full access API token, a Custom API token whose File storage paths include the folders, or a personal access token whose user can open the files. See Create an API token or Personal Access Token and Restrict an API token to specific Celigo Storage paths. Querying Celigo Storage is enabled per account on request; if it isn't enabled for your account, every request fails with 403 STORAGE_QUERY_NOT_ENTITLED.

Authentication and authorization

Send the token as a bearer token in the Authorization header. Celigo checks every file and folder the statement references against the token's access at submit:

  • A Full access API token reads any file in the account's Celigo Storage.
  • A Custom API token reads only files at or under the folders in its File storage paths. The reference itself must sit inside a granted folder — Celigo checks the reference, not the files it would expand to. A Custom token with no File storage paths can't reach Celigo Storage, and its submit request fails with 401 STORAGE_QUERY_ACCESS_REFUSED.
  • A personal access token reads the files its user can open, under the user's role and integration access.

A path outside the token's scope and a path that doesn't exist return the identical 404 STORAGE_QUERY_SOURCE_NOT_FOUND, so the API never confirms or denies that an out-of-scope file exists. Jobs are private to the identity that submitted them — another identity's job returns 404 STORAGE_QUERY_JOB_NOT_FOUND.

Submit a query job

POST /v1/storage/query

The query API has no templating. Handlebars expressions aren't rendered — anything that isn't rendered before the request is sent reaches the API as literal text. Unquoted, it fails to parse at submit with 400 STORAGE_QUERY_INVALID_SQL; inside quotes, it passes the checks and either fails when the query runs or compares as text.

A successful submission returns 202 with the job's identifier:

{
  "jobId": "66e1f2a3b4c5d6e7f8a9b0c1",
  "status": "queued"
}

A 4xx response means Celigo rejected the statement, a path, or the token's access, and no job exists. The body carries an errors array:

{
  "errors": [
    { "code": "STORAGE_QUERY_SOURCE_NOT_FOUND", "message": "No storage file found at path \"/staging/orders/\"" }
  ]
}

Get a job's status

GET /v1/storage/query/{jobId}/status

The status field is one of queued, running, completed, or failed. A completed job reports the result's shape alongside its timing:

{
  "jobId": "66e1f2a3b4c5d6e7f8a9b0c1",
  "status": "completed",
  "createdAt": "2026-09-12T18:02:11.418Z",
  "startedAt": "2026-09-12T18:02:12.907Z",
  "completedAt": "2026-09-12T18:02:14.115Z",
  "executionMs": 1208,
  "rowCount": 4680,
  "truncated": false,
  "pageCount": 3,
  "columns": [
    { "name": "region", "type": "VARCHAR" },
    { "name": "orders", "type": "BIGINT" },
    { "name": "revenue", "type": "DOUBLE" }
  ]
}
Field Description
status queued, running, completed, or failed.
createdAt, startedAt, completedAt When the job was submitted, started running, and finished. The 300-second timeout counts from startedAt.
executionMs How long the query ran, in milliseconds.
rowCount The total number of result rows across all pages.
truncated true when the result was cut off at the 1,000,000-row cap and rowCount reflects only the rows that were kept; otherwise false. Present on every status response.
pageCount The number of result pages to fetch. A query that matched no rows completes with rowCount: 0 and pageCount: 0.
columns The result's column names and the types the engine inferred or computed, in result order.
error On a failed job only — an object with code and message.

A failed job reports the error instead of the result fields:

{
  "jobId": "66e1f2a3b4c5d6e7f8a9b0c1",
  "status": "failed",
  "createdAt": "2026-09-12T18:02:11.418Z",
  "startedAt": "2026-09-12T18:02:12.907Z",
  "completedAt": "2026-09-12T18:07:13.002Z",
  "error": {
    "code": "STORAGE_QUERY_TIMEOUT",
    "message": "Query exceeded the maximum execution time of 300 seconds"
  }
}

Fetch a result page

GET /v1/storage/query/{jobId}/results?page={n}

Results are always delivered in pages of roughly 5 MB, and a row is never split across pages. Page numbers start at 1. Request the next page while hasMore is true, up to pageCount.

{
  "page": 1,
  "pageCount": 3,
  "rowCount": 2000,
  "hasMore": true,
  "rows": [
    { "region": "NA", "orders": "1840", "revenue": 91230.5 },
    { "region": "EU", "orders": "1211", "revenue": 60117.25 }
  ]
}
Field Description
page The page returned.
pageCount The total number of pages.
rowCount The number of rows on this page. The result total is on the status response.
hasMore true while later pages remain.
rows The result rows, each a JSON object keyed by column name.

A completed job with no rows returns page: 1, pageCount: 0, hasMore: false, and an empty rows array for page 1.

In the example, orders is a string and revenue is a number: Celigo returns 64-bit integers and decimals as strings so that large values keep every digit, and whole-number columns read from CSV and JSON files are typed as 64-bit integers. To receive a number, declare the column's type in the reader or cast the expression. See Celigo storage query read options and format behavior.

Job record retention

Celigo keeps each job's record and result pages for about 24 hours after the job finishes. After that, status and results requests return 404 STORAGE_QUERY_JOB_NOT_FOUND. Celigo doesn't keep the SQL text or the list of files a job read once the job has finished. There is no cancel request — a job runs until it completes, fails, or reaches the timeout.

Limits

Limit Value
Execution timeout 300 seconds, counted from startedAt. Time spent queued doesn't count. A job that exceeds it fails — never a partial result.
Result size 1,000,000 rows. A larger result is cut off and the status response reports truncated: true; use LIMIT to stay under it.
Sources per query 10 distinct file or folder paths. A folder counts once, however many files it holds.
Files per folder 100. A folder reference that expands to more is rejected.
Input per query 10 GB, summed across every file the query reads.
SQL text 16 KB per request.
Statements per request Exactly one.
Generated rows 1,000,000: the row counts of all generate_series and range calls in the statement, multiplied together, including calls in separate UNION ALL branches.
Submission concurrency No cap. Jobs queue and run as capacity allows; a submission is never rejected because jobs are already running.
Rate limiting Per environment. Query executions aren't billed as API calls.

Error codes

A rejected submission returns a 4xx with an errors array, and no job exists. A failure after submit surfaces as status: "failed" on the job with an error code and message. For symptom-based guidance, see Troubleshoot a Celigo Storage query job.

Request and statement errors

Code Status Meaning and resolution
STORAGE_QUERY_SQL_REQUIRED 400 The sql field is missing or blank. Include a non-empty sql string.
STORAGE_QUERY_SQL_TOO_LARGE 400 The statement exceeds 16 KB. Shorten it or split it.
STORAGE_QUERY_INVALID_SQL 400 The SQL didn't parse. The message carries the parser's error.
STORAGE_QUERY_NOT_SELECT 422 The statement is outside the SELECT family (which includes WITH, DESCRIBE, and SUMMARIZE), or is a SHOW form that lists engine state.
STORAGE_QUERY_MULTIPLE_STATEMENTS 422 The request carries more than one statement. Remove the second statement. A single trailing semicolon is fine.

Function, generator, and reader errors

Code Status Meaning and resolution
STORAGE_QUERY_FUNCTION_NOT_ALLOWED 422 A function outside the allowlist, named in the error. Functions that report engine or host state, such as version() and getenv(), are never allowed. A misspelled function name produces the same error.
STORAGE_QUERY_TABLE_FUNCTION_NOT_ALLOWED 422 A table function other than the file readers, unnest, generate_series, or range. Reference a storage path instead.
STORAGE_QUERY_GENERATOR_ARGUMENT_NOT_LITERAL 422 An argument of generate_series or range isn't a whole-number literal.
STORAGE_QUERY_GENERATOR_TOO_LARGE 422 The row counts of the statement's generators, multiplied together, exceed 1,000,000. Calls in separate UNION ALL branches are multiplied too. Lower a bound.
STORAGE_QUERY_NO_STORAGE_REFERENCE 422 The statement generates rows, through a generator, a recursive common table expression, or unnest of a literal list, without reading any storage file. Add a file reference.
STORAGE_QUERY_READER_PATH_NOT_LITERAL 422 A reader's first argument isn't a literal path string or a literal list of them.
STORAGE_QUERY_INVALID_READ_OPTION 422 An option the reader doesn't accept, a duplicate, a positional argument, or a wrong-type value, named in the error. filename, hive_partitioning, and encoding are never accepted.
STORAGE_QUERY_READER_FORMAT_MISMATCH 422 The reader doesn't match the file's format, for example read_json on a .csv file.

Source and path errors

Code Status Meaning and resolution
STORAGE_QUERY_INVALID_SOURCE 422 The FROM target isn't a valid storage path — a wildcard (* or ?), a schema-qualified name, a . or .. segment, or a character Celigo Storage doesn't allow. Reference a file, or a folder with a trailing slash.
STORAGE_QUERY_SOURCE_NOT_FOUND 404 No file exists at the path, or the path is outside the token's scope. Both cases return this same error.
STORAGE_NOT_A_FILE 422 The path names a folder but has no trailing slash. Add the trailing slash.
STORAGE_ITEM_NOT_FOLDER 422 The path names a file but ends in a slash. Remove the trailing slash.
STORAGE_ITEM_NOT_ACTIVE 422 The file's upload hasn't completed. Wait, then submit again.
STORAGE_AMBIGUOUS_PATH 422 Two folders with the same name exist at the same level of the path. Rename one.
STORAGE_QUERY_MIXED_FORMAT_FOLDER 422 The files in the folder don't share one extension and MIME type. Move each format into its own folder.
STORAGE_QUERY_UNSUPPORTED_FILE_TYPE 422 A file's extension and MIME type can't be mapped to CSV, JSON, NDJSON, or Parquet.
STORAGE_QUERY_FOLDER_EMPTY 422 The folder holds no files with completed uploads. Files in subfolders don't count.
STORAGE_QUERY_FOLDER_TOO_MANY_FILES 422 The folder holds more than 100 files. Split the files across folders.
STORAGE_QUERY_TOO_MANY_SOURCES 422 The statement references more than 10 distinct paths.
STORAGE_QUERY_INPUT_TOO_LARGE 4xx The files the query would read total more than 10 GB.

Access and enablement errors

Code Status Meaning and resolution
STORAGE_QUERY_ACCESS_REFUSED 401 A Custom token with no File storage paths. Add the folders to the token, or use a Full access token.
STORAGE_SCOPED_TOKEN_ASHARE_NOT_ALLOWED 403 The request sent an integrator-ashareid header with a Custom token that has File storage paths. Remove the header.
STORAGE_QUERY_NOT_ENTITLED 403 Querying isn't enabled for the account. It's enabled per account on request. Every submit, status, and results request returns it. Contact your Celigo account team to turn it on.

Status and results errors

Code Status Meaning and resolution
STORAGE_QUERY_INVALID_JOB_ID 400 The jobId isn't a valid identifier. Copy it exactly from the submit response.
STORAGE_QUERY_JOB_NOT_FOUND 404 The jobId is wrong, the job belongs to a different identity, or the job finished more than about 24 hours ago.
STORAGE_QUERY_JOB_NOT_READY 409 The job isn't completed. Poll the status endpoint. A failed job has no results.
STORAGE_QUERY_PAGE_OUT_OF_RANGE 400 page is above pageCount or isn't a positive integer. Page numbers start at 1.

Service errors

Code Status Meaning and resolution
STORAGE_QUERY_RESOLVER_RATE_LIMITED 429 Too many requests reached Celigo Storage while it looked up your paths. Wait and submit again with a longer interval.
STORAGE_QUERY_RESOLVER_UNAVAILABLE 503 Celigo Storage couldn't look up your paths. Retry shortly.
STORAGE_QUERY_ENQUEUE_FAILED 503 The query passed every check but couldn't be queued. No job exists. Retry shortly.
STORAGE_QUERY_REWRITE_VERIFICATION_FAILED 500 An internal check failed before execution. Retry once; if it recurs, contact Celigo Support with the statement and the time of the request.
STORAGE_QUERY_RESOLVER_REJECTED_REQUEST 500 An internal check failed. Retry once; if it recurs, contact Celigo Support.

Job failure codes

These appear in the error field of a job with status: "failed", not as HTTP errors.

Code Meaning and resolution
STORAGE_QUERY_TIMEOUT The query ran for more than 300 seconds. Narrow it with WHERE, reference a smaller folder, or select fewer columns.
STORAGE_QUERY_EXECUTION_FAILED The engine failed the statement — most often type drift past the inference sample, a failed cast, a file purged after submit, or a malformed JSON file. The message carries the engine's error, which names the storage path.
STORAGE_QUERY_WORKER_FAILED The process running the query stopped unexpectedly. Submit the query again; if it recurs on the same statement, reduce the data it reads and contact Celigo Support with the jobId.
STORAGE_QUERY_JOB_INTERRUPTED Celigo's service stopped while it held the job, for example during a deployment. The statement isn't at fault. Submit it again.

Learn more