Extract data from documents

Extract data from documents in integrator.io by giving an AI agent three things: the document, instructions that describe the fields you need, and a JSON schema for the result. With intelligent document processing turned on, Celigo reads the document first — extracting text and tables, with OCR for scanned pages — and the agent returns the fields in your schema for the rest of the flow to validate and deliver.

Use this article when a flow receives invoices, purchase orders, remittance advice, shipping documents, forms, or spreadsheets, and a downstream application needs specific values from them. Document extraction isn't a separate step or resource in AI studio: you configure an AI agent step, and the agent recognizes from its input and instructions that it needs to extract data.

Prerequisites:

• Turn on intelligent document processing for your account. An account owner or administrator does this once in Account > Company > AI settings. See Turn on intelligent document processing.
• Make sure you have Manage access to the integration where you build the flow.
• Choose a model that supports file input. The Celigo AI models do; for bring-your-own-key (BYOK) connections, support depends on the provider and model. See Learn about Celigo AI agents.
• Collect a few sample documents to test with: at least one clean digital file, one scan, one multi-page document, and one of each layout you receive.

Bring documents into the flow

A flow picks up documents from wherever they arrive and passes each file to the AI agent as a file reference (a blobKey) with its file name.

Each file can be up to 20 MB, and the agent reads the first 50 pages of each PDF. For all supported formats and limits, see Supported file types and limits for intelligent document processing.

Add and configure an AI agent

1. In Flow Builder, select + after the step that brings in the document, and select Add AI agent. You can create an agent or choose an existing one. See Create an AI agent.
2. For Type, select Celigo AI or Bring your own key (BYOK), and then select a model.
3. Open the model's advanced settings and, if available, set Temperature to 0 (this is model dependent), for consistent results on extraction.
4. Set the output type to JSON. You define the schema in a later step.

Start with the default model, and compare others on the same sample documents in the agent's preview. Smaller models cost less and are often enough for well-structured documents; dense scans or multi-page tables may need a more powerful model.

Map the document to the agent

1. Select Mapping next to the AI agent to open Edit mapping.
2. Map the document to the files destination. Each file needs its file reference (blobKey) and its file name, including the extension.
3. Optional: map other context to text — for example, the sender and subject of the email the document came in — or map the whole record to record.

Celigo uses the file name's extension to decide how to read the file. If a file name has no extension, Celigo identifies the format from the file's contents. You can map more than one file to the same agent run, such as an invoice and its supporting statement. See Input for Celigo AI agents.

Write the instructions

The agent's Instructions tell it what the document is, which fields to return, and how to handle ambiguity. Be specific: the agent follows what you write, and every rule you state is one it doesn't have to guess. For example:

You extract data from vendor invoices. Return the fields defined in the output schema.

• For total_due, use the amount labeled Total, Total due, Amount due, or Balance due. Never use the subtotal.
• Return dates as YYYY-MM-DD and amounts as numbers, without currency symbols or thousands separators.
• Return the currency as a three-letter ISO 4217 code, such as USD. If no currency is printed, return null.
• Copy the invoice number and purchase order number exactly as printed, including leading zeros and dashes.
• Return every line item in order, including items on later pages. Don't return repeated header rows, subtotal rows, or tax rows as line items.
• If a value isn't in the document, return null. Never guess, and never calculate a value that isn't printed.
• Set needs_review to true and add a short reason to review_reasons when a required value is missing, unreadable, or ambiguous, or when the document isn't an invoice.

Tips for instructions:

• Name the tie-breakers — Documents often contain several candidates for one field: a subtotal and a total, an order date and a ship date, a bill-to and a remit-to address. Say which one to use.
• Standardize formats — State the format for dates, amounts, units of measure, and codes, so every document produces the same shape regardless of how it's printed.
• Cleanse values — Tell the agent to trim labels, remove currency symbols, split combined fields such as a city, state, and postal code, or map printed terms to your values, such as "Net 30" to NET30.
• Leave arithmetic to the flow — Ask the agent to return what's printed, and check totals in the flow, where the check is exact. See Validate extracted data and route exceptions for review.

Define the output schema

The JSON schema defines the fields the agent returns, their types, and which ones are required. Celigo validates every response against it. If the response doesn't conform, the record fails with a clear message, and no partial data moves downstream.

This schema returns invoice header fields, a line-item array, and two review fields:

{
  "type": "object",
  "properties": {
    "invoice_number": { "type": ["string", "null"], "description": "Invoice number exactly as printed" },
    "invoice_date": { "type": ["string", "null"], "description": "Invoice date as YYYY-MM-DD" },
    "due_date": { "type": ["string", "null"], "description": "Payment due date as YYYY-MM-DD" },
    "vendor_name": { "type": ["string", "null"], "description": "Legal name of the vendor that issued the invoice" },
    "po_number": { "type": ["string", "null"], "description": "Purchase order number referenced on the invoice" },
    "currency": { "type": ["string", "null"], "description": "Three-letter ISO 4217 currency code" },
    "subtotal": { "type": ["number", "null"], "description": "Amount before tax and shipping" },
    "tax_total": { "type": ["number", "null"], "description": "Total tax amount" },
    "total_due": { "type": ["number", "null"], "description": "Total amount due, including tax" },
    "line_items": {
      "type": "array",
      "description": "Every line item in the order printed, across all pages",
      "items": {
        "type": "object",
        "properties": {
          "item_code": { "type": ["string", "null"], "description": "Vendor item or SKU code" },
          "description": { "type": ["string", "null"], "description": "Line description" },
          "quantity": { "type": ["number", "null"], "description": "Quantity billed" },
          "unit_price": { "type": ["number", "null"], "description": "Price per unit" },
          "amount": { "type": ["number", "null"], "description": "Line total" }
        },
        "required": ["item_code", "description", "quantity", "unit_price", "amount"],
        "additionalProperties": false
      }
    },
    "needs_review": { "type": "boolean", "description": "true if a required value is missing, unreadable, or ambiguous" },
    "review_reasons": { "type": "array", "items": { "type": "string" }, "description": "Why the document needs review" }
  },
  "required": ["invoice_number", "invoice_date", "due_date", "vendor_name", "po_number", "currency", "subtotal", "tax_total", "total_due", "line_items", "needs_review", "review_reasons"],
  "additionalProperties": false
}

Tips for schemas:

• List every field in required, and allow null for values a document may not contain — The agent returns null instead of inventing a value, and the response stays valid.
• Describe every field — The model reads each description, so a clear description improves accuracy more than a clever field name.
• Use arrays for repeating data — Line items, shipments, or payments applied belong in an array. When a table continues across pages, the rows come back as one array.
• Keep identifiers as strings — Invoice numbers, PO numbers, postal codes, and item codes can have leading zeros that a number type drops.
• Use enum for fixed values — For example, limit a document_type field to the types you expect, including an other value.
• Add review fields — needs_review and review_reasons let the flow route uncertain documents to a person.

Test the schema on the model you chose. Providers support slightly different sets of JSON schema features, and the agent's preview shows an error if a model rejects the schema. See Output for Celigo AI agents.

Handle files that contain several documents

Some files hold more than one document — a PDF with several invoices, or a scan of a whole day's packing slips. To process each one separately:

1. In the schema, make the top-level result an array of documents, such as an invoices array where each item has the invoice fields.
2. In Instructions, tell the agent to return one item per document in the file, in order.
3. On the step after the agent, use one-to-many with the path to that array, so each document becomes its own record for validation and delivery.

Each agent run is independent. The agent doesn't remember documents from earlier runs, so to compare a document with an earlier one — a credit memo against its invoice, for example — bring the earlier record into the flow with a lookup and map it to the agent's record input.

Preview the agent with sample documents

Preview runs the agent with the same configuration it uses in the flow, so you can check extraction before any data reaches a downstream application.

1. In the agent, select Agent output to open the preview.
2. Select the Add file icon in the chat input, select Click here, and select a sample document.
3. Enter a prompt, such as Extract the invoice data, and send it.
4. Review the returned JSON: required values are present, types match the schema, line items are complete and in order, and formats match your instructions.
5. Repeat with each kind of sample document. To compare models, change the model and run the same document again.

Files you attach in the preview can be up to 10 MB and are available only during the active conversation; to test a larger file, run the flow with it. The preview also shows the AI credits each test used. See Preview an AI agent.

Use the extracted data in the flow

1. Add a response mapping on the agent step to bring the extracted fields into the record. The agent's JSON output is available under _json, for example _json.invoice_number and _json.line_items.
2. Validate the data before you write it anywhere. Common checks are a lookup that confirms the purchase order exists, a comparison of the totals, and a duplicate check on the invoice number. See Validate extracted data and route exceptions for review.
3. Map the fields to your destination, such as the vendor, reference number, dates, and lines of a vendor bill in your ERP.
4. Run the flow. Failed records appear on the step's Errors page, and each agent run is recorded in execution logs. See Monitor AI agent and guardrail activity with execution logs.

For an installable version of this flow — invoices from a Celigo Storage folder, email, or FTP to NetSuite vendor bills, with purchase order matching and duplicate screening — install the AI AP Invoice Ingestion Agent (Email/PDF to NetSuite Vendor Bill) template from Marketplace.

When a file can't be read

If the agent can't read a file, the record fails and the error appears on the step's Errors page and in Flow Builder > Run console. The most common cause is a file type the model can't read on its own because intelligent document processing is turned off.

For every error message, its cause, and what to do, see Troubleshoot intelligent document processing.

What document extraction doesn't do

• Handwriting — Handwritten content isn't supported.
• Contracts — Clause identification and free-form extraction from contracts need a different approach to schema design and aren't supported.
• Memory across runs — Each run is independent. The agent doesn't carry context from a document it processed earlier.
• Non-Latin scripts in scans — OCR recognizes languages written in the Latin alphabet. Digital text with selectable characters is read in any language. See Supported file types and limits for intelligent document processing.

AI credits, actions, and document retention

• Reading a document with intelligent document processing doesn't consume AI credits. The agent's model consumes AI credits as in any other agent run, and larger documents use more. See Track your AI agent actions and credits usage.
• Each file the agent processes counts as one AI agent action.
• Celigo doesn't keep the document after the run. Execution logs retain the extracted data and its metadata for 30 days by default; an administrator can set retention from 0 to 90 days. A retention of 0 leaves no extraction output to troubleshoot from after a run. See Monitor AI agent and guardrail activity with execution logs.

Related articles

• Intelligent document processing overview
• Turn on intelligent document processing
• Classify documents and route them by type
• Validate extracted data and route exceptions for review
• Supported file types and limits for intelligent document processing
• Troubleshoot intelligent document processing
• Create an AI agent
• Input for Celigo AI agents
• Output for Celigo AI agents
• Preview an AI agent