A publish step sends your flow's records to a topic as messages, so that other flows can react to them without your flow knowing they exist. It behaves like any other destination step: you map data to it, failures land in error management, and it reports success and failure counts in the run console. This article covers adding a publish step, mapping records to messages, and tuning how messages are batched.
Prerequisites:
- The Topics license must be enabled for your account.
- You need access to at least one topic. If you have none, create one first — see Create and manage topics.
Add a publish step to a flow
- Open the flow in the flow builder.
- Add a destination step.
- Select Celigo Topics.
- Select Import messages into a topic.
- In the Publishing section, select a Topic.
- Enter a Name for the step.
The step now publishes to that topic every time the flow runs.
Note: There is no Connection field on a publish step. See Flow steps don't use a connection.
Choose a topic
Select a topic in the Topic picker, or select Create topic in the picker to create one without leaving the flow. See Automatic registration.
Map records to messages
The publish step maps data through the standard Mapping step component, the same as any other import. Whatever you map becomes the message payload.
A topic has no schema, so there are no required fields and nothing to validate against. Map the fields your subscribers need to act — they get the payload exactly as published.
Warning: Each message must be 1 MB or smaller. A record that maps to a payload larger than 1 MB fails to publish and appears in error management.
Publish several messages from one record
By default, each record becomes one message.
To publish each element of an array within a record as its own message, select One to many in the General section and set Path to one to many to the array's path. This is the same behavior as on any other import.
Tune the batch size
For throughput, the step publishes in batches rather than sending one request per message.
A batch is sent when whichever of these is reached:
- The standard 5 MB page-size limit.
- The configured Batch size — the number of messages per request.
- The maximum wait time — a partial batch is sent once it has waited this long.
Batch size is in the Advanced section.
Lower the batch size when payloads are large or when you want messages to reach subscribers faster. Raise it when you are publishing high volumes and throughput matters more than latency.
Batching doesn't change how errors work. Each message's outcome maps back to its own record, so error management and retry stay record-level.
Handle publish failures
A message that fails to publish becomes a record in the flow's error management, with retry data. Retrying the record republishes it.
Common causes are a payload over the 1 MB limit and a topic the flow no longer has access to. See Troubleshoot topics.
Note: Error records and their retry data are independent of the topic's retention. See Monitor a subscription.
Check what was published in execution logs
Execution logs record each publish under the account's Execution log preferences. In Run history > Run logs > Trace view, select the publish step to see the topic name, the timestamp, and the status code. At the Detailed and Debug levels, the entry also includes the published records.
Note: The status code shows that the topic accepted the batch, not that every record was published. Records that failed appear in error management.
See Log data captured by log level.
Publish step form reference
| Section | Fields |
|---|---|
| General | Name, Description, One to many, Path to one to many |
| Publishing | Topic |
| Advanced | Batch size, Concurrency ID lock template, Data URI template |
| Custom settings | Settings |
The step also supports the standard import step components from Actions: input filter, pre-map hook, mapping, post-map hook, post-submit hook, transformation, response mapping, post-response-map hook, and error handling.
Things to know
Several flows can publish to the same topic. A topic doesn't belong to the flow that publishes to it.
Message order. When several flows publish to one topic at the same time, ordering between them is best-effort. See Keep messages in order.
Usage metering. Messages published from a flow count toward the "messages published" usage metric.
Cloning and promoting. A cloned or pulled publish step keeps its topic, but retained messages are never copied. See how ILM handles topics.
Publishing from a tool. Because the publish step is an ordinary import, a tool can wrap it — which is how MCP servers and AI agents publish to a topic. The topic reference travels with the step and isn't overridden when the tool is called.