A publish step sends your flow's records to a topic as messages, so that other flows can react to them without your flow knowing they exist. It behaves like any other destination step: you map data to it, failures land in error management, and it reports success and failure counts in the run console. This article covers adding a publish step, mapping records to messages, and tuning how messages are batched.
Before you begin
- The Topics license must be enabled for your account.
- You need access to at least one topic. If you have none, create one first — see Create and manage topics.
Add a publish step to a flow
- Open the flow in the flow builder.
- Add a destination step.
- Choose the Celigo Topics application.
- Choose the step type Import messages into a topic.
- In the Publishing section, select a Topic.
- Name the step and select Save.
The step now publishes to that topic every time the flow runs.
Note: There is no Connection field on a publish step, and none is needed. Your own access to the topic is what determines which topics you can publish to.
Choose a topic
The Topic picker lists every topic you can access — including topics that aren't yet registered to this integration.
If you need a topic that doesn't exist yet, select Create topic in the picker to create one without leaving the flow.
When you save the step, the topic you selected is registered to this integration automatically — the same behavior as selecting a connection. See Grant access to topics.
If you have no topics yet, the picker shows:
You don't have access to any topics yet. Create one from Resources → Topics or from this integration's Topics tab, then select it here.
Map records to messages
The publish step maps data through the standard Mapping step component, the same as any other import. Whatever you map becomes the message payload.
A topic has no schema, so there are no required fields and nothing to validate against. Map the fields your subscribers need.
Warning: Each message must be 1 MB or smaller. A record that maps to a payload larger than 1 MB fails to publish and appears in error management. Map only the fields subscribers need rather than passing entire source records.
Think about what a subscriber needs to act, not what your source system happens to return. Subscribers can't ask you for more later — they get the payload as published.
Publish several messages from one record
By default, each record becomes one message.
To publish each element of an array within a record as its own message, use the standard One to many option in the General section and set Path to one to many to the array's path. This is the same behavior as on any other import.
Tune the batch size
For throughput, the step publishes in batches rather than sending one request per message.
A batch is sent when either limit is reached first:
- The standard 5 MB page-size limit.
- The configured Batch size — the number of messages per request.
Batch size is in the Advanced section.
Lower the batch size when payloads are large or when you want messages to reach subscribers faster. Increase it when you publish high volumes and throughput matters more than latency.
Batching doesn't change how errors work. Each message's outcome maps back to its own record, so error management and retry stay record-level.
Handle publish failures
A message that fails to publish becomes a record in the flow's error management, with retry data. Retrying the record republishes it.
Common causes are a payload over the 1 MB limit and a topic the flow no longer has access to. See Troubleshoot Topics.
Note: Error records and their retry data are independent of the topic's retention. A failed record stays retryable from its stored retry data even after the retention window it would have landed in has passed.
Publish step form reference
| Section | Fields |
|---|---|
| General | Name, Description, One to many, Path to one to many |
| Publishing | Topic |
| Advanced | Batch size, Concurrency ID lock template, Data URI template |
| Custom settings | Settings |
The step also supports the standard import step components from the … menu: input filter, pre-map hook, mapping, post-map hook, post-submit hook, transformation, response mapping, post-response-map hook, and error handling.
Things to know
Several flows can publish to the same topic. A topic doesn't belong to the flow that publishes to it.
Message order. Messages are stored in the order they arrive. When several flows publish to one topic at the same time, relative ordering between them is best-effort. To keep strict order, publish from a single producer and use a Concurrency ID lock template on subscribing imports. See Celigo Topics overview.
Usage metering. Messages published from a flow count toward the "messages published" usage metric.
Publishing from a tool. Because the publish step is an ordinary import, a tool can wrap it — which is how MCP servers and AI agents publish to a topic. The topic reference travels with the step and isn't overridden when the tool is called.