An integrator.io export queries (or copies) data from an application or system. You also have options for transforming and filtering the data as it is extracted from the source application, Salesforce.
To begin exporting data from Salesforce, choose one of the following options:
-
Form the Resources menu, select Exports. In the resulting Exports page, click + New export. For the application type, choose Salesforce.
– or –
-
From the Tools menu, select Flow builder, and click Source Applications +. For the application type, choose Salesforce.
You can configure a Salesforce export to run on a scheduled basis or in real-time triggered by actual events that happen within Salesforce (like when a record is saved).
There are two options available in the What would you like to do? drop-down menu:
-
Export records from source application: Choose this option for scheduled exports. Scheduled exports are a better fit for syncing data in bulk, or when the timing of a sync is not critical, or when it is just not possible due to where the data is being sent. Scheduled exports are easier to configure and manage, and can also be a better option when getting started with a new integration use case.
-
Listen for real-time data from source application: Choose this option for real-time exports. Real-time exports are more difficult to configure and manage but are ideal for time-sensitive data changes.
You can also use a combination of scheduled and real-time exports. For example, you could enable a real-time export to send Account updates to your ERP, and then use a nightly scheduled export to send Contact updates.
Next, you can describe your use case to find similar exports in your existing flows or the Celigo platform's Marketplace. If you want to create an export from scratch, click Create flow step.
Click Next.
If you selected Export records from the source application, the following page displays.
Name and describe your export so that you can easily reference it from other parts of the application. For example: "Salesforce - Query All Accounts". Don't forget to create or add a Salesforce connection! Then, choose a Salesforce API type.
The REST API option allows you to export one record at a time.
The Bulk API 2.0 option allows you to insert, update, upsert, or delete many records asynchronously. Salesforce processes the request in the background. Learn more about Bulk API 2.0.
In both REST API and Bulk API 2.0, you can choose to include archived and deleted records in your search results.
For more information, see Salesforce Bulk API 2.0 Guide.
Use the SOQL query field to export data from Salesforce resources. Below is an example of querying Salesforce Accounts, where the name is “SF_Composite_Account%.”
select Name,Email__c,Phone,(select LastName,Title,Email from contacts where Name LIKE 'SF_Composite_Contact%') from Account where Name LIKE 'SF_Composite_Account%'
Provide the query and select the export type, then click Preview data to review the sample data that is coming from Salesforce.
Learn more about Bulk API 2.0 queries. There are various considerations you should take into account when using SOQL queries for Bulk API 2.0. Bulk API 2.0 doesn’t support SOQL queries that include any of these items:
-
GROUP BY, LIMIT, ORDER BY, OFFSET, or TYPEOF clauses.
-
Don’t use ORDER BY or LIMIT, as they disable PKChunking for the query. With PKChunking disabled, the query takes longer to execute, and potentially results in query timeouts. If ORDER BY or LIMIT is used, and you experience query time outs, then remove the ORDER BY or LIMIT clause before any subsequent troubleshooting.
-
Aggregate Functions such as
COUNT(). -
Date functions in GROUP BY clauses. (Date functions in WHERE clauses are supported.)
-
Compound address fields or compound geolocation fields. (Instead, query the individual components of compound fields.)
-
Parent-to-child relationship queries. (Child-to-parent relationship queries are supported.)
When exporting data from Salesforce, integrator.io provides four options for the Export type. Learn more about export types.
Note
The Once option is not available for Bulk API 2.0 exports.
Rest API export types
Bulk API 2.0 export types
Exports all new data since the date/time value that defines the last time the flow was run.
Store the timestamp from the Salesforce application in the Date field so that integrator.io can keep track of records that have changed since the last time the export was run. Choose a system-generated field to update the timestamp on each flow run. Applications maintain a standard 'Date Last Modified' (or 'Last Modified Date') field that always contains the date and time that a record was last changed. You can also select a non-system-generated field if you have your own logic in place (in the export application) to update the timestamp.
In the above image, the Created Date tracks the records that have changed since the last time the export was run, so records modified after the last export was run will be exported through the Created Date field reference.
Exports any data that has not been exported already. Once also automatically update records to mark them as exported (which can impact flow times).
Note
Not available for Bulk API 2.0 exports.
Select a boolean field (i.e. a checkbox field) from the export application that integrator.io can use to keep track of records that have already been exported. Integrator.io will only export the records where this boolean field is false (unchecked), and integrator.io will also automatically make a subsequent request back into the export application to set this boolean field to true for all the records that were exported (so that those records are not exported again).
In the above image, a boolean field named Once Export is a checkbox that tracks the export status for all records.
When you select the export type as Limit - export a set number of records, another field How many records would you like to export? is enabled for you to enter the maximum number of records to export each time this flow runs while developing and testing an integration. The default field value for existing users is set to 1. You can export a maximum of 100 records and if you enter a value more than the specified limit, an error message “Value cannot be less than 1 or greater than 100” is displayed.
The Advanced section includes advanced configurations that will allow you to customize your export in more detail. The following settings are available for both REST API and Bulk API 2.0.
-
Page size: When an export runs in the context of a data flow (where the data from the export is sent right away to an import queue) integrator.io will break the data being exported into one or more smaller pages of records. Saying this another way, integrator.io uses streaming to export data out of one app and import it into another app. The 'Page Size' field can be used to specify how many records you want in each page of data. The default system value (when you leave this field blank) is 20. There is no max value, but a page of data will automatically get capped when it exceeds 5 MB. Most of the time, the application that you are importing data into will bottleneck the page size value. For example, if you are importing data into NetSuite or Salesforce they each specify (in their API guides) a maximum number of records that can be submitted in any single request.
-
Data URI template: When your flow runs but has data errors this field can be really helpful in that it allows you to make sure that all the errors in your job dashboard have a link to the original data in the export application. This field uses a handlebars template to generate the dynamic links based on the data being exported. For example, if you are exporting a customer record from Shopify, you would most likely set this field to the following value '
https://your-store.myshopify.com/admin/customers/{{{id}}}'. Or, if you are just exporting a CSV file from an FTP site then this field could simply be one or more columns from the file:{{{internal_id}},{{{email}}}, etc... -
Override trace key template: Define a trace key that integrator.io will use to identify a unique record. Any value you provide overrides the default trace key for your app. You can specify a single field, such as
{{record.field1}}, or use a handlebars expression. For example,{{join "_" record.field1 record.field2}}generates a trace key such as123_456. Note: If you have applied a transformation to exported data, reference its fields in the trace key template without the path record. – for example,{{field1}}.
There are advanced settings specifically for Bulk API 2.0 as well:
-
Automatically delete jobs after exporting: Check this box to automatically delete this job after it’s executed successfully in Salesforce. This helps keep your job page clean if you run jobs frequently.
-
Maximum records per request: Enter the maximum number of records to retrieve for each set of query results. The request is subject to size limits, and returns data based on your service type. Use this field to reduce the maximum number of records per request if you receive timeout errors.
Comments
Hey,
Facing issue "entity type cannot be inserted: Connection" on saving a listener of Salesforce Object -
Can anybody guide me on this?
Hi Anuj kumar Maurya,
Sorry that we missed this. I recommend posting to the Celigo Community under the Troubleshoot custom flows section.
Thanks,
Kathyana
Please sign in to leave a comment.