2023 integrator.io release notes

Celigo AI: intelligent assistance for flow building

AI-assisted flow building addresses the many challenges associated with syntax complexity, as well as the time-consuming nature of creating and understanding expressions. For customers grappling with intricate handlebars and SQL expressions when building flows, Celigo AI acts as a creative facilitator in several ways:

By simplifying the creation and understanding of expressions, Celigo AI empowers everyone to build flows with confidence and complexity, fostering a more creative and efficient integration experience.

Up next, targeted for early 2024

AI-generated summaries and descriptions of flows

Celigo AI will provide a concise summary and detailed description of each flow. You can add notes to these descriptions to share knowledge within an account.

What's new and enhanced

AFE editor: auto-suggestions enhanced

The enhanced auto-suggestions feature eliminates the hassle of scrolling through endless lists by allowing you to easily find the helper or field you need, making handlebars expression writing even more straightforward and eliminating the need to navigate away from integrator.io to external help articles.

Here’s what this enhancement includes:

Introducing OpenAPI specification generation

With the introduction of OpenAPI specification generation, a standardized document is made readily available for exports, imports, and data flows, allowing you to seamlessly embed OpenAPI documentation into your product.

Why this is valuable:

How does it work?

Whether utilizing exports, lookups, or imports within a flow or as standalone operations, generating OpenAPI specifications is simple. The OpenAPI spec considers filters, transformations, and hooks associated with your operations, ensuring accuracy and completeness.

For exports and lookups, request and response schemas are generated based on the configuration. For imports, the request schema is crafted from the import configuration. In case of mock data availability, response structures are based on that.

When OpenAPI spec generation is not possible, guidance will be provided to update the configuration.

Error management: error tagging, filtering, and sorting

Add tags to errors for easy categorization

Simplify the error search process with tag-based filtering

Filter errors by tags with the new filter icon in the Open errors tab.

Sort errors for easy batch selection

A new sorting feature in the Open errors tab provides the ability to choose from columns like Message, Code, Source, Classification, and Timestamp.

Test mode: accurate previews, script customization, and mock data flexibility

Preview data now matches the selected source

When setting input filters on imports in flows that have multiple sources, the preview data will now correspond to the selected source. This ensures that the selected test run source is visible in downstream steps, allowing you to perform mapping, transformations, filters, and more.

Adapt scripts for test runs with the testMode flag

The testMode flag allows you to customize scripts and observe the resulting behaviors in test mode without altering your production data. The flag is sent as input to all scripts supported by the platform and will be set to false by default.

integrator.io will handle errors and exceptions related to this flag by providing clear error messages or fallback behavior when needed.

The testMode flag is set to true in preview calls as well, enabling the hooks logic configured for test runs to be utilized during previews. This removes the risk of executing unnecessary code in your hooks.

More flexibility with mock data

Several improvements have been made to the way mock data is handled in test mode.

MySQL connector: bulk inserts, automapper support, and updated import types

Enhanced file filtering in file transfer exports

These enhancements are designed to improve the file-transfer exports experience, reducing effort and simplifying the file selection process.

Improved navigation in Celigo Marketplace

These changes are designed to make navigation of the Celigo Marketplace more intuitive and convenient.

What's fixed

Fixed data preview for oneToMany imports

Previous behavior: In the past, for oneToMany imports (row-based or record-based, whether pathToMany is set or not), the send feature would send all the child records to the application for processing and display the response of only one child record.

Changes made: Now, when previewing a oneToMany import, the platform will display information for only one child record. The same changes apply to row/array-based data, which will now show and send data for only one row, as well as when pathToMany is configured (record-based data).

Other changes

Support: Help Center updates

New connector articles

Revised connector articles

New integrator.io articles

Revised integrator.io articles

New integration apps articles

Revised integration apps articles

What's new

Introducing Transformation 2.0

Transformations allow you to modify source records before they are processed by your flows. Simplifying complex source records before being processed by your flows is often a best practice for making flows easier to build and maintain. The new transformation 2.0 functionality addresses all the pain points and shortcomings of transformation 1.0 and makes transforming records much easier to visualize, build, and test.

Key features:

Auto-recovery for rate limit errors

New universal connector: JDBC

Scheduled to be released on October 10th

Enhanced debugging capabilities for flow test runs

Keep the feedback coming, we’re listening.

• NetSuite SuiteScript hooks are now supported for test runs, allowing you to see how the scripts are modifying the incoming data.

• For mock data fields, the option to populate with a canonical stub has been added, allowing you to simply copy/paste your JSON mock data, removing the need to manually format the data.

Left navigation redesigned

Enhanced configuration for HTTP connections

JWT Bearer authentication is now supported for universal OAuth 2.0 connections:

UI enhancements for Amazon Redshift and Microsoft SQL cloud connections

Enhancements for Amazon Redshift and Microsoft SQL cloud connections.

Improved experience for running scripts

New postResponseMap scripts on lookups will run per page of data.

integrator.io will now automatically detect the connector application

Clearer insights into account endpoint usage

We’ve simplified your view of endpoint utilization.

• See your usage clearly: Under the Subscriptions tab within My account, you can now easily view the exact number of endpoints you’re using against your endpoint entitlements.

• No more surprises: Worried about exceeding your endpoint limit? Don’t be. integrator.io will prompt you when you’re building a flow that will exceed your maximum endpoints, preventing any unexpected hiccups.

Additional hmacOptions fields

We’ve added three additional hmacOptions fields for including signatures in an API request body:

New connectors

Find complete instructions and popular integration scenarios – along with connector resource enhancements – at the Celigo platform connectors 2023.9.1 release notes.

What’s fixed

Highlight: you report it, we fix it

FTP JSON files were not processing handlebars that were sitting outside of an {{each}} code block. Your feedback brought this issue to our attention. We made changes to handlebars written before or after an {{each}} code block. The expressions are now processed with the file, but only if the expression includes custom settings and does not include record fields as input (data) outside the {{each}} code block.

Who is this for?

Anyone creating FTP imports with JSON files.

What’s required of you?

When creating an FTP import with JSON that includes an {{each}} code block within a handlebar template, be sure to include custom settings and to omit handlebars that have fields based on records.

Celigo University

If you have any feedback about the courses, please email university@celigo.com.

What's new

Debug flows in Test mode (beta)

You can now debug a flow in test mode to ensure the flow processes your records as expected before enabling the flow. Enter mock data manually or make calls to your source and destination apps to retrieve live data to use for testing purposes.

When you run a flow in test mode, you can review the processing behavior of each flow step option (transformation, mapping, filter, or hook) and fine-tune your flow configuration to handle all variations of results or responses from each application endpoint.

Parse records from NetSuite File Cabinet exports and more

You can now connect to the NetSuite File Cabinet to export, parse, and import files. Use this to manage, transfer, or lookup existing files, or even generate new files from existing records. You can use various file types, including, but not limited to, CSV, JSON, XLS, and EDI.

New connectors

Prebuilt connectors are now available for ABBYY, ActiveMQ, Databricks, Freightview, Oracle Autonomous Data Warehouse, and RabbitMQ.

Find complete instructions and popular integration scenarios – along with connector resource enhancements – at the Celigo platform connectors 2023.8.1 release notes.

What’s enhanced

User experience (UX) enhancements: Manage user access and permissions

Owners and admins now have clearer insight into the permissions available to the Manager and Monitor roles, including how the access is inherited within an account. The new visual tools help you to scale user access and the permissions model for your organization, and more easily assign permissions to specific integrations.

Support pull revisions for real-time exports with authentication

From the Revisions tab, you can seamlessly pull real-time exports with authentication from a remote integration to your current integration. You must enter and validate authentication credentials for real-time exports only the first time.

New option not to enclose values in quotes for CSV imports

You can now choose to disable quote validation and strip enclosing quotes from your CSV imports.

New iClient grant type authorization for PKCE

When connecting to your chosen applications, both Proof Key for Code Exchange (PKCE) grant types are now supported, ensuring your authentication flows meet the highest standards of security and compliance.

What’s fixed

Customer-reported issues resolved

Celigo University

Updated versions of the Shopify-NetSuite (Self-Paced and Partner) IA trainings are now available. New interactions in the courses clarify the IA setup, processes, and flow configurations.

If you have any feedback about the courses, please email university@celigo.com.

What’s enhanced

A Celigo platform, integrator․io, infrastructure upgrade was released on July 27th (EU) and August 1st (NA), 2023. This upgrade offers increased stability, enhanced performance, and advanced security features, including an upgrade to OpenSSL 3.0.

As part of this upgrade, support for insecure security mechanisms, such as TLS 1.0, and vulnerable ciphers will be discontinued. TLS 1.0 has been identified as an outdated and insecure protocol with various security vulnerabilities.

Action required

While we have temporarily enabled support for TLS 1.0 to ensure uninterrupted connectivity during the transition, we strongly encourage you to reach out to your vendors to confirm whether any of your certificates still rely on TLS 1.0.

To align with current security standards, you must review your SSL/TLS certificates and ensure they are configured to use TLS 1.2 or higher. If any of your certificates still rely on TLS 1.0, we urge you to take the necessary steps to upgrade your certificates and align them with the latest security standards. Additionally, we recommend migrating from legacy and outdated ciphers to stronger ones to enhance the overall security of your integrated systems using integrator․io. Please address these actions as soon as possible to maintain a secure and reliable connection to the integrator․io platform.

While extensive testing has been performed to ensure a seamless transition, in rare cases involving deprecated protocols and vulnerable ciphers, you may experience issues due to this upgrade. Should you encounter any connectivity issues during the upgrade process, please do not hesitate to contact Celigo support.

FAQs

What does this TLS upgrade impact?

Any resources in the Celigo platform that use connections via secure HTTP to external systems could be affected if the certificates used for connections are not TLS 1.2 compliant. They should work fine after this upgrade, but to ensure their security, we recommend you review the certificates with your admin/partners.

Will all types of connections be impacted?

The changes in this release apply only to connections of type HTTP, REST, Wrapper, and AS2, based on the connection’s configuration.

Can you assure me specifically about NetSuite and Shopify connections?

All prebuilt application connections – for example, to NetSuite, Shopify, and Salesforce – will not have any impact.

How about FTP/SFTP connections with PGP-based cryptography?

17647965544091-as2-url.png

Are any corrections needed for AS2 connections?

Required changes to AS2 connections depend on the configuration of the partner URL:

If the URL goes to an HTTPS connection, implying TLS, then it must be validated with your partner to verify that it is TLS 1.2 compliant.

What's new

New connectors

Prebuilt connectors are now available for Azure Form Recognizer, Confluent Kafka, Cvent, Lever, MariaDB, Meta, Nimble, Pinecone, Salsify, and Trello.

Find complete instructions and popular integration scenarios – along with connector resource enhancements – at the Celigo platform connectors 2023.7.1 release notes.

What’s enhanced

Select or look up a table for Snowflake imports and automap destination fields

When configuring Snowflake imports, you can now select the table to receive bulk SQL inserts from a list of validated tables that you can filter by name. You can also define rules in a dynamic lookup to identify the destination table, which you can reference in a handlebars expression.

Upon saving the import, you can immediately view default auto-mapped fields. You can also choose specific columns to map from the Snowflake table, allowing you to update the mappings as needed.

User experience (UX) enhancements: Script resources

You can start a script for any hook event, transformation, form initialization, or filter while working in Flow builder. You don’t have to go to another menu item and return to select it. In addition, the script stubs all contain complete details.

Also, when you examine a script’s properties (Resources menu > Scripts > Actions…), the support for JavaScript API scripts is now improved:

An iClient resource lets you select from among all APIs supported

You can now create an iClient for any supported API in a connection. For example, if you connect to Meta, you can customize your iClient so it’s specific to the Instagram API.

Increased performance in rendering complex flows

We are excited to announce a range of enhancements to the platform’s performance, stability, and security. In this release, we upgraded essential dependencies and libraries to the latest versions:

More resilient flow processing in the event of a crash

As part of ongoing efforts to improve the integration runtime, we have made significant enhancements in handling application crashes. Previously, all flows would be canceled if they encountered a crash resulting from uncaught exceptions, memory issues, or stuck third-party routines.

Now, when a crash occurs in an import or a lookup, all unaffected pages of data will continue to process. For pages in progress during the crash, you will receive the error, “Unexpected crash. Unfortunately, we do not know if the record was processed successfully before the crash,” alerting you to review and retry any unprocessed records.

Similarly, if a crash happens during a non-delta export, the pages in progress will be completed, and an error will be added to the dashboard: “Unexpected crash. Please re-run the flow to try again.”

In the case of a delta export, the flow will be canceled as usual. It is important to note that crashes during real-time flows or error retries haven’t changed.

New options for integrator.io API JavaScript runtime module functions

You can pass additional data in the input parameter’s options object when calling exports.run() and imports.run(). Customers have requested a way to send values, especially when running JavaScript APIs.

Slack webhook can respond with the required challenge value

A Slack app configured to subscribe to events will send a challenge along with a URL verification to the flow’s webhook listener. To receive the message, integrator.io now responds as expected with the challenge value and a response code of 200.

What’s fixed

Customer-reported issues resolved

Celigo University

If you have any feedback about the courses, please email university@celigo.com.

What's new

Download files from URLs in HTTP exports and lookups

You can download files provided in the Path to file URLs in the HTTP response body setting of HTTP exports and lookups. You can then transfer these files to subsequent flow steps.

The file URLs must be public or self-signed, not requiring authentication.

New option for custom settings: displayAfter

You can now place custom settings at a specified location in your export or import panels. Add the displayAfter value to indicate the setting after which you want to place your custom setting. This feature is available only for newly updated connectors that use the Simple vs HTTP toggle.

Review multi-factor authentication (MFA) audit logs

You can now view audit logs for MFA updates. Note that only account owners or administrators can require and edit MFA settings. Audit logs are recorded when account owners or administrators:

New connectors

Prebuilt connectors are now available for Ceridian, Clearbit, DocParser, JumpCloud, Listrak, Pendo, Splunk, Tallyfy, and Twitter Ads.

Find complete instructions and popular integration scenarios – along with connector resource enhancements – at Celigo platform connectors 2023.5.1 release notes.

What’s enhanced

Destination fields suggested in Mapper 2.0

When mapping import fields, Mapper 2.0 now suggests destination fields from the connector definition or the sample file you provide so that you no longer need to type the full name of each destination field during import mapping. The available suggestions also include the full JSON hierarchy for child fields.

NetSuite JDBC enhancements

Various updates to the NetSuite JDBC connector include the ability to configure the NetSuite.com connection with basic authentication.

User experience (UX) enhancement: Option to create/edit connection moved to selection list

One of the first requirements in creating a flow step, such as an export or import, is to choose a connection to your application. In many cases, you can pick a connection that you’ve already established for that endpoint.

Ongoing customer research revealed that the Create (+) and Edit () buttons to the right of the connection drop-down list could be a source of confusion, slowing this process. Now, the create action is offered in plain English within the list, whether or not you have built any connections to choose from. And editing is more clearly associated with the connection resource.

iClient enhancements

Various updates to iClients include the ability to toggle iClients between Simple and HTTP views.

Dashboard and consoles indicate who canceled a flow

You can easily view whether a flow was canceled by any user or the system by clicking the info icon next to the Canceled status in the Run console and Run history.

A new Canceled by option in the flow Run history and Dashboards (Completed flows > Runs > Run history) enables you to additionally filter canceled flow runs based on who canceled the flow.

Delta export preview enhancements

Delta export flow steps now allow you to select a custom date to populate the preview data. On preview, if no new data is available from the current delta export date setting, the Delta preview dialog prompts you to select an earlier time and date.

What’s fixed

Customer-reported issues resolved

What's new

Connect to OpenAI

You can now add a flow step to integrate with OpenAI with a prebuilt connector.

Download integrator.io Postman collection

You can easily download the integrator.io Postman collection at any REST API article in our Help Center. This includes the popular connections, flows, and users API endpoints.

What’s enhanced

User experience (UX) enhancement: Consolidated homepage selection for new items

In order to help our customers accomplish the most frequent tasks, the Celigo platform homepage offers newly redesigned menu options available from the Create and Upload buttons.

UX enhancement: Template installation streamlined

Previously, when installing a template, you had to read and accept a disclaimer that the template, unlike an integration app, is by definition not further managed by Celigo or the template’s author. To simplify installation, this step is no longer present.

UX enhancement: Keyboard access in Mapper 1.0 fields

When a source or destination record field has focus in Mapper 1.0, you can use the Home key as a shortcut to reposition the cursor to the start of the field and the End key to move it after the last character in the field. (Mapper 2.0 already supported these keyboard shortcuts.)

UX enhancement: “Use existing” flow step de-emphasized

When creating a flow step in Flow builder, you always have the choice to reuse an existing resource, such as a similar export or import that you’ve created for another flow. This option was presented in a list labeled Would you like to use an existing…?.

We’ve enhanced the setting to make it more intuitive for most integration developers to click Next and proceed directly to configure the flow step. If you do intend to insert an existing step at this point in the flow, all you have to do is check Use existing import (and similar) and select the flow step before continuing.

It is important to keep in mind that any edits that you make to an existing resource will also apply to all other flows that use it. Verify that no other flows will break as you modify the step in another flow.

UX enhancement: Color-coded “Delete” and “Remove” menu items

Throughout menu items in the Celigo platform, Delete actions (including removal to the Recycle bin) are more easily identified with a red icon and label.

What’s fixed

Customer-reported issues resolved

What's new

Account administrators can reset MFA for account owners

We expanded functionality for account administrator profiles. Admins can now reset the account owner’s multifactor authentication (MFA) settings if the account owner loses their authentication device and is unable to sign in to integrator.io. See Reset MFA for another user.

New connectors

Prebuilt connectors are now available for Afterpay, Amazon Vendor Central, Datadog, Etsy, LinkedIn Ads, SuiteAnalytics Connect using NetSuite JDBC, Sage Intacct, and ZoomInfo.

Find complete instructions and popular integration scenarios – along with connector resource enhancements – at Celigo platform connectors 2023.4.1.

What’s enhanced

Additional integration details in error notifications

Error notification email is more informative now – making it easier for you to identify where errors have occurred. Your email notifications will include details such as the environment (sandbox), integration name, and integration tag name.

Connector enhancements and updates

This release also includes several improvements to application connectors and API updates, as described in detail at Celigo platform connectors 2023.4.1.

When determining database endpoints, only the host is taken into account

We’ve relaxed the rules for calculating the number of endpoint app entitlements that an account uses for database and data warehouse connections, such as Microsoft SQL, MySQL, MongoDB, Oracle DB (SQL), PostgreSQL, and Snowflake.

Only the Host setting is taken into consideration for these connections. All identical host values now count as a single endpoint app.

In the case of Google BigQuery, all connections with a non-unique Client email setting are added to the endpoint app entitlements.

Support for flow branching in Integration apps

You can now add branching to managed flows in Integration apps. For more information on developing your own Marketplace apps, see Publish templates and Integration Apps.

What’s fixed

Customer-reported issues resolved

Support: Help Center updates

Integration app articles – new and updated

The following articles have been added to individual integration apps:

Updated articles (part of the release):

Updated articles (not part of release):

What’s enhanced

SuiteApp SuiteScript 2.x is now the default NetSuite API version when configuring NetSuite flow steps

We recommend that you select SuiteApp SuiteScript 2.x for the API version in the Advanced section of your NetSuite exports and imports to comply with the NetSuite recommendation.

SuiteScript 2.0 is supported only in SuiteApp. So, when you configure a flow to export data from or import data into NetSuite, the Celigo platform will default to SuiteApp SuiteScript 2.x and prompt you to install the integrator.io SuiteApp if it is not already installed.

Also, when you install a custom (DIY) integration using a ZIP file or when you clone an integration or flow that has SuiteApp resources, a step for installing and validating the SuiteApp will now be shown.

Agent updates are now automatically applied

The on-premise agent now automatically checks for updates and upgrades your agent to the latest version. You must download and install the latest version of the agent to take advantage of the auto-upgrade feature.

If you provide the same agent token to the new agent software, your existing connections that connect through that on-premise agent should work without interruption.

Connector enhancements and updates

This release also includes several improvements to application connectors, new EDIFACT templates, and API updates, as described in detail at the Celigo platform connectors 2023.3.1 release notes.

Data Loader sample file setting updated

For clarity and consistency, we revised the setting Upload the file to use to read Sample file (that would be parsed). See Create a Data Loader flow for more information.

Changes to required settings in wrapper connectors

The following settings in the Application details section are now optional instead of required when building wrapper connectors:

What’s fixed

Customer-reported issues resolved