Articles in this section

Developing with integration lifecycle management

Kathyana Queeman
Updated

An Integration Lifecycle Management (ILM) development lifecycle lets you build and test integration changes in one copy of an integration, then promote them through each stage — development, QA, and production — by pulling and merging. This article explains how ILM supports development teams, then walks you through setting up a two-environment, three-environment, or single-environment lifecycle. If you use ILM regularly, Celigo recommends creating environments to separate your development, test, and production integrations.

Prerequisites

Before you begin, confirm the following:

  • Environments — To create or manage environments, you're an account owner or administrator in the Production environment. See Invite users and manage access to an account or environment.
  • Permission — You have Manage access to the integration in every environment you pull between, or you're an account owner or administrator. See Manage user roles & permissions to an account.
  • Multiple environments — For two- and three-environment lifecycles only, your account uses the multiple-environment structure, which is available on Professional and Enterprise editions. You've created at least one non-production environment and enabled it. New environments are disabled by default. See Create and manage multiple environments.

How ILM supports integration development

Use cloning to speed up integration creation and change management: clone an integration, make changes in either copy, and pull them into the other. For definitions of clones, revisions, snapshots, pulls, and reverts, see Key concepts.

ILM works alongside your broader DevOps practice and supports the way integration specialists build and maintain integrations.

Tip: Avoid independently cloning integration resources (such as lookup caches, exports, or imports) and reusing them within the same integration, as this can lead to inconsistencies across revisions.

How teams use ILM

ILM supports teams that work in parallel and across multiple environments, and that need to test changes and roll them back if needed.

For example, you clone a production integration into a development environment and start building. Because each clone is independent, other integration specialists can work in their own clones without changing yours.

You update a flow or API, test it in your development environment, and create a snapshot whenever you want a restore point. When you're ready, you pull the approved changes into the next environment, review the differences between the remote and current integrations in the Review changes panel, and merge — development to QA, then QA to production — promoting the change through each stage in sequence rather than pushing directly to production. Each snapshot, pull, and revert is recorded as a revision.

If something breaks after a promotion, revert to a prior revision to roll back the change; see Revert to a pull or previous revert revision.

This workflow isn't limited to flows. For the full list of supported integrations and resources, see Supported integrations and resources. APIs and tools, including tools used as a step within a flow, are lifecycle-managed resources too, so you can clone, snapshot, pull, and revert them the same way. See Create and manage APIs using API builder and Introduction to Celigo tools.

Because cloning requires Manage access and promotion happens through pulls, ILM helps you control what reaches your production integration: each pull shows every change in the Review changes panel before you merge it. To keep direct edits out of production, give developers Manage access only to their development integrations, and have a user with Manage access to the production integration create and merge each pull.

Why use multiple environments

Without multiple environments, you build several integrations in the same environment. That works when you don't have many, but it gets confusing when one environment holds multiple versions of the same integration, such as development and QA. With multiple environments, you can separate each stage of your lifecycle into its own environment, keep track of every stage, and reduce the risk of accidentally pulling changes into your production integration.

Because each environment keeps its own isolated resources, you clone an integration into the next environment to promote it the first time, then pull changes after that.

Choose your lifecycle pattern

Before you start, decide which pattern fits your workflow:

  • Two environments (Development and Production) — Use this when you have one development stage before production. Changes move directly from a Development environment into Production.
  • Three environments (Development, QA, and Production) — Use this when you need a separate QA or testing stage between development and production. Changes move through the chain in sequence — you can't skip environments.
  • One environment — Use this when your account doesn't have non-production environments. You daisy-chain clones within a single environment.

Important: If you clone your integration from Development directly to Production, you can't add a QA environment later and move changes from Development to QA to Production. The Development and Production integrations are already a direct clone pair, and a QA integration you add later won't be part of that chain.

Example: one source integration and its clones

Imagine your original integration, Amazon-NetSuite, is running in production. You clone it to create integration A, named Clone of Amazon-NetSuite. The original is the source, and integration A is the clone. You can keep the two integrations in sync or introduce new changes in the source and then pull the new changes into your clone or vice versa.

The clone contains copies of the original's flows, steps, scripts, and configurations. Connections aren't copied, so you select or create them for the clone. See Supported integrations and resources. To introduce a change, make and test it in one integration, then pull it into the other and merge the changes. In a development lifecycle, you make changes in the development integration and pull them into the production integration.

You can create multiple clones of the original integration. This is helpful when you want to create and maintain a large number of integrations based on a common source integration. In this case, you can introduce changes in the source integration and then pull changes into any of the clones.

Tip: Repopulate lookup cache data after pulls and reverts.

Legacy Production and Sandbox accounts

Note: If you see Production and Sandbox in integrator.io, your account hasn't upgraded to the multiple-environment structure. In legacy Production and Sandbox accounts, a change in one environment affects both, because the two environments share the same underlying resources. This includes connections, imports, exports, flows, scripts, and API tokens, so use caution when editing resources in either environment. This behavior doesn't apply to accounts on the multiple-environment structure, where each environment keeps its own isolated resources. See Upgrade to the multiple-environment structure.

If your account hasn't upgraded to the multiple-environment structure, you can still use ILM across your Sandbox and Production environments. Develop and test changes in a Sandbox integration, then pull the stable changes into your Production integration.

Tip: Create your Development and QA integrations in the Sandbox environment, and your Production integration in the Production environment.

Set up a two-environment lifecycle

In this pattern, you create a Development environment, build an integration and flows there, clone the integration into Production, and pull changes from Development to Production.

ILM_in_two_environments.png

  1. Create a Development environment and enable it. See Create and manage multiple environments. You now have your Production environment and a Development environment.

  2. Select your Development environment from the list at the top of the page. The list shows your account and environment, for example, MyCompany: Production.

    Prod_and_development_dropdown.png

  3. Create an integration and create your flows in the Development environment.

    Dev_integration.png

  4. After you've developed the integration, clone it into your Production environment. You clone the entire integration, not individual flows.

    Clone_integration_for_ILM.png

    After cloning, select or create connections for the Production integration. See Configure the cloned integration.

  5. Pull your changes from Development into Production. In Production, open the cloned integration and select Revisions > Create pull. Select the source Environment (Development), and then the Integration to pull from. Review the changes, resolve any conflicts, and merge the pull. See Review changes and Merge changes.

    pull_changes_multi_env.png

Set up a three-environment lifecycle

In this pattern, you create Development and QA environments, build an integration and flows in Development, clone it into QA, clone the QA integration into Production, and then pull changes from Development to QA and from QA to Production. You can't pull from Development directly to Production.

ILM_three_environments.png

  1. Create Development and QA environments and enable them. See Create and manage multiple environments. You now have Production, Development, and QA.

  2. Select your Development environment from the list at the top of the page.

    Environment_dropdown_only.png

  3. Create an integration and create your flows in the Development environment.

    Dev_integration.png

  4. Clone the integration into your QA environment. You clone the entire integration, not individual flows.

    Clone_integration_for_ILM.png

  5. Clone the QA integration into your Production environment. You now have a daisy-chained lifecycle: Development, then QA, then Production. After each clone, select or create connections for the new integration. See Configure the cloned integration.

  6. Pull your changes from Development into QA. In QA, open the cloned integration, select Revisions > Create pull, select Development as the source Environment, and then select the Integration to pull from. Review the changes, resolve any conflicts, and merge the pull. See Review changes and Merge changes.

    pull_changes_multi_env.png

  7. Pull your changes from QA into Production. In Production, open the cloned integration, select Revisions > Create pull, select QA as the source Environment, and then select the Integration to pull from. Review the changes, resolve any conflicts, and merge the pull. See Review changes and Merge changes.

    Pull_from_QA_to_Prod.png

Set up a lifecycle in one environment

If your account has only one environment, you can still separate development and testing by daisy-chaining clones inside that environment. You can pull only between an integration and its direct clone or source, so the order in which you create the clones sets the order in which changes move.

In this example, you update your original integration through three daisy-chained clones: a Development (Dev) clone, a Quality Assurance (QA) clone, and a Dev+QA clone. The Dev+QA clone is created from the original integration, the QA clone is created from the Dev+QA clone, and the Dev clone is created from the QA clone.

6771462951195-Dev_QA_OG_pull.drawio.png

  1. Clone your original integration to create the Dev+QA clone.
  2. Clone the Dev+QA clone to create the QA clone.
  3. Clone the QA clone to create the Dev clone.
  4. Make and test your changes in the Dev clone.
  5. Pull the changes from the Dev clone into the QA clone.
  6. Pull the changes from the QA clone into the Dev+QA clone.
  7. Pull the changes from the Dev+QA clone into your original integration.

Because the clones start as copies of the same integration, you might see the same merge conflicts at each pull. See Resolve conflicts.

Split work among teams or developers

Tip: Give developers Manage access only to the Development integrations, and give the QA team Manage access only to the QA integrations.

You can separate work into development and QA teams by creating clones of each integration and granting the Development and QA teams access to the cloned integrations. When development work is complete, the QA team pulls the changes into the QA integration for testing. When everything works, a user with Manage access to the original integration pulls the final changes into it.

To let developers work independently, assign a clone to each developer. When a developer completes a task, a user with Manage access to the original integration pulls the changes into it. Other developers then pull from the original integration to bring the changes into their own clones.

Notes and constraints

  • You clone the entire integration at each stage, not individual flows; see Clone an integration for Integration Lifecycle Management (ILM).
  • In a three-environment chain, you can pull only between adjacent environments (Development to QA, and QA to Production). You can't pull from Development directly to Production.
  • Newly created environments are disabled by default. Enable each environment before use.
  • Connections aren't copied when you clone, and topics carry their configuration but not retained messages; see Supported integrations and resources.

Related articles