Lookup caches enable you to store content in the Celigo platform for quick and efficient access later. Before you start creating lookup caches, study the integrations in your environment and determine the data which would benefit inclusion in lookup caches – for example, extensive or repetitive mappings.
Any lookup cache you create is listed in the Lookup caches page with all the details. You can sort the columns and view lookup caches based on your requirements.
Tip
-
Since a lookup cache is reusable across flows, click Actions → Used by in the Lookup caches page to check all the flows that reuse the resource before you edit it. (Also, if a lookup cache is widely used, carefully consider whether the best option is to reuse the lookup cache or clone it. If you want to have a single source of truth, then opt for reuse. However, if you want to create a copy, clone the lookup cache if the Include data when lookup cache is cloned or downloaded option is selected in the original lookup cache.)
-
If you want to view changes made to a lookup cache, click Actions → View audit logs in the Lookup caches page.
When you create or edit lookup caches, you can enter values in the following datatypes: string, number, boolean, null, and object (including nested objects). Values in the lookup cache will be saved as object type by default. Any values you provide in double quotation marks (including numbers, true/false, null, and the like) will be considered as strings. Any values that you provide without quotation marks will be considered as numbers.
You can create a lookup cache by:
Note
You can also create lookup caches using the API endpoints. See, Lookup cache API endpoints.
In cases where you have a small dataset that you want to reuse and don't require or prefer a CSV file to store details, then manually enter the Key-Value details in the Edit lookup cache form. For example, if you map the same values in different flows, then instead of having different static lookups, you can manually enter and save the mappings in a lookup cache.
-
In the left pane, click Resources → Lookup caches.
-
In the top-right corner of the Lookup caches page, click +Create lookup cache.
-
For Name, enter a unique and meaningful name for the lookup cache.
-
For Description, enter a description that clearly provides an idea of the contents stored within because lookup caches are global resources that can be reused.
-
Check the Include data when lookup cache is cloned or downloaded box if required. This option will be useful if you'd like to share or reuse the data in the CSV file. By default, the option is not selected, and your file data will not be copied when you clone or download the lookup cache.
-
Ignore the section, Lookup cache creation settings, which is required only if you're using a CSV file to upload details. Click Next.
Tip
When you click Next, a lookup cache is successfully created, but it will not have any values. You can continue and add Key-Value details. (Or, you can close the window, see it listed in the Lookup caches page, and then add these details later.)
-
In the Edit lookup cache page that opens, click Add row near the lookup cache name , and enter Key-Value details. Repeat based on the number of rows you want to add.
-
Click Save or Save and close.
In cases where you have an existing CSV file or can create one to store data, for instance – field mappings, then create lookup caches using this option.
Tip
If your CSV file is very large (greater than 5 MB), then use the data loader to import it to the Celigo platform, split the data into chunks so that each file remains within the 5 MB limit.
-
In the left pane, click Resources → Lookup caches.
-
In the top-right corner of the Lookup caches page, click +Create lookup cache.
-
For Name, enter a unique and meaningful name for the lookup cache.
-
For Description, enter a description that clearly provides an idea of the contents stored within because lookup caches are global resources that can be reused.
Tip
Mention the Lookup cache key in your description so that can quickly find this detail later, when required.
-
Check the Include data when lookup cache is cloned or downloaded box if required. This option will be useful if you'd like to share or reuse the data in the CSV file. By default, the option is not selected, and your file data will not be copied when you clone or download the lookup cache.
-
For the CSV file (sample extract below), select a valid file. Instead of uploading the file, you can also enter the details.
Tip
-
Expand and view Valid CSV file for lookup caches before you upload the file or enter details.
-
If you want to change the selected file, click CSV file and select another file.
-
-
For Lookup cache key, select the primary key in your file, that is a distinctive column with unique values in the rows. Click Next.
Note
Include the lookup cache key in the Description field for quick and easy recall later, when required.
-
In the preview page that opens, check the details and click Save or Save and close.
You will now see it listed in the Lookup caches page.
See also, Example 1 in Use lookup caches in flows – Examples.
In cases, when the data is not in CSV format, or it is not possible to download data from the source application, or the data tends to change, then use APIs to populate the lookup cache. First, create a lookup cache via the UI and note its ID. Then, upsert data into that lookup cache using a flow.
Given below is an example that you can consider to create and update a lookup cache using a flow.
Flow step 1: Use a Shopify connection to export inventory items.
Flow step 2: Use a lookup to add inventory level details, such as location, sku, and threshold to the source records based on the item_id.
Flow step 3: Use an import step to create/update the "Product SKUs" lookup cache data and store the inventory items and inventory level details.
Map the details as required, and check if the details are shown in the preview.
See also, Example 2 in Use lookup caches in flows – Examples.)
After a lookup cache is created, make sure to monitor and ensure that it is up-to-date and remains within the file size limits permitted. You can perform the following actions on your lookup cache.
Note
-
When you are mapping the data in a flow step, then you can select the lookup type for mapping and search for a lookup by name or description. You can also view, open, and edit lookup caches while mapping from within a flow step.
-
When reviewing changes in Integration Lifecycle Management (ILM), lookup cache data is not included in any revisions. References to the lookup cache are included (like in mappings and transformations), but the lookup cache data itself is not available.
You can monitor and edit the lookup cache on the Celigo platform itself to ensure that the details are up-to-date.
-
In the left pane, click Resources → Lookup caches.
-
In the Lookup caches page, select the lookup cache and click the Actions (...) menu at the end of the row.
-
Select Edit lookup cache. Make any required data changes.
-
Click Save and close.
You could also clone or download the caches and make changes.
In case you want to reduce API calls or perform operations faster, you can use hook scripts to programmatically create or manage lookup caches. Your lookup cache must be a part of an integration via mapping/lookups. To create hook scripts for lookup caches, you can follow instructions in Create hook scripts for exports and imports.
Tip
You can access and edit lookup cache data via script hooks only when that lookup cache is being used in any of the mappings in a flow step within that integration.
If your lookup cache is not used/referenced by a flow step in an integration, then you can create a dummy export or import step in the integration, and map a field using that lookup cache in Transformation 2.0 or Mapper 2.0. After this, you can use hook scripts to edit the lookup cache data.
Example
For a list of lookup cache operations that you can use in your JavaScript hook, see Lookup cache runtime objects in integrator.io API and JavaScript runtime objects.
You can clone or download a lookup cache if the size is not more than 5 MB and the Include data when lookup cache is cloned or downloaded box is checked. Cloning a lookup cache makes it easy and quick to create a copy, make any changes, then use the new cache in flows. Downloading a lookup cache enables you to create a backup before you delete the lookup cache or purge the entries. You can also share it, or edit and upload it, based on your requirements.
-
In the left pane, click Resources → Lookup caches.
-
In the Lookup caches page, select the lookup cache and click the Actions (...) menu at the end of the row.
-
Select either Clone lookup cache or Download lookup cache.
Tip
You can clone lookup caches across multiple environments in your account, based on the requirement and permissions that you have.
-
Click Save and close.
You can delete the lookup cache or purge all lookup cache entries
Delete the lookup cacheWhen you delete the lookup cache, it will be moved to the Recycle bin for 30 days. During this period, you can restore the lookup cache as required. After this period, the lookup cache will be automatically purged.
-
In the left pane, click Resources → Lookup caches.
-
In the Lookup caches page, select the lookup cache and click the Actions (...) menu at the end of the row.
-
Select Delete lookup cache.
-
Read the message and click Delete.
The lookup cache will now be moved to the Recycle bin. In the left pane, click Resources → Recycle bin and check that the lookup cache is listed here.
Purging all lookup cache entries permanently deletes the data and you cannot restore it. So, before you purge all lookup cache entries, consider if you need to create a backup and download the lookup cache if required. Also, check if the lookup cache is used in any other flow by clicking Actions > Used by at the end of a row, before you purge all entries so that flows do not run into any issues.