Lookup caches are very useful resources that enable you to store data such as field mappings or environment-specific variables on the Celigo platform. The next time you require the data for lookups, you can quickly and easily retrieve the stored content instead of re-entering mappings or re-fetching data from the original and slower source. So, lookup caches (in-memory lookups) enable you to simplify and speed up the planning or running of your flows.
In Flow builder, lookup caches are global resources – the data stored can be accessed and reused across multiple flows, which reduces duplication of data and ensures consistency. Lookup caches support batch operations and allow users to handle large datasets efficiently. Therefore, they are a good option to consider for your flows in instances where static lookups are difficult and time-consuming to update or where dynamic lookups involve very large volumes.
Important
Based on your requirement, you can use the lookup cache capability to store data for quick access and reuse across integration flows in the account. Celigo prioritizes security, and we take reasonable measures to safeguard the data. However, Celigo makes no warranties for any sensitive information stored in the lookup cache on the Celigo platform, and you are fully responsible for ensuring compliance with your corporate policies and applicable regulatory guidelines surrounding such data.
Note
The Lookup caches feature is available only in Enterprise and Professional plans.
Based on your subscription plan, you can use:
-
100 million read + write units per month: For Enterprise account
-
25 million read + write units per month: For Professional account
Whether you are reading from or writing to the lookup cache, the request will be considered in terms of units.
-
1 read unit is a retrieved “key-value” that is equal to or less than 4 KB. (1 read unit ≤ 4 KB)
-
1 write unit is a written “key-value” that is equal to or less than 1 KB. (1 write unit ≤ 1 KB)
If you are within the monthly limits, you can have any ratio of read to write requests.
Lookup caches provide the following benefits:
-
Manage data efficiently: They can be quicker than manually entering the details each time.
-
Lower load on data sources: They help to maintain in-memory lookups/data storage so that you do not have to depend on external systems.
-
Enhance cost efficiency: They can lower operational costs, especially in scenarios where data sources charge based on queries or API calls.
Lookup caches can be very useful in these scenarios:
-
Repeated use: If you frequently use information, for example – shipping vendor details, then you can create a lookup cache with all the details one time and reuse it across flows.
-
Large datasets: If you want to use a large dataset of unique product IDs, for example – a master lookup table with 10,000 or more entries to translate Shopify SKUs to NetSuite internal IDs, you can create a lookup cache for quick and easy use across flows.
Note
Currently, lookup caches don't support:
-
Integration Lifecycle Management (ILM)
-
Scripts and handlebars
-
Mapper 1.0 or Rules 1.0 (that is, Transformation 1.0)
However, plans to support ILM, scripts, and handlebars are targeted for future releases.
You can create lookup caches to store data in any of the following ways:
-
Using a CSV file
-
By entering key-value details
-
Using APIs
For more information, see Create and manage lookup caches. and Lookup caches API endpoints.
You can perform the following actions on lookup caches based on your role and permissions:
|
What you can do |
Role & permissions |
|
View a list of all lookup caches |
|
|
View data in a lookup cache |
|
|
Get page-wise data from a specific lookup cache |
|
|
Create a lookup cache |
|
|
Update top-level fields in a lookup cache |
|
|
Delete a lookup cache |
|
|
Add data in bulk to a lookup cache (using an API) |
|
|
Delete data from a lookup cache |
|
|
Purge data from a lookup cache |
When you create or manage lookup caches, consider the following size limits:
|
Item |
Limit |
|
CSV file |
5 MB |
|
CSV row (key-value pair) |
64 KB |
|
Overall lookup cache |
50 MB |
|
Character limit for the lookup cache key |
256 characters |
|
Account storage |
1 GB per environment |
Ensure that your look up caches are within these limits. If you have unused or rarely used entries or lookup caches, you could delete them to free space to avoid issues when the size limits are breached. For example, you might not be able to save a lookup cache if a size limit is breached.