Branched flows cannot refresh access tokens
This is probably a bug, I will also raise a ticket but am posting it here because I've seen more posts of people having issues with branching so I hope this might help someone.
The basic premise is this:
If an access token is invalidated while running a branched flow, the specific call that got the 401 access denied error will be retried, but the result will not be passed to the next step in the flow. Also the error will be resolved immediately so its really hard to detect what is going wrong exactly, except for not all data reaching the target system, eg your export 4 records but only 3 get imported.
Thanks Bas and Steve, we're looking into it. There is a Jira ticket associated with Bas' original ticket that is in QA, but I am asking the team to be sure that it covers the failure scenario(s) you've described here.
We've seen this very same error in one of our new flows with branching. In our case the call receiving the 401 is an http lookup.0
Ticket #139419 submitted
If your flow is a Delta flow, the aborted run will update the flow timestamp value so subsequent runs skip over the records that were abandoned on the run with the 401 error. I've just carefully verified this is the case with the evidence clearly in the logs.
Overview of my steps:
- Flow that has a lookup that uses OAuth2
- The lookup fails with an expired token error (HTTP 401)
- IIO auto-resolves the error but doesn't retry and instead aborts the flow
- The delta timestamp for the flow IS updated as if it were a success
- The next flow run skips over the records exported in the previous run because the timestamp delta excludes them
In our situation, this has resulted in a flow that has failed to update the target system records due to this bug.
Additionally, the expired token should not be surfaced to the dashboard as an error. Our flows are littered with red due to this completely expected and normal behavior of OAuth2. It should be handled quietly and the flow allowed to continue to run.0
I've encountered similar issues and opened a ticket. Support indicated they couldn't find any open bugs related to our shared issue of failed token refresh with branched flows. You indicated you'd open a ticket - did you, and if so, how did it turn out?
One thing we're hearing from support that concerns me is that the intended design is for IO to "test" the connection prior to a lookup. The idea being that if the connect test receives a 401 they can refresh before trying the lookup. This is being presented as the explanation of why a 401 during a lookup isn't resolved and retried, but instead moves on.
I find this worrisome. A connection test can succeed, but the token can expire just after the test, there is nothing in the OAuth spec (that I can find) that would guarantee otherwise. In that scenario the lookup could still receive a 401 and abort.
My assertion with support is that a 401 should always trigger a refresh and retry of the offending operation.
What do you think?0
Here's another scenario I just encountered which proves and punctuates my earlier comments.
- I ran the flow
- It exported 16 records from NetSuite
- It successfully looked up the 16 records from the target endpoint using http OAuth2 (good)
- It was in the process of creating records in the same endpoint and the last of the 16 encountered a 401 - expired token. Flow stopped, completed, last run timestamp updated
- The record that experienced the 401 is now abandoned by the integration
I'd really like to hear from someone from Celigo on this. This is a major issue with no realistic workaround.0
Hi Steve, My workaround is to not use any branching. My guess is that branched flows use a different codebase to decide which tile to pass the page of data to, and that when Celigo refreshes the token that data sort of falls through the cracks in the system.
I havent seen the "connection test" occur in my testing (I made a custom webservice as a poc that de-auths every other call). In an unbranched job that receives a 401 it gets a new token and retries, in a branched job that receives a 401 it gets a new token and then stops.0
Please sign in to leave a comment.