Intro
When we reach out to an API, sometimes it only shows us 20 items on one page. This is similar to going to eBay and wanting to buy antique furniture. We click the next page each time we want to view items.
Pagination is a technique where we hit an API and get the page's "0." Then, we want to iterate to reach the next page. We can do this by dynamically updating the page "index" when we reach out to the given service.
Example: Link
What data can be used for a loop?
API pagination often requires looping through multiple pages of data. We'll want to first check to see when we reach out to a service if it has additional pages.
Let's check this pokemon API to see when we reach out to an endpoint if it has more results.
Press Play/Pause to follow along with the GIF below
When reaching out, we can see that there is a "next" value with a URL. Indicating that we have more pages of data. We can also see that there is a pretty large number for the results, but we only get a handful of pokemon.
Example steps:
Pull in an HTTP request action
Set the name
Set the URL
Set the content type
JSON
Set the method
GET
Most services that don't require you to authentication with a password or API token will typically be GET by default.
Run the action to review the results in the event data
Review that there is a URL in the "next" value
How can you dynamically call an API service?
While looping, we need our HTTP requests to be dynamic. This means adjusting them based on the current state of the loop, so we’re always pulling the correct data from the correct page.
Press Play/Pause to follow along with the GIF below
Notice, we retain the same HTTP request name for the action. This is because when we loop later on we want to have our trigger look at the "get_data.body.next".
Example steps:
Select your HTTP request
CMD + C to copy the action (CTRL + C on windows)
CMD + V to paste (CTRL + V on windows)
Drag arrow from previous action to new one
Click into the URL and delete all the text
Click the + and then value
Set to get_data.body.next
This is the URL for the next page
Set the name
Set the URL
Select the first action and click run
Check the event data and review the results
We'll see that we have the next page of data and now "next" has a new value for the following page
We can also see the "previous" page
Notice there is an "offset" in the URL
How can you validate a loop?
To ensure our loop is functioning correctly, we implement a trigger that validates the loop’s progression. This trigger acts as a checkpoint to confirm that our loop is moving forward and pulling in the correct data.
Press Play/Pause to follow along with the GIF below
With our trigger set in place, we can now loop the data. By dragging from the bottom of the HTTP request to the Trigger. Having it go through and iteratively going to the next page. Since we dynamically reference the "get_data.body.next" it will look at the next page.
Example steps:
Drag a trigger into your story
Set the name
In this example I'm calling it "Continue?"
Drag it between the two HTTP requests
Pull the arrow from the first action to the trigger
Click into the data pill under rules that says "somekey.subkey.subkey.goal"
Set the JSON path to be "get_data.body.next"
This tells us what the next page of the URL we'd want to call
Click to the front of the JSON path and type out "IS_PRESENT" and click on it
This function tells us if the data is present. If it is, the result will be true. Otherwise, the result will be false. This is a really great way to test using a trigger to see if something is there or not.
Click the drop down that says "is equal" and change it "formula is true"
Drag the arrow from the trigger to the second HTTP request action
Drag the arrow from the bottom of the second HTTP request action and pull it into the Trigger
This is the loop that we've been developing. Now when we run it from the top it will
Check the trigger if there is another page
Make an API call to the next page
Loop these steps until there is no value for "next"
If your trigger is not set up correctly, the loop will continue indefinitely. If that happens you can click off of your action and in the properties panel click Disable. This will stop your story from continuing to execute
Check the event data for the last thing that executes from the second HTTP request action
You'll see that the next value is "null" and that the "results" are only showing a portion of the data. Where when we first started making the HTTP call there were a lot more items. This is just because we are at the "last" page.
Which action does the loop end?
During the loop, we might need to capture specific elements dynamically. It’s important to be mindful of our triggers here, ensuring they exit correctly to prevent any errors or endless loops.
Press Play/Pause to follow along with the GIF below
Example steps:
Drag an event transform action into your story
Set the name
In this example I'm calling it "Loop done"
Hover over the trigger and drag from the "No match" into the event transform
Anytime the trigger condition isn't met, it'll go to this action
Click into the event data of the trigger
Click the ... for "get_data" and find the previous value
This is the second to last page from the loop
Click the two squares to the left of previous to copy that JSON path
Click into your event transform
Click into the purple text to the right of "message" in the properties panel
Delete that and CMND + V to paste the copied JSON path (CTRL + V on windows)
If you want to add some extra flair you can go to the beginning of that data pill and add text like "Second to last page is: " just to add some static value before the URL.
Click on the trigger and then the 3 dots on the bar under it
Select re-emit last event
Click on the event transform action's event data to review the final results