Overview
When communicating with APIs, you may receive a response that doesn't have the entire results expected. API pagination is a way to break large amounts of retrieved data into smaller, more manageable chunks. Compare this to online shopping, where we click the "next" page each time we want to view more items.
This article explains how to dynamically manage API pagination in Tines, ensuring you retrieve all the data you need.
Tines references
Before getting started, we recommend familiarizing yourself with the surrounding functionality this article covers:
Tines Tip: Import our Implement pagination with these techniques story into your tenant to help visualize how pagination can work into your story flow!
Make it happen
Identify pagination data
API pagination often involves looping through multiple pages of data. So, first, we need to determine if the API endpoint we're accessing has additional pages.
In this video, we reach out to the Pokémon API endpoint and verify if it has more results than what's provided in the "first" page (aka, the first response).
In the API response (via the HTTP request action's events), we can see that there is a next value with a URL. This URL indicates we have more pages of data:
Dynamically call an API endpoint
While looping, we need our pagination HTTP requests to be dynamic. This means adjusting them based on the current state of the loop, so that we’re always pulling the correct data from the correct page. Think of it as flipping through pages of your favorite book!
In this video, we look at how referencing the next value of the upstream Get data action works to dynamically pull the next page of the API endpoint.
Note:
When creating a pagination loop in Tines, we retain the same name for the actions that run the HTTP requests. When we build our loop later on, we want to have our trigger action dynamically use whatever value is at the
get_data.body.nextpath.Other platforms may provide pagination values in their API responses differently than our example in this article. We always recommend checking your endpoint's documentation for more information.
In the output of the second HTTP request action (aka, the "pagination" action), we can see in its events a next value (showing there's more data to pull), and a previous value (showing that there were results pulled before this request).
Validate a pagination loop
To ensure our pagination loop is functioning correctly, we implement a trigger that validates the loop’s progression, acting as a checkpoint to confirm that our loop is moving forward and pulling in the correct data.
In this video, we set up a trigger action to check if the next object of the Pokémon API endpoint returns a non-null value.
With the trigger action in place, we can now loop the data; having the flow iterate to the next page.
End a pagination loop
All books must come to an end! This logic can also be applied to API pagination, where we can utilize the prior trigger action to set up a "no match" flow (essentially, what should the story do if next returns a null value?).
In this video, we connect an event transform action to act as the ending of the loop should the trigger condition not match.
Once next returns a null value, the story flow will go to the Loop done action, closing out the pagination loop.
