Overview
When communicating with APIs, you may receive a response that doesn't include all the results you expected. API pagination is a way to break large amounts of retrieved data into smaller, more manageable chunks. Compare this to online shopping, where you click "next" each time you want to view more items.
This article explains how to dynamically manage API pagination in Tines, ensuring you retrieve all the data you need.
Tines references
Before getting started, we recommend familiarizing yourself with the functionality this article covers:
Tines tip: Import our Implement pagination with these techniques story into your tenant to help visualize how pagination can work in your story flow.
Make it happen
Identify pagination data
API pagination often involves looping through multiple pages of data. So first, we need to determine if the API endpoint we're accessing has additional pages.
In this video, we reach out to the Pokémon API endpoint and verify if it has more results than what's provided in the first page (i.e., the first response).
In the API response (via the HTTP Request action's events), we can see that there is a next value with a URL. This URL indicates there are more pages of data:
Dynamically call an API endpoint
While looping, we need our pagination HTTP Requests to be dynamic. This means adjusting them based on the current state of the loop, so that we're always pulling the correct data from the correct page.
In this video, we look at how referencing the next value of the upstream Get data action works to dynamically pull the next page of the API endpoint.
Note:
When creating a pagination loop in Tines, we retain the same name for the actions that run the HTTP Requests. When we build our loop later on, we want to have our Condition action dynamically use whatever value is at the
get_data.body.nextpath.Other platforms may provide pagination values in their API responses differently than the example in this article. We always recommend checking your endpoint's documentation for more information.
In the output of the second HTTP Request action (the "pagination" action), we can see in its events a next value (showing there's more data to pull) and a previous value (showing that there were results pulled before this request).
Validate a pagination loop
To ensure our pagination loop is functioning correctly, we add a Condition action that validates the loop's progression, acting as a checkpoint to confirm that the loop is moving forward and pulling in the correct data.
In this video, we set up a Condition action to check if the next object of the Pokémon API endpoint returns a non-null value.
Note: In this video, you'll see the action referred to as a "Trigger." This action has since been renamed to "Condition action." The functionality is the same.
With the Condition action in place, we can now loop the data, having the flow iterate to the next page.
End a pagination loop
All loops must come to an end. We can use the Condition action's "no match" flow to handle this, essentially defining what the story should do when next returns a null value.
In this video, we connect an Event Transform action to act as the end of the loop when the Condition doesn't match.
Once next returns a null value, the story flow will go to the "Loop done" action, closing out the pagination loop.
