This blog is part of a collection of blog entries that shows architectural concepts and configuration of the SAP PI REST Adapter. We also added some sample scenarios to make it easier for you to understand how your scenario can be implemented using the PI REST Adapter.
If you haven’t done so far, best is to start with the very first blog PI Rest Adapter - Don't be afraid within the blog series covering the concepts of the REST adapter. A complete list of all blog entries can be accessed from here PI REST Adapter - Blog Overview.
The current blog describes how to poll a REST API. The sender REST adapter in polling mode has been supported since release 7.31 SP16 / 7.4 SP11.
Scenario
We would like to frequently poll video information from Google's Youtube REST API, and store the same on a file system. Furthermore, the resultset should be split into individual messages per video ID, and already polled results should be discarded.
In the SAP Process Integration Designer perspective of the NetWeaver Developer Studio (NWDS), I have defined an Integration Flow with a REST sender channel and a file receiver adapter.
Let's focus on the configuration of the sender channel.
Configuring the REST sender channel
Double-click on the sender adapter to open the channel editor. Select the REST adapter type, and from the Message Protocol drop down menu the entry REST Polling.
Option 1: Incremental request based on timestamp of last call
In order to better understand the configuration below, let's take a look at a sample response in JSON format. The REST API returns an array of items each having a unique ID stored in the etag field.
First switch to sub-tab Data Format below tab Adapter-Specific.
The format is JSON, so select JSON from the Data type drop down menu. We would like to convert the JSON into XML, so select the Convert JSON to XML check box, and add a wrapper element to ensure that the converted XML format contains one root element only.
As mentioned below, the message should be split into individual messages per video ID. Select Split Result into Multiple Messages check box. As Array Containing Messages maintain items (See message format above).
Furthermore, duplicates should be removed. Select the Filter out Duplicates check box, and maintain etag as Unique ID Element (See message format above).
To place incremental requests, we would like to use the timestamp of the latest call. The value is stored between the calls and can be used in the REST URL as a placeholder with name incrementalToken. By the way, alternatively you can also use an XPath expression or a JSON element from the response of the last call such as next Page indicator or similar (shown below). From the Incremental Type drop down select entry Timestamp of Last Call. The timestamp format of the API complies with ISO 8601, and needs to be specified as follows: yyyy-MM-ddTHH:mm:ssZ with T and Z being constants. The REST adapter follows the Joda time format, see DateTimeFormat (Joda time 2.2 API). So, we need to place the constants in quotes. Maintain the Timestamp Format attribute as yyyy-MM-dd'T'HH:mm:ss'Z', and define an initial value that complies with the format.
Switch to tab HTTP Request, and maintain the Target URL as follows: https://www.googleapis.com/youtube/v3/search?key=<your API key>&part=id&q=SAP&maxResults=50&order=date&publishedAfter={incrementalToken}. Note, I added the placeholder incrementalToken in curly brackets which holds the timestamp of the last call. As HTTP Operation select GET. Finally, define a polling interval, here 3600 seconds.
Option 2: Incremental request based on response content
As another option I would like to show you how to configure the incremental request in case that many pages need to be requested. As you can see below, in the response two additional elements are added holding the previous (prevPageToken) and the next page token (nextPageToken). Latter can be used in the URL to gather the next response page.
From the Incremental Type drop down select the entry Response Content, and maintain the incremental token element as nextPageToken. This will store the value of the nextPageToken element into the placeholder incrementalToken.
From release 7.31 SP17 / 7.4 SP12 / 7.5 SP01 onwards, you can define an action in case that the very last page has been reached and hence the nextPageToken element is missing. You have the following options:
- treat as error which is the default
- skip current poll and retry later
- use an empty token value
- use the initial token value
- use a custom token value
Here, we chose the option Use Empty Value for Token.
On tab HTTP Request, maintain the Target URL as follows: https://www.googleapis.com/youtube/v3/search?key=<your API key>&part=id&q=SAP&maxResults=5&pageToken={incrementalToken}. Note, I added the placeholder incrementalToken in curly brackets which holds the token of the next page.
I hope this blog was helpful to understand how to configure the REST sender channel in polling mode. If you like to learn more, check out the other blogs in the series, accessible from the main blog PI REST Adapter - Blog Overview.
- SAP Managed Tags:
46 Comments
