REST Connector

REST API Connector can be used to fetch data from any REST endpoint. You can use the Flowrest API, which extends Unirest, to make rest calls.

Pagination With Page Number

Page Count Script

Record r = Flowrest.get(connection.getBaseUrl())
            .basicAuth(connection.getUsername(), connection.getPassword())
            .record();
return (r.count / 100) + 1; // this api returns 100 rows per page

Fetch Records Script

String url = "${connection.baseUrl}?page=${page}";
Record r = Flowrest.get(url)
            .basicAuth(connection.username, connection.password)
            .record();
return r.results; // same as r.get("results")

Sequential Pagination With Last Update Date

Page Count Script

return 1;

Fetch Records Script

String url;
if (scope.nextPageUrl != null) { // check if nextPageUrl is set in scope
  url = scope.nextPageUrl; // incremental pages...
} else if (!offset.isInitialRun()) { // incremental runs, use the previous run's max last update date value from the offset store and pass to the api filter
  url = "${connection.baseUrl}/api?limit=1000&q=lastUpdateDate gt ${offset.formatLastUpdateDateValue("yyyy-MM-dd'T'HH:mm:ssXXX")}";
} else {
  url = "${connection.baseUrl}/api?limit=1000"; // first time call
}
Record r = Flowrest.get(url)
            .basicAuth(connection.username, connection.password) // basic auth
            .record(); // returns JSON Object as Record
scope.hasNextPage = r.next != null; // scope.hasNextPage must be set to true for this script to be invoked in loop      
scope.nextPageUrl = r.next; // set the next page url to be used in the next run... see line 3 above
return r.results; // same as r.get("results")

scope is of type Record that can be used to store any variables to be re-used across multi-page requests

Schema Editor

1. Paste a sample payload with either a single JSON Object or an Array of Objects

2. Click Generate Schema to auto-generate schema based on the sample payload

3. Verify and update the schema as necessary

The more accurate schema will result in less runtime data validation errors