This blog is about Integration Gateway in SAP Mobile Platform 3.0 (SMP).
When you create an OData service based on Integration Gateway (IGW), connecting to REST data source, and you deploy it to SMP, you can invoke a collection and you can apply a system query option like $select, it will work out of the box without any further scripting in your custom code.
But there’s no support for $top out of the box.
However, it can be achieved via manual coding in your script.
In one of my previous tutorials, I’ve explained how you can add $filter functionality to your OData service by adding the required implementation to your custom code.
In that tutorial, the procedure was to modify the URI of the REST service, before it gets called.
Thus, we added the code in the processRequestData() method.
The URI was modified in order to call the backend-REST-service with the (adapted) URI that includes the filter in the syntax that the REST-service supports.
So this procedure makes only sense if the backend REST service does support filtering.
In the present tutorial, we cannot apply the same procedure for the $top, because our backend-REST service doesn’t support a similar functionality.
Note:
if you’re using a backend-REST-service that does support a kind of $top functionality, then you would proceed similar like described in the filter-tutorial:
modify the URI in the processRequestData() method such that it complies to the requirements of the backend-REST-service.
In the present tutorial, we proceed differently:
we implement the $top manually in our script, since we cannot delegate to the backend-REST-service.
This tutorial is based on SMP SP08, but it should be working fine with other versions of SMP
Note: the code can be found attached to this blog.
Table of contents
1. Prerequisites
2. Preparation
3. Create a Service based on REST data source
3.1. OData model
3.2. Bind data source for query operation
3.3. Custom code
3.4. Test the query operation
4. Implement $top
4.1. Overview
4.2. Implement the $top
4.3. Test the $top
5. Summary
6. Links
7. Appendix
1. Prerequisites
If you’re new to IGW and REST data source, I’d recommend that you go through some of my previous tutorials , explaining REST data source, QUERY and READ operation.
In any case, you should have checked the tutorial that uses an xml parser for manipulating the response payload:
Please check the Links section at the bottom for more info.
Furthermore, you need:
- Eclipse with SAP Mobile Platform Tools installed
- SAP Mobile Platform
- Basic knowledge about OData provisioning using the Integration Gateway component of SMP
2. Preparation
REST Service
For this tutorial, we’re using a REST service that is public, you only need to sign up, afterwards you can access it with your SCN user and password.
Please see the following document for details:
Sign up and get started with the SAP Gateway - Demo Consumption System
Finally, you should be able to access it via the following URL:
https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies
Destination
In your SMP, you need to create an HTTP destination to the following URL:
https://sapes1.sapdevcenter.com/
Please refer to the Appendix section for some hints about configuring the destination
3. Create the Service based on REST data source
3.1. OData Model
Define OData model
The backend REST service that we’re using provides the following sample data:
So for our OData model, we create the following Entity Type:
3.2. Bind data source for query operation
We bind the entityset to REST data source for the QUERY operation and provide the following relative URI:
/sap/opu/rest/address/companies
3.3. Custom Code
We create the custom code for the QUERY operation, choosing Groovy as language.
The generated file content has to be replaced with the following (note that this code works for SMP SP08)
import java.nio.charset.StandardCharsets
import javax.xml.parsers.DocumentBuilder
import javax.xml.parsers.DocumentBuilderFactory
import javax.xml.parsers.ParserConfigurationException
import javax.xml.transform.OutputKeys
import javax.xml.transform.Transformer
import javax.xml.transform.TransformerConfigurationException
import javax.xml.transform.TransformerException
import javax.xml.transform.TransformerFactory
import javax.xml.transform.dom.DOMSource
import javax.xml.transform.stream.StreamResult
import org.apache.olingo.odata2.api.uri.KeyPredicate
import org.apache.olingo.odata2.api.uri.NavigationSegment
import org.apache.olingo.odata2.api.uri.UriInfo
import org.w3c.dom.Document
import org.w3c.dom.Node
import org.w3c.dom.NodeList
import org.xml.sax.InputSource
import org.xml.sax.SAXException
import com.sap.gateway.ip.core.customdev.logging.ILogger
import com.sap.gateway.ip.core.customdev.logging.LogMessage
import com.sap.gateway.ip.core.customdev.util.Message
def Message processRequestData(message) {
return message;
}
def Message processResponseData(message) {
message = (Message)message;
String bodyString = (String) message.getBody();
/* CONVERT PAYLOAD */
InputSource inputSource = new InputSource(new ByteArrayInputStream(
bodyString.getBytes(StandardCharsets.UTF_8)));
inputSource.setEncoding("UTF-8"); // REQUIRED, input has a BOM, can't be parsed
Document document = loadXMLDoc(inputSource);
// now do the refactoring: throw away useless nodes from backend payload
document = refactorDocument(document);
// convert the modified DOM back to string
String structuredXmlString = toStringWithoutProlog(document, "UTF-8");
/* FINALLY */
message.setBody(structuredXmlString);
return message;
}
def Document loadXMLDoc(InputSource source) {
DocumentBuilder parser;
try {
parser = DocumentBuilderFactory.newInstance().newDocumentBuilder();
} catch (ParserConfigurationException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error while creating parser");
return null;
}
// now parse
try {
return parser.parse(source);
} catch (SAXException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error parsing source");
return null;
}
}
def Document refactorDocument(Document document){
// find nodes
Node rootElement = document.getFirstChild();
Node asxValuesNode = rootElement.getFirstChild();
Node proddataNode = asxValuesNode.getFirstChild();
NodeList snwdNodeList = proddataNode.getChildNodes();
//rename all nodes of the feed
document.renameNode(proddataNode, proddataNode.getNamespaceURI(), "Companies");
for(int i = 0; i < snwdNodeList.getLength(); i++){
Node snwdNode = snwdNodeList.item(i);
document.renameNode(snwdNode, snwdNode.getNamespaceURI(), "Company");
}
//replace node
Node cloneNode = proddataNode.cloneNode(true);
document.replaceChild(cloneNode, rootElement);
return document;
}
/**
* Transforms the specified document into a String representation.
* Removes the xml-declaration (<?xml version="1.0" ?>)
* @Param encoding should be UTF-8 in most cases
*/
def String toStringWithoutProlog(Document document, String encoding) {
// Explicitly check this; otherwise this method returns just an XML Prolog
if (document == null) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error: document is null.");
return null;
}
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer t = null;
try {
t = transformerFactory.newTransformer();
} catch (TransformerConfigurationException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error creating Transformer");
return null;
}
t.setOutputProperty(OutputKeys.METHOD, "xml");
t.setOutputProperty(OutputKeys.INDENT, "yes");
t.setOutputProperty(OutputKeys.ENCODING, encoding);
t.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
OutputStream os = new ByteArrayOutputStream();
OutputStreamWriter osw = null;
try {
osw = new OutputStreamWriter(os, encoding);
} catch (UnsupportedEncodingException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error creating Writer");
return null;
}
BufferedWriter bw = new BufferedWriter(osw);
try {
t.transform(new DOMSource(document), new StreamResult(bw));
} catch (TransformerException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Error during transformation");
return null;
}
return os.toString();
}
3.4. Test the QUERY operation
Deploy the project and try it. The following request should give a valid response:
https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies
4. Implement $top
Until now, everything was preparation only.
Now that we have a working service, let’s have a closer look into the actual topic of this Blog:
Implementing the $top
4.1. Overview
The code the we've created up to here has already been explained in detail in the respective tutorial
What it does is:
- Get the payload string
- Parse the xml / convert to DOM
- Refactor the DOM
- Transform the DOM to String
- Set the converted payload string to the message object
This gives us a chance to hook in.
What do we want to do?
The “normal” QUERY operation provides a number of entries.
When adding the $top to the URL, we want to see only a reduced number of entries.
Example:
The result of the following URL is a list with 2 entries:
https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies?$top=2
These 2 entries are the first 2 entries of the “normal” QUERY.
How to implement it?
After step "3. Refactor the DOM", we have a well-structured list with all entries.
From this list we have to take the first 2 and remove all the rest.
The remaining steps 4 and 5 can remain untouched.
So our new script that supports $top will have the following sequence:
- Get the payload string
- Parse the xml / convert to DOM
- Refactor the DOM
- Apply $top
- Transform the DOM to String
- Set the converted payload string to the message object
4.2. Implement the $top
We create an additional method which will contain the implementation for handling the $top
First we have to retrieve the value of the $top expression that has been provided by the user.
If the user has invoked the following URL
https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies?$top=2
Then we need the value ‘2’
This is retrieved from the UriInfo object.
The UriInfo object can be accessed from the headers
The UriInfo instance can be asked for the $top number
It returns null if the user doesn’t specify the $top option.
This has to be considered in our code
UriInfo uriInfo = (UriInfo) message.getHeaders().get("UriInfo");
Integer topOption = uriInfo.getTop();
if(topOption == null){
return;
}
int topNumber = topOption.intValue();
Now that we know how many entries (starting from the beginning) have to be displayed, we need to adapt the payload to it.
Our method receives the Document instance which is the DOM representation of the response of the backend-REST-service.
Since it is already refactored, it contains the full payload in the following structure:
<Companies>
<Company>
<ID>“1”</ID>
<NAME>“SAP”</NAME>
<STREET>“Dietmar-Hopp-Allee”</STREET>
…
</Company>
<Company>
<ID>“2”</ID>
<NAME>“<companyName>”</NAME>
<STREET>“<companyStreet>”</STREET>
…
</Company>
…
</Companies>
The first node is the parent node for a list of (Company-) nodes that has to be modified according to the specified $top value.
So what we have to do is to delete all child nodes, but leave the first ones.
One possible solution is:
Check if the number of existing entries is higher the the $top
Delete the last node
Check again
Delete the last node
And so on
while(entitySetNode.getChildNodes().getLength() > topNumber){
Node lastCompany = entitySetNode.getLastChild();
entitySetNode.removeChild(lastCompany);
}
Furthermore, some details to consider:
$top=-1
What if the value that the user has given in the $top expression is invalid?
This is handled by the Olingo library, we don’t need to take care.
$top=0
This is a valid number, we return an empty list
Instead of deleting each child node one by one, we replace the current root node with an empty one
$top=9999
If the user specifies a number that is too high, then we can just return the full list.
It is up to the service implementor to decide if he wants to throw an exception for such a case.
And this is how our helper method looks like:
def applyTop(Document document, Message message){
UriInfo uriInfo = (UriInfo) message.getHeaders().get("UriInfo");
Integer topOption = uriInfo.getTop();
if(topOption == null){
return;
}
int topNumber = topOption.intValue();
Node entitySetNode = document.getFirstChild();
NodeList entities = entitySetNode.getChildNodes();
if(topNumber > entities.getLength()){
return;
}else if (topNumber == 0){
document.replaceChild(document.createElement("Companies"), entitySetNode);
}else{
while(entitySetNode.getChildNodes().getLength() > topNumber){
Node lastCompany = entitySetNode.getLastChild();
entitySetNode.removeChild(lastCompany);
}
}
}
At the end, we have to call our new applyTop()-method.
We invoke it in the processRequestData(), after we’ve refactored the payload:
def Message processResponseData(message) {
message = (Message)message;
String bodyString = (String) message.getBody();
/* CONVERT PAYLOAD */
InputSource inputSource = new InputSource(new ByteArrayInputStream(
bodyString.getBytes(StandardCharsets.UTF_8)));
inputSource.setEncoding("UTF-8"); // REQUIRED, input has BOM, can't be parsed
Document document = loadXMLDoc(inputSource);
// now do the refactoring: throw away useless nodes from backend payload
document = refactorDocument(document);
// handle system query options
applyTop(document, message);
// convert the modified DOM back to string
String structuredXmlString = toStringWithoutProlog(document, "UTF-8");
/* FINALLY */
message.setBody(structuredXmlString);
return message;
}
4.3. Test the $top
Deploy the project and try the following URLS:
- https://localhost:8083/gateway/odata/SAP/<yourService>/Companies
Result: The full payload, in order to compare to the following - https://localhost:8083/gateway/odata/SAP/<yourService>/Companies?$top=1
Result: only one entry is displayed - https://localhost:8083/gateway/odata/SAP/<yourService>/Companies?$top=2
Result: only 2 entries are displayed - https://localhost:8083/gateway/odata/SAP/<yourService>/Companies?$top=9999
Result: the same amount like in the first URL, without $top - https://localhost:8083/gateway/odata/SAP/<yourService>/Companies?$top=0
Result: an empty feed - https://localhost:8083/gateway/odata/SAP/<yourService>/Companies?$top=-1
Result: error message
5. Summary
In this tutorial we’ve learned how to implement the system query option $top.
Since the backend REST service doesn’t support a kind of $top functionality, we had to go for the approach to fetch the full data from the backend and then remove the unneeded entries in our script.
From performance perspective, this is not ideal, but if the backend REST service doesn’t support it, it is the only way to achieve it.
6. Links
Installing SMP Toolkit:
Tutorial for OData provisioning in SMP:
Preparing Eclipse for Groovy scripting: http://scn.sap.com/docs/DOC-61719
Introduction in REST datasource part 1: Understanding the return structure in xml
Introduction in REST data source part 3: Implementing the QUERY operation
How to use XML parser to convert the payload of the REST service
Integration Gateway: Understanding REST data source [5]: using XML parser
Implement $filter in your OData service, based on REST data source
Integration Gateway: Understanding REST data source [4]: Filtering
Overview of all REST blogs
The official documentation: http://help.sap.com/mobile-platform/
7. Appendix
Configuring the destination
The URL for the Destination:
https://sapes1.sapdevcenter.com
Since the URL is HTTPS, you need to download the certificate and import it into your SMP keyStore.
Test Connection:
For this Destination, it isn’t possible to do a “Test Connection”, as the server doesn’t send a valid response for the configured URL
As a workaround, you can proceed as follows:
Create a second destination, which is only used to test if the target host can be reached.
This second destination points to a URL that actually can send a valid response.
For example, enter the following URL as destination URL:
https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies
As such, you use this second destination to do the “Test Connection”.
If this succeeds, then the first destination should be fine as well.
This first destination will be used to configure our OData service, that we create in this tutorial.
Proxy:
If you get an error message on connection test, you might consider the following:
You might need to enter proxy settings in your SMP:
https://localhost:8083/Admin/ -> Settings-> System
Note that you might need to restart the SMP server after changing the proxy settings.
- SAP Managed Tags:
2 Comments
