Integration Gateway: Understanding REST data sourc...

former_member186731 · ‎09-21-2015

This blog is about Integration Gateway (IGW) in SAP Mobile Platform 3.0 (SMP).

And it is about using a REST-service as data source for creating an OData service based on IGW.

In the beginning of this series of tutorials, which show how to implement scripts required to consume REST data source, we’ve learned how to build the response as a String in the correct xml or json structure, which is expected by the IGW framework.

In the meantime, IGW supports an additional way to provide the data:

Instead of passing a String, it is possible to pass an instance of HashMap.

In the following blog, we’ll see how exactly this has to be done.

The description focuses on this topic, if you’re new to SMP and IGW and OData, please check here for more info.

Prerequisites

I expect that you've gone through my previous tutorials, explaining REST data source.

See here.

The prerequisites are:

Eclipse with SAP Mobile Platform Tools installed
SMP SP07
Basic knowledge about OData provisioning using the Integration Gateway component of SMP

Preparation

The data source

The backend REST service that we’re using in this exercise:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies

Please check the Appendix section below for more information about how to access this service

The OData model

According to the backend REST service, our OData model looks as follows

Note:

you can find the edmx file attached to this blog, you can use it to import in your Eclipse project

Bind data source in Eclipse

Bind data source to QUERY operation, using the following relative URI:

QUERY companies:

/sap/opu/rest/address/companies

READ company:

/sap/opu/rest/address/companies/{ID}

Configure Destination in SMP

The Destination host: https://sapes1.sapdevcenter.com/

User: your SCN user and SCN password

See Appendix section for some tips.

Custom Code

The following sample code is based on Groovy script.

Hardcoded dummy payload

As we’ve done in the first blogs of our series “Understanding REST data source”, let’s use a hard-coded response, for easier understanding.

Advanced readers my skip this section.

What we’re learning here is:

1) We store the data in a java.util.Map, where

- the key of the Map corresponds to the property name of the REST service

- and the value for the key corresponds to the value of the REST-property

2) we declare the used format by setting the content type as HashMap

message.setHeader("Content-Type", "HashMap")

The custom code for QUERY operation

For each entry of the REST service, we have to create an instance of such java.util.Map.

All the map instances are added into a java.util.List

This list is then set as body of the Message object in the processResponseData() method

Here’s the sample code of providing sample data in HasMap format for the QUERY operation



def Message processResponseData(message) {

    // some hardcoded dummy data as key-value pairs in a Map

    Map companyMap = new HashMap();

    companyMap.put("ID", "1234");

    companyMap.put("NAME", "TestCompanyName");

    companyMap.put("Category", "Monitors");

    companyMap.put("STREET", "DummyStreet");

    companyMap.put("POSTAL_CODE", "111-222");

    companyMap.put("CITY", "DummyCity");

    companyMap.put("COUNTRY", "DummyCountry");

    companyMap.put("WEB_ADDRESS", null); // property value null is allowed

    companyMap.put("PHONE_NUMBER", null); // result is <d:WEB_ADDRESS m:null="true"/>

    // in case of QUERY operation, the response is a list of maps

    List responseList = new ArrayList();

    responseList.add(companyMap);

    // specify the content-type is required

    message.setHeader("Content-Type", "HashMap");

    message.setBody(responseList);

    return message;

}

The custom code for READ operation

The approach is exactly the same.

The backend REST service provides one entry.

For this entry, we create a java.util.Map

Then we add this map into a java.util.List which is set as body.

Note:

The list is expected to contain exactly one entry, because we’re responding to a READ request.

However, if the list that you provide contains more than one entries, this won’t cause an error.

The framework will simply ignore the additional entries and just use one entry (the first one) as response for the READ request.

In our hard-coded dummy implementation, the sample code is exactly the same like above.

So we can skip it here.

Note: empty properties

If there's no value for a property, then we have 2 options.

1) as commented in the sample code above:

We can create an entry in the HashMap with the property name as key and with null as value

2) don't create a HashMap entry for an empty property

Properties that aren't key properties can be omitted in the HashMap

Accordingly, the following sample code will work fine as well:



def Message processResponseData(message) {

    Map companyMap = new HashMap();

    companyMap.put("ID", "1234"); // put only the key prop

    List responseList = new ArrayList();

    responseList.add(companyMap);

    message.setHeader("Content-Type", "HashMap");

    message.setBody(responseList);

    return message;

}

Generic implementation

In the above section, I’ve already explained everything that I intended.

Now let’s do a the work for a generic implementation.

If we want to use HashMap instead of XML- or JSON-String, we have to proceed as follows:

1. Get the response body from the backend REST service

2. Parse the XML into a Document object

3. Traverse the document, put each property node into a Map and each Map into the List

4. Set the List as body to the com.sap.gateway.ip.core.customdev.util.Message object

Sample Code for the QUERY operation

Our sample implementation of the processResponseData() method reflects these four steps



def Message processResponseData(message) {

    // 1. get the response string of the REST service

    String bodyString = (String) message.getBody();

    // 2. parse the response string into a Document

    Document document = convertStringToDocument(bodyString, message);

    // 3. read the data from document and store in a List of Maps

    List responseList = convertDocumentToListOfMaps(document, message);

    // 4. finally, set the List as body

    message.setHeader("Content-Type", "HashMap");

    message.setBody(responseList);

    return message;

}

Step 1: get the REST response

This step doesn’t require explanation



    String bodyString = (String) message.getBody()

Step 2: parse the REST response

This step is already covered by this tutorial

Step 3: compose the List

After converting the response body string to a Document object, we can traverse the nodes of the document.

This document has the following structure:



<asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">

  <asx:values>

    <COMPANIES>

      <item>

        <ID>1</ID>

        … more property nodes

    … more item nodes

This means that we have to navigate through the document until we obtain the list of “item” nodes:



Node rootElement = document.getFirstChild();

Node asxValuesNode = rootElement.getFirstChild();

Node companiesNode = asxValuesNode.getFirstChild();

NodeList itemList = companiesNode.getChildNodes();

Then we can loop over all item nodes and access all the property nodes.



for(int i = 0; i < itemList.getLength() ; i++){

    Node companyNode = itemList.item(i);

    // now get the properties

    NodeList propertyNodes = companyNode.childNodes;

For each property node we retrieve the name and the value, which are set as key and value to a Map



for(int j=0; j < propertyNodes.getLength(); j++){

    Node propertyNode = propertyNodes.item(j);

    String propertyName = propertyNode.getNodeName();

    String propertyValue = propertyNode.getFirstChild().getNodeValue();

    // put the properties and values in the map

    companyMap.put(propertyName, propertyValue);

}

Then we have to add the map instance to a List.

But before we do that, just one consideration:

This code will set the propertyValue as null to the map, if the xml node of the backend REST-service doesn’t contain any data (which can be the case for e.g. the property web-address).

This is ok and will be properly displayed in the response of our OData service.

However, we don’t have special treatment for the ID-property, so this might be set as null in the map.

But since it is the key field and as such it is mandatory, we have to check if the ID property is present and if it has a value.

If not, we mustn’t add the map to the ArrayList, otherwise our OData service will fail with an “Internal Server Error”



if( (!companyMap.isEmpty()) && (companyMap.get("ID") != null)){

    responseList.add(companyMap);

}

And finally the complete sample code of the helper method:



def List convertDocumentToListOfMaps(Document document, Message message){

    // the response is a list

    ArrayList responseList = new ArrayList();

    //the nodes

    Node rootElement = document.getFirstChild();

    Node asxValuesNode = rootElement.getFirstChild();

    Node companiesNode = asxValuesNode.getFirstChild();

    NodeList itemList = companiesNode.getChildNodes();

    // Compose the response Structure as ArrayList of HashMaps

    for(int i = 0; i < itemList.getLength() ; i++){

        Node companyNode = itemList.item(i);

        if (companyNode.getNodeType() == Node.ELEMENT_NODE){

            // the map for a Company

            Map companyMap = new HashMap();

  

            // now get the properties

            NodeList propertyNodes = companyNode.childNodes;

            for(int j=0; j < propertyNodes.getLength(); j++){

                Node propertyNode = propertyNodes.item(j);

                if(propertyNode.getNodeType() == Node.ELEMENT_NODE){

                    String propertyName = propertyNode.getNodeName();

                    String propertyValue = null;

                    Node propertyValueNode = propertyNode.getFirstChild(); // to get the value from e.g. <ID>123</ID>

                    if(propertyValueNode != null){

                        if(propertyValueNode.getNodeType() == Node.TEXT_NODE){

                            propertyValue = propertyValueNode.getNodeValue();

                        }

                    }

                    // put the properties and values in the map

                    companyMap.put(propertyName, propertyValue);

                }

            }

  

            //this is required, otherwise the service might fail

            if( (!companyMap.isEmpty()) && (companyMap.get("ID") != null)){

                responseList.add(companyMap);

            }

        }

    }

    return responseList;

}

Step 4. Configure the Message object

The only thing to mention here:

Don’t forget the set the Content-Type header to HashMap.



    // 4. finally, set the List as body

    message.setHeader("Content-Type", "HashMap");

    message.setBody(responseList);

Sample Code for READ operation

In general:

The implementation is very similar, because – as mentioned above – a List with Map has to be returned.

The only difference:

In our example, the backend REST-service has a different xml structure in case of READ, so we have to modify our parsing logic.



/* The response from backend REST service has the following payload structure:

 <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">

   <asx:values>

     <COMPANY>

       <ID>1</ID>

*/

def List convertDocumentToListOfMaps(Document document, Message message){

    // the response is a list

    ArrayList responseList = new ArrayList();

    //the nodes

    Node rootElement = document.getFirstChild();

    Node asxValuesNode = rootElement.getFirstChild();

    Node companyNode = asxValuesNode.getFirstChild();

    NodeList propertyNodes = companyNode.getChildNodes();

    // the map for a Company

    Map companyMap = new HashMap();

    // now get the properties

    for(int j=0; j < propertyNodes.getLength(); j++){

        Node propertyNode = propertyNodes.item(j);

        if(propertyNode.getNodeType() == Node.ELEMENT_NODE){

            String propertyName = propertyNode.getNodeName();

            String propertyValue = null;

            Node propertyValueNode = propertyNode.getFirstChild(); // to get the value from e.g. <ID>123</ID>

            if(propertyValueNode != null){

                if(propertyValueNode.getNodeType() == Node.TEXT_NODE){

                    propertyValue = propertyValueNode.getNodeValue();

                }

            }

            // put the properties and values in the map

            companyMap.put(propertyName, propertyValue);

        }

    }

    //this is required, otherwise the service might fail

    if( (!companyMap.isEmpty()) && (companyMap.get("ID") != null)){

        responseList.add(companyMap);

    }

    return responseList;

}

Error handling

One consideration that we can cover in this example:

If the user invokes a URL for an entity that doesn’t exist, we should react properly.

In our sample, we just use the status code and the error message that is sent by our backend REST service and set it as status code and error message for our OData service.



def Message processResponseData(message) {

    String bodyString = (String) message.getBody();

    int statusCode = (int) message.getHeaders().get("camelhttpresponsecode");

    if (statusCode < 100 || statusCode >= 300) {

        message.setHeader("camelhttpresponsecode", statusCode);

        message.setBody(bodyString);

        return message;

    }

This means, if we invoke e.g. the following URL

https://localhost:8083/gateway/odata/SAP/REST_DEMO_MAP;v=1/Companies('9999')

then the result in the browser will be

We can compare this result with the result that we get if we do the corresponding call directly to the backend REST service:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies/9999

The result:

The complete custom script

The following sample code contains the full content of the custom script for the QUERY operation (Groovy).

The content is the same as the script file attached to this blog.

The full content of the custom script for the READ operation can be found in the script file attached to this blog.



import java.nio.charset.StandardCharsets

import javax.xml.parsers.DocumentBuilder

import javax.xml.parsers.DocumentBuilderFactory

import javax.xml.parsers.ParserConfigurationException

import javax.xml.transform.OutputKeys

import javax.xml.transform.Transformer

import javax.xml.transform.TransformerConfigurationException

import javax.xml.transform.TransformerException

import javax.xml.transform.TransformerFactory

import javax.xml.transform.dom.DOMSource

import javax.xml.transform.stream.StreamResult

import org.apache.olingo.odata2.api.uri.KeyPredicate

import org.apache.olingo.odata2.api.uri.NavigationSegment

import org.apache.olingo.odata2.api.uri.UriInfo

import org.apache.olingo.odata2.api.uri.expression.CommonExpression;

import org.apache.olingo.odata2.api.uri.expression.OrderByExpression;

import org.apache.olingo.odata2.api.uri.expression.OrderExpression;

import org.apache.olingo.odata2.api.uri.expression.PropertyExpression;

import org.apache.olingo.odata2.api.uri.expression.SortOrder;

import org.w3c.dom.Document

import org.w3c.dom.Node

import org.w3c.dom.Element

import org.w3c.dom.NodeList

import org.xml.sax.InputSource

import org.xml.sax.SAXException

import com.sap.gateway.ip.core.customdev.logging.ILogger

import com.sap.gateway.ip.core.customdev.logging.LogMessage

import com.sap.gateway.ip.core.customdev.util.Message

def Message processRequestData(message) {

    return message;

}

def Message processResponseData(message) {

    message = (Message)message;

    // 1. get the response string of the REST service

    String bodyString = (String) message.getBody();

    // 2. parse the response string into a Document

    Document document = convertStringToDocument(bodyString, message);

    if(document == null){

        handleError("Error while parsing response body to xml: could not load inputSource to xml-document", message);

        return;

    }

    // 3. read the data from document and store in a List of Maps

    List responseList = convertDocumentToListOfMaps(document, message);

    if(responseList == null){

        handleError("Error: failed to convert the backend payload to hashmap", message);

    }

    // 4. finally, set the List as body

    message.setHeader("Content-Type", "HashMap");

    message.setBody(responseList);

    return message;

}

def Document convertStringToDocument(String bodyString, Message message){

    // parse the response string into a Document

    InputStream inputStream = new ByteArrayInputStream(bodyString.getBytes(StandardCharsets.UTF_8));

    InputSource inputSource = new InputSource(inputStream);

    inputSource.setEncoding("UTF-8"); // note: this is REQUIRED, because the input has a BOM, and has encoding UTF-16, which can't be parsed

    Document document = null;

    return loadXMLDoc(inputSource, message);

}

def Document loadXMLDoc(InputSource source, Message message) {

    DocumentBuilder parser;

    try {

        parser = DocumentBuilderFactory.newInstance().newDocumentBuilder();

    } catch (ParserConfigurationException e) {

        handleError("Error: failed to create parser: ParserConfigurationException", message);

        return null;

    }

    // now parse

    try {

        return parser.parse(source);

    } catch (SAXException e) {

        handleError("Exception ocurred while parsing source: SAXException", message);

        return null;

    }

}

/* The response from backend REST service has the following payload structure:

<asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">

  <asx:values>

    <COMPANIES>

      <item>

        <ID>1</ID>

*/

def List convertDocumentToListOfMaps(Document document, Message message){

    // the response is a list

    ArrayList responseList = new ArrayList();

    //the nodes

    Node rootElement = document.getFirstChild();

    Node asxValuesNode = rootElement.getFirstChild();

    Node companiesNode = asxValuesNode.getFirstChild();

    NodeList itemList = companiesNode.getChildNodes();

    // Compose the response Structure as ArrayList of HashMaps

    for(int i = 0; i < itemList.getLength() ; i++){

        Node companyNode = itemList.item(i);

        if (companyNode.getNodeType() == Node.ELEMENT_NODE){

            // the map for a Company

            Map companyMap = new HashMap();

    

            // now get the properties

            NodeList propertyNodes = companyNode.childNodes;

            for(int j=0; j < propertyNodes.getLength(); j++){

                Node propertyNode = propertyNodes.item(j);

                if(propertyNode.getNodeType() == Node.ELEMENT_NODE){

                    String propertyName = propertyNode.getNodeName();

                    String propertyValue = null;

                    Node propertyValueNode = propertyNode.getFirstChild(); // to get the value from e.g. <ID>123</ID>

                    if(propertyValueNode != null){

                        if(propertyValueNode.getNodeType() == Node.TEXT_NODE){

                            propertyValue = propertyValueNode.getNodeValue();

                        }

                    }

                    // put the properties and values in the map

                    companyMap.put(propertyName, propertyValue);

                }

            }

    

            //this is required, otherwise the service might fail

            if( (!companyMap.isEmpty()) && (companyMap.get("ID") != null)){

                responseList.add(companyMap);

            }

        }

    }

    return responseList;

}

/*

*  HELPERS

* */

def void handleError(String errorMessage, Message message){

    message.setBody(errorMessage);

    ((ILogger)log).logErrors(LogMessage.TechnicalError, errorMessage);

}

Appendix

REST Service in SCN

The REST service used in this tutorial is publicly available, you only need to sign up, afterwards you can access it with your SCN user and password.

Please see the following document for details:

Getting started with the SAP Netweaver Gateway Service Consumption System:

After registration, you should be able to access it via the following URL:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies

Configuring the destination in SMP

The URL for the Destination:

https://sapes1.sapdevcenter.com

Certificate

Since the URL is HTTPS, you need to download the certificate and import it into your SMP keyStore.

Test Connection:

For this Destination, it isn’t possible to do a “Test Connection”, as the server doesn’t send a valid response for the configured URL

As a workaround, you can proceed as follows:

Create a second destination, which is only used to test if the target host can be reached.

This second destination points to a URL that actually can send a valid response.

For example, enter the following URL as destination URL:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies

Proxy

If you get an error message on connection test, you might consider the following:

You might need to enter proxy settings in your SMP:

Go to:

https://localhost:8083/Admin/ -> Settings-> System

Note that you might need to restart the SMP server after changing the proxy settings.