Application Development Blog Posts
Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
pfefferf
Active Contributor

Intro

A time ago we evaluted the Integration of R in SAP HANA (SAP HANA R Integration Guide - SAP Library) to be able to execute statistical calculations using the wide range of available open source R implementations of statistical algorithms. During the evaluation we also thought about the option to directly integrate R in ABAP without SAP HANA, for such customers who do not plan to migrate to SAP HANA in the near future or do not want to call a RLANG procedure on SAP HANA from ABAP. The result was a little tool called FastRWebABAPConnector which integrates R in ABAP using Rserve and the additional component FastRWeb. Rserve is a TCP/IP server which allows the execution of R script code. FastRWeb uses the integrated webserver of Rserve >1.7 (or any other webserver) to allow access to the R script coding stored on the server via HTTP(s). Via a simple URL a R script can be executed. The only pre-condition is that the R script file contains a function "run" which is called by the FastRWeb component. The "run" function can have n input parameters which are mapped to the URL parameters used for the URL call. If for instance the run function has an input parameter named "inputData", the URL has to be called with "?inputData=Test" to transfer the value "Test" to the input parameter.

On the SAP NetWeaver ABAP side the connection to the Rserve/FastRWeb HTTP server is done by a "HTTP connection to external server" RFC connection maintained in transaction SM59. The connection points to the host/port under which the HTTP server component of Rserve is avaialable. Making requests to the HTTP server is done by the ABAP HTTP client class CL_HTTP_CLIENT.

The solution allows executing R script coding which does calculations or creates plots. Structured input/result parameters are transferred in JSON format. JSON serizalizing/deserializing is done on ABAP side using the standard ABAP transformation functionality which is available since NW 7.40 (in older releases one of the open source ABAP implementations could be used). On R side the serialization/deserialization functionality of R library jsonlite is used. Returning plots from R to ABAP is done by returning just the binary string of the created plot.


Following component model shows the high level architecture.

Configuration

In the following steps I describe briefly what configuration steps are necessary regarding Rserve/FastRWeb and I explain the details of the FastRWebABAPConnector component. I will not go into the very deep details of each step regarding the configuration, because the open source product sites already provide good installation guides and tips. One thing to mention is, that for the evaluation I used Rserve for Windows, for production scenarios always Rserve on a Linux bases systemhas to be used, because the Windows version has some limitations (for instance regarding concurrent processing).

Rserve

Pre-condition to install Rserve is of course to have installed an R Runtime. Rserve itself can be installed either by downloading and extracting the binary for the relevant platform from Rserve - Binary R server - download/files - RForge.net or via the R console (I use the console of RStudio) with command install.packages("Rserve"). To start the Rserve TCP/IP server in the R console the Rserve library has to be loaded with library("Rserve"). After that the process can be executed by calling function Rserve(). The Rserve TCP/IP server can be reached by default on port 6311. For an easier start up process it is recommended to create a shell script which starts the Rserve TCP/IP server using the executable for Rserve which can be found in the Rserve installation folder.

FastRWeb

After the Rserve TCP/IP server is installed the FastRWeb component can be installed and configured. In the following steps the %R_HOME% is the placeholder for the R runtime installation folder. All paths are related to a Windows x64 systems.

StepDescription
Install FastRWeb packageInstall the FastRWeb package using the R console with command install.packages("FastRWeb"). As result the FastRWeb sources are available in folder %R_HOME%\library\FastRWeb.
Copy Rserve files to R binary folderThe files "Rserve.dll", "Rserve.exe" and "Reserve_d.exe" have to be copied from folder %R_HOME%\library\Rserve\libs\x64 to folder %R_HOME%\bin\x64. Copying the files is necessary because the "R.dll" file stored in the target folder is necessary for the execution of the executables. Remark: With additional configuration it is not necessary to copy the files.
FastRWeb Rserve configuration

For a proper execution of FastRWeb on Rserve following configuration settings have to be done in file "rserve.conf" located in folder %R_HOME%\library\FastRWeb\code.


http.port 9060


http.raw.body TRUE


root C:/Program Files/R/R-3.2.2/library/FastRWeb


source C:/Program Files/R/R-3.2.2/library/FastRWeb/code/rserve.R


FastRWeb.root C:/Program Files/R/R-3.2.2/library/FastRWeb


control assent


























The HTTP port of the FastRWeb Component, the FastRWeb roots and the R script which is executed during the server start are maintained. In that example the web server is reachable at port 9060.

Rserve R start script

During the start up of Rserve a R script is executed. As defined in the step before for the FastRWeb configuration it is the script "rserve.R" located in folder %R_HOME%\library\FastRWeb\code. The script has to be enhanced at the beginning with following lines to load the FastRWeb package and to handle HTTP requests via FastRWeb.


library("FastRWeb")


.http.request <- FastRWeb:::.http.request
























Store R Scripts in web folder

R scripts to be called by a URL have to be stored in folder %R_HOME%\library\FastRWeb\web.R. That is the web folder of the FastRWeb component.

For instance a script named "test.R" is then available via URL http(s)://<host>:<port>/test.

Script to start Rserve with FastRWeb configuration

It is recommended to create a shell script to start the Rserve TCP/IP server with the specific FastRWeb configuration within folder %R_HOME%\bin\x64.

For instance:


R CMD Rserve_d.exe --RS-conf "C:\Program Files\R\R3.2.2\library\FastRWeb\code\rserve.conf"






















FastRWeb ABAP Connector

To connect ABAP to FastRWeb respectively to a specific R script hosted on the web server, class CL_HTTP_CLIENT is used. To avoid hard coded host and port information for the web server in the ABAP coding, the connection settings are stored in a "HTTP Connection to External Server" RFC connection which is maintained in transaction SM59. Using method CREATE_BY_DESTINATION of class CL_HTTP_CLIENT, a HTTP client for that destination can be created. The following image shows the configuration of the destination using the host and the port configured for the web server.

Parameters are transferred to the R scripts from ABAP as JSON strings in URL parameters. To convert internal values like structures and tables, the ABAP integrated ABAP JSON transformation functionality is used. Also the result is returned as JSON string, except in case a plot R script is called. Plot R scripts return the result as binary string.

For a comfortable way of calling R scripts without doing the HTTP call and the JSON transformation manually, an FastRWebABAPConnector adapter was implemented which encapsulates these things. The sources are available at GitHub. The adapter consists of interface ZZ_IF_FASTRWEB_CONNECTOR and class ZZ_CL_FASTRWEB_CONNECTOR.

The usage of the adapter can be seen in following dummy example.


" create FastRWeb Connector instance


DATA(lo_rcon) =


  NEW zz_cl_fastrweb_connector(


    iv_result_type = zz_if_fastrweb_connector=>mc_result_type_json


    iv_destination = 'Z_RSERVE_FP'  " RFC Destination pointing to web server


    iv_rscript_path = '/R/test' ).  " path to R script in web folder + name of R script w/o file extension



" set input parameter


DATA lt_parameter TYPE zz_if_fastrweb_connector=>mtt_key_value.


DATA(lt_test) = value ltt_test( ... ).


APPEND VALUE #( key = 'inputTable' value = REF #( lt_test ) ) TO lt_parameter.



DATA(ls_settings) = VALUE lts_test( ... ).


APPEND VALUE #( key = 'inputTest' value = REF #( ls_test ) ) TO lt_parameter.



" set result variable


DATA lt_result TYPE zz_if_fastrweb_connector=>mtt_key_value.


DATA lt_cluster TYPE ltt_result.


APPEND VALUE #( key = zz_if_fastrweb_connector=>mc_result_key


                value = REF #( lt_calc_result ) ) TO lt_result.



" execute R script


TRY.


  lo_rcon->execute_script( EXPORTING it_parameter = lt_parameter


                           CHANGING ct_result = lt_result ).


CATCH cx_t100_msg INTO DATA(lx_t100_msg).


  WRITE: / lx_t100_msg->get_text( ).


ENDTRY.


















Following points describes briefly the coding steps.

StepDescription
Create FastRWebConnector instance

A FastRWebConnector instance is created by creating a new instance of class ZZ_CL_FASTRWEB_CONNECTOR. Mandatory arguments are the result type, the destination and the R script path.

For the result type there are two options:

  • ZZ_IF_FASTRWEB_CONNECTOR=>MC_RESULT_TYPE_JSON: The result returned from the R script has to be interpreted as JSON string.
  • ZZ_IF_FASTRWEB_CONNECTOR=>MC_RESULT_TYPE_IMAGE: The result returned has to be interpreted as binary string, no JSON transformation is done in that case.

The destination identifies the RFC destination which points to the Rserve web server.

The R script path parameter defines the path to the R script and the name of the R script (w/o file extension). The path is relative to the web folder defined for FastRWeb. "/R/test" means, that a R script file "test.R" in folder "R" within the web folder has to be used.

Setting of input parametersInput parameters are transferred as key/value pairs to the connector. This allows a flexible number of parameters. The "key" attribute has to be set to the parameter name of a parameter of the "run" function in the R script. The "value" attribute has to be supplied with a reference of a variable (e.g. structure, internal table) defined in the ABAP coding. It has to be considered that the field names of e.g. a structure needs to be used in the R script to asccess the fields.
Setting the result parameter referenceAt the moment the adapter supports one result parameter identified by key ZZ_IF_FASTRWEB_CONNECTOR=>MC_RESULT_KEY in the result key/value parameter. The value attribute has to be set to a reference of a variable fitting the result type of the R script. Assuming as value a reference with fields "VAL1" and "VAL2" is set, the R script has to return a JSON value with the attributes "VAL1" and "VAL2".
Execute R scriptTo call the R script, method EXECUTE_SCRIPT of the FastRWeb Connector instance has to be called.

R Script parameter handling

Parameters are transferred serialized as JSON string via URL parameters to the R script. To be able to access the data in the R script, the JSON string has to be transformed to a R data format. The Conversion is done using the R library jsonlite which allows transforming from and to JSON strings. To be able to use the package, it has to be installed using install.packages("jsonlite"). In the script to be used the library needs to be loaded by statement library("jsonlite"). Transforming data from a JSON string to a R list is done using function "fromJSON". Function "toJSON" transforms a R list to a JSON string. This function is used to prepare the result data to return it to the ABAP caller (except for plots).

Transfer parameters to R script

The following example describes how an internal ABAP table transferred as JSON string in an URL parameter is accessed in a R script.

ABAP internal table

C1C2C3
100101Test 1
200201Test 2
300301Test 3

With the ABAPtoJSON functionality used by the FastRWebConnector implementation, the internal table is converted to following JSON string.


"DATA": [


  { "C1": 100, "C2": 101, "C3": "Test 1" },


  { "C1": 200, "C2": 201, "C3": "Test 2" },


  { "C1": 300, "C2": 301, "C3": "Test 3" }


]














Within the R script the JSON string is converted to a R list using function "fromJSON".


input <- fromJSON(inputParameter)














The columns of the list can be accessed like following. It has to be considered that the FastRWebConnector intorduces the "DATA" identifier. This is necessary as additional identifier required by the ABAPtoJSON functionality.


input$DATA$C1


input$DATA$C2


input$DATA$C3














Return result from R script

Returning data from a R script is done as JSON string too (again except for plots). The result data has to be transfomred to a list and then the list has to be transformed with function "toJSON" to a JSON string. Function "done" sets in the R script the return value. It has to be considered that the ABAP variable for the result needs to have the same structure than the R list. That is necessary to receive the data in ABAP and successfully transform it from a JSON string to the ABAP variable.


jsonString <-> toJSON(dataList)


done(jsonString, cmd = "html", type = "application/json")













Examples

The following examples can be found at GitHub. Beside the ABAP coding, the R script coding is contained in the files too (commented out at the bottom of the files).

k-Means clustering of customer data

The first example shows a simple k-Means clustering for customers based on their Life Spend, News Spend, Income and Loyalty. The example is the same as used by the SAP HANA Academy to demonstrate the k-Means algorithm implemented in SAP HANA PAL (test data for that can be found at GitHub - saphanaacademy/PAL: Predictive Analysis Library).

The complete ABAP coding can be watched at FastRWebABAPConnector/ZZ_TEST_FASTRWEBCON_KMEANS.abap · GitHub.

The R script looks like following.


run <- function(settings, customer){


    # load jsonlite library for JSON data handling (i/o)


  library(jsonlite)


 


    # load amap library for k-Means algorithm


  library(amap)


 


    # convert input parameters in JSON to list


  customerData <- fromJSON(customer)


  settingsData <- fromJSON(settings)


 


    # remove ID for clustering


  inputWoID <- data.frame(lifespend=customerData$DATA$LIFESPEND,


                          newspan=customerData$DATA$NEWSPEND,


                          income=customerData$DATA$INCOME,


                          loyalty=customerData$DATA$LOYALTY)



    # k-Means execution; no. of clusters set via parameter, max 100 iterations,


  # Euclidean method


  resultKMeans <- Kmeans(inputWoID,


                         centers=settingsData$DATA$CENTERS,


                         iter.max=100,


                         method="euclidean")



    # combine customer information with clusters


  customerAssignedToCluster <- data.frame(ID=customerData$DATA$ID,


   CLUSTER=resultKMeans$cluster)



    # convert result data frame to JSON


  customerAssignedToClusterRoot <- list(DATA=customerAssignedToCluster)


  resultJSON <- toJSON(customerAssignedToClusterRoot)


 


  done(resultJSON, cmd = "html", type = "application/json")


}










The example program simply displays the customers with the calculated cluster ID.

Plotting

The second example (FastRWebABAPConnector/ZZ_TEST_FASTRWEBCON_PLOT.abap · GitHub) shows how a simple plot is created. The visualization of the plot image in UI5 and Webdynpro would be easy, so the example displays the image on a classical dynpro screen which is a bit more "complicated" (screen definition and GUI status are not included in the Github repository).

Conclusion

With the usage of available open source components and a simple ABAP coding it is possible to use R scripts directly from ABAP. Customers w/o SAP HANA also have a chance to do lightweight statistical calculations and plotting with R without a complicated setup. Because with NW 7.50 the improved Push Channels with TCP support were released, the next step is to evaluate if it is possible to connect directly to the Rserve TCP/IP server, to be able to remove the HTTP component to create a slightly performance improvement.

2 Comments