Couch your Big Data using SAP Lumira and Couchbase

[shankarsgs]

SAP Lumira is indeed a tool built for Big Data. In my last blog, I was able to connect SAP Lumira to RethinkDB and get insights on that which you can read http://scn.sap.com/community/lumira/blog/2015/07/23/rethink-your-big-data-with-sap-lumira-and-rethin....In my quest for Big Data connectivity for SAP Lumira,  I have consumed data from another popular NoSQL Data store, Couchbase. In this blog, we will describe how to connect SAP Lumira to this NoSQL Data store, Couchbase.

We will be using a Custom Data Access Connector that connects to Couchbase and consumes the data into SAP Lumira. Here, we will be using the game simulation sample, which comes by default as a sample data along with the Couchbase installation.You can download the data from https://github.com/couchbase/couchbase-examples

The gamesim example

The gamesim example contains the results of a game simulation with scores and details about each gamer. It is loaded as “gamesim-sample” bucket as part of Couchbase setup. Typical data from that bucket would look like this.

Finding the needle in a haystack

Couchbase is built on top of JSON as its data store. Large volume of unstructured data can be stored in Couchbase very effectively. In order to visualize this unstructured data, we need to find and bring only the data that is needed to be used inside SAP Lumira. We will be using process called MapReduce, which maps our data and reduces the result down to meaningful data that is needed.

We will be creating views in Couchbase, which helps us execute Mapreduce on original dataset and return the results. We can create different views for different type of analysis,and also perform aggregation and calculation, using Javascript.

Here is a basic view that returns the necessary details about each player.

After publishing the view to production post testing, we can execute to see the result on the complete dataset.

Now that we have the required data, let’s go ahead and use it inside SAP Lumira.

Threading the needle

In order to consume that data in SAP Lumira, we will be making use of Couchbase API to expose the data. Pass the API URL found in the Filter results, to obtain the data via the API into the Couchbase Connector. You can download and install the Couchbase Connector into SAP Lumira from https://github.com/sgsshankar/lumira-extension-da-couchb

You can also add additional filter parameters and limit the number of results returned via the URL parameter. You can read more about views in the Couchbase documentation.

Download the couchb Connector from the github link mentioned above and install it in SAP Lumira. For Installation steps, please refer the SAP Lumira Data Access Connectors Developer Guide After successful installation of the Connector into SAP Lumira, select couchb from the External Data source to run the Couchbase data access Connector.

Input the API URL, Username and password if you have set it into the Connector and click on Ok to fetch the data. It would fetch the data that is emitted from the result field of the Mapreduce view.

Here, we are getting sample data of 10 rows for the demo. The data is fetched into SAP Lumira which we can then use to Visualize.

We have now got the data, let’s analyse some stats about the game.

Stitching the Data

Here is some Information about the game, presented as a SAP Lumira Storyboard.

We have now consumed data from Couchbase and analysed it in SAP Lumira. This is another hop in the Quest for my journey of exploring Big Data Solutions and integrating it with different solutions. This is just a beginning as Couchbase becomes more powerful with features like N1QL support, MapReduce Queries that can be made possible with the Connector.