Tutorial for creating a simple Custom Data Source.

[Former Member]

I recently developed a custom data source extension to access indicator data from the World Bank API. It was my first crack at the Design Studio SDK, and I thought I’d upload a tutorial blog to help those who are looking to create their own data extensions for the first time. I know there are already a few similar blogs out on the community already, so I am assuming no prior experience on the part of my readers. I am going to try and make it as detailed and simple as possible (while simultaneously keeping this post as short as possible).

The extension I have developed for this blog was the stepping stone to more versatile extensions which were used in an infographic which was presented at the SAP Analytics Infoday 2015 event on the 7^th of July. You can find a video of the final infographic here.

Before we start, you should have the SDK Guide, and should already have added the SAP provided SDK extensions to your Eclipse workspace.

Note: You can find the SDK Guide for version 1.5 here and the SDK extensions here.

Before I get into the extension, let me tell you a little about the World Bank data, and how data can be accessed from their API.

World Bank APIs

The World Bank Indicators API lets you programmatically access more than 8,000 indicators and query the data in several ways, using parameters to specify your request. Many data series date back 50 years, and can be used to create interesting applications. The Indicators API has three types of response formats: XML, JSON, and JSONP. I have used JSONP format to circumvent the Same Origin Policy issue.

Note: Further reading on the Same Origin Policy, and ways to work around it can be found here, here, and here.

There are four basic arguments the developer needs to form their data request:

1. Country/List of countries
2. Indicator
3. Year/Range of years
4. Output format

  Note: Find a more detailed description of the call structure here.

The figures below give an example of a sample request and its corresponding response.

Data Source SDK

Diving into the development of the extension, from a technical point of view, an SDK extension is an Eclipse plugin, which contains the following files.

1. Contribution XML file
2. Component JavaScript file
3. Component CSS file
4. Icon file
5. Script Contribution file
6. Additional Properties Sheet HTML file
7. Additional Properties Sheet JavaScript file

Of these seven files, only the Contribution XML file is required to develop an extension. It provides the definition of the SDK extension and its extension components. One of these components is the Component JavaScript file. This file contains the actual code for the extension, without which, let’s face it, the extension won’t really do anything. So for the rest of this post these are the only two files that I shall focus on.

Out of the remaining five files, the Script Contribution file, and Additional Properties Sheet HTML and JavaScript files can help in providing extra functionality to our extension but to keep it simple, I will not be describing their use in this post; I’ll cover them in a later blog post.

Contribution XML File

  The Contribution XML file specifies the SDK extension and all its extension components. SAP provides a documented XML schema definition file (sdk.xsd) that defines the format of the Contribution XML file. It is here where we provide Design Studio with information about our extension; its name, version, vendor. We also specify which files are included in the extension and where they’re stored.
Here is the contribution.xml for the extension described in this post.

<?xml version="1.0" encoding="UTF-8"?>

<sdkExtension

      xmlns="http://www.sap.com/bi/zen/sdk"

      id="com.sap.worldbankdatasource"

      title="Design Studio SDK Extension for World Bank REST API"

      version="1.0"

      vendor="TCS">

      <component

              id="WorldBankDataSource"

              title="World Bank Data Source"

              tooltip="Allows access to data from the World Bank API"

              icon="res/icon.png"

              handlerType="datasource">

              <jsInclude>res/js/component.js</jsInclude>

              <property id="countries" title="Countries" type="String"></property>

              <property type="String" title="Range" id="range"></property>

              <property type="String" title="Indicators" id="indicators"></property>

      </component>

</sdkExtension>

sdkExtension Element

The extension is defined by the sdkExtension element. Its important attributes are:

• title: Title of the SDK extension.
• id: Specifies the ID of the SDK extension to avoid conflict with its own components and with other extensions. It is recommended that you give it the same, fully qualified names as the folder name of the corresponding Eclipse project.
• version: Provides the version of your extension. Now, if you’ve read the manual you’ll see that the method to create a new extension in Eclipse is to copy an existing one within the workspace and then modifying the newly-created extension’s identifiers. When you do this, pay attention to the version. If you create a new extension from an old one, do not adopt the latter’s version. Start from “1.0”.
• vendor: Provides the vendor name of your extension. You can mention your name here, or your employer’s name, or you can simply leave the value as “SAP”. Please note, that the values of the vendor attribute in the contribution.xml file and the Bundle-Vendor attribute in the MANIFEST.MF file should match. Should there be a discrepancy, Design Studio will not load the extension.

component Element

The component element is a child element of the sdkExtension element. The extension component is specified by this element. Its important attributes are:

• id: Specified the ID of the extension component. This ID appended after the sdkExtension element’s ID form the JavaScript class name of your extension component.
• title: Title of the extension component.
• handlerType: Specifies the technology that implements this extension component. For custom data sources, this attribute must be specified as “datasource”.

jsInclude Element

It is a child element of the component element. It references the JavaScript file which is to be included with its parent extension component at runtime.

property Element

This element is used to specify properties of the extension element. Properties can be thought of as variables that can be set at design time. As you can see from the contribution.xml file, the properties I’ve defined are the same as the arguments that are used to make up the data request. These properties allow me to modify the data request at design time.

The important attributes of the property element are:

• id: ID of the property. The ID mentioned in the contribution.xml file translates to variable name of the property in the Component JavaScript file.
• title: Title of the property. This is the textual representation of the property which can be viewed at design time.
• type: Type of the property. Specify one of the followingint, float, boolean, String, Text, ScriptText, Color, Url.

Component JavaScript File

You implement a Component JavaScript class for each extension component. You can implement the Component JavaScript class using both JavaScript and jQuery (version 1.10.2 at the time of writing), as jQuery is included in the SDK framework.

There are two ways of implementing the Component JavaScript part of a custom data source:

• You can extend your custom data source from the DataSource JavaScript class, which is provided by the SDK framework. This method requires the developer to produce the data in Design Studio’s SDK JSON format.
• You can extend your custom data source from the DataBuffer JavaScript class, which is provided by the SDK framework. This class sits extends the basic DataSource JavaScript class. It offers you a more convenient way of implementing a custom data source.

We are going to be exploring the second method, extending the DataBuffer class.

Extension Component Lifecycle

Before we get into the intricacies extending the DataBuffer JavaScript class, it is important to understand how a component is to be structured.

For any extension component, the SDK framework performs the following JavaScript function calls:

• init()
• beforeUpdate()
• Update all extension component properties using their setter/getter functions
• afterUpdate()

  The order in which these functions are performed depends on whether or not the component has already been rendered. The distinction is explained in the image below.

Note: It is not imperative to implement all of these functions. For instance if the developer leaves out the beforeUpdate() function, the SDK framework will move forward and start updating the property values.

Extending the DataBuffer class

Custom data sources that extend from the DataBuffer JavaScript class share the same Component JavaScript API as other SDK extension components, which implement or call JavaScript functions like init, beforeUpdate, afterUpdate. However, there are a few additional JavaScript functions that are specific to custom data sources, which extend from the DataBuffer JavaScript class. These additional functions are:

Function defineDimensions

This function is used to set the dimensions of a custom data source. The result of this function can be thought of as nothing more than the structure of a crosstab. We specify for each dimension whether its members will be in the rows or columns of the crosstab.

The syntax of the defineDimensions function is:

defineDimensions(aoDimensions, oExternalMeasuresDimension)

It has two arguments. The argument aoDimensions contains an array of JSON objects, where each object is a dimension. The argument oExternalMeasuresDimension is optional. We may either define the measure dimension externally using this argument, or we can configure one of the dimensions in the argument aoDimensions to contain measures. The attribute which is used to specify whether a dimension contains measures is containsMeasures, and it must be set to true.

Given below is an example of the defineDimensions function.

this.defineDimensions([

                      {

                          "key": "year",

                          "text": "Year",

                          "axis": "COLUMNS"

                      }, {

                          "key": "Indicators",

                          "text": "Indicators",

                          "axis": "ROWS",

                          "containsMeasures": true

                      }, {

                          "key": "Country",

                          "text": "Country",

                          "axis": "ROWS"

                      }

                    ]);

As you can see it does not contain the oExternalMeasuresDimension argument, and that the Indicators dimension is configured to contain measures.

Function setDataCell

This function is called to set the value of a single data cell of the custom data source. The syntax of the function is:

setDataCell(aCoordinates, value)

The argument aCoordinates is an array which is used to specify the coordinates of the next argument value. The array may contain either dimension member names or dimension member indices. Using the dimensions we defined in the example above, let’s assume that GDP of India for the year 2013 was $1861801615477.9, then the corresponding setDataCell call will be:

setDataCell(["2013","GDP","India"], 1861801615477.9);

There are a few things to remember while using the setDataCell function

• The order in which we mention the members for each dimension must the same order in which the dimensions were defined.

  When populating your data source with data cells, you must strictly follow this sequence: Set the data cells left-to-right first, then top-to-bottom. Adding data cells randomly (with respect to their coordinates) may lead to an unusual arrangement of data cells. Refer the images below to better understand of the order in which the data cells must be set.

Function fireUpdate

This function is called to notify the SDK framework that the custom data source has been updated. Its syntax is:

                fireUpdate(bWillUpdateServer)

If the optional argument bWillUpdateServer is true, then the SDK framework also notifies the server on the backend of the change. This function is called after all the data cells have been set.

Other functions

There are two more functions provided by the SDK for the DataBuffer JavaScript class, they are clear and fillWithArray. The clear function is used to reset the custom data source to its initial state, and the fillWithArray function is an alternative way to set data into the custom data source, but can only be used if the data source contains 2-dimensional data.

Using the Extension in Design Studio

  Step 1: Add a Chart component and a Simple Crosstab component into the Analysis Application, and then add a Custom Data Source.

Step 2: Configure the property values for the Custom Data Source

Step 3: Assign the Custom Data Source to both the Chart and the Simple Crosstab components.

Step 4: Run the Analysis Application.

Conclusion and Future Development

So this is all that you need to know to put up a basic, functioning custom data source using a Web Service. The Data Source SDK allows you bring in data from almost anywhere as long as you can access data using JavaScript (or JQuery).

I have uploaded the  Eclipse project for this extension to GitHub here which you can add to your own workspace and test on Design Studio.

I will be building on this post with a couple more entries, where we will explore the Script Contribution file, and the Additional Properties Sheet HTML and JavaScript files. Maybe add some more functionality to the same use case. For instance, create a data source which has data for more than one World Bank Indicators.

My aim is to build up to a single, multipurpose data extension which can be used for the various types of charts used in this infographic

I hope this post was enough provide enough insight into the Data Source SDK to help you get started in creating your own custom data sources. Please feel free to ask any questions or leave any feedback in the comment section.

Cheers!

Update: I've uploaded Part two for this blog. It can be found here. I will be uploading Part three soon.