After spending some time trying to load data using a C# ETL loader we created, I noticed that SAP exposes a "Bulk load" option in their driver. I couldn't find any examples on using it, so I figured I would post an example. Performance is pretty good, with the BatchSize = 1000, I am able to load 1 million records in about 10 seconds.


        private string dbConnectString;
        private HanaCommand dbConnection;
        public void LoadData(DataTable dt, string tablename, string[] columns)
            string[] fullName = tablename.Split('.');
            dbConnectString = ConfigurationManager.AppSettings["DatabaseString"];
            HanaConnection dbConnection = new HanaConnection(dbConnectString);
            HanaCommand da = new HanaCommand("SET SCHEMA " + fullName[0], dbConnection);
            HanaBulkCopy blkcmd = new HanaBulkCopy(dbConnection);
            blkcmd.BulkCopyTimeout = 1000;
            blkcmd.DestinationTableName = fullName[1];
            HanaBulkCopyColumnMappingCollection mappings = blkcmd.ColumnMappings;
            for (int i = 0; i < columns.Length; i++)
                mappings.Add(new HanaBulkCopyColumnMapping(i, columns[i]));
            blkcmd.BatchSize = 1000;

A few notes about implementation and some things I found:


1.The datatable has to have matching datatypes for the columns, but the rows can be all strings, so when you initially build your datatable you have to do something like this for each column:


DataTable dt = new DataTable();                                        
DataColumn dc = new DataColumn("col" + j.ToString(), typeof(double));


And specify if it is a double, string or DateTime etc.


2.The BatchSize has some strange behavior. I didn't play with it much, but if you don't define it then performance is really terrible. Not any better than inserting one record at a time. I found 1000 gives great performance, but I bet it could be even better, so if someone is able to do some metrics that would be great.


3. I am using the HanaBulkCopyColumnMapping in my example, but it is completely optional to use. If you don't provide the mapping, then column 0 in the datatable gets mapped to column 0 in Hana 1:1, 2:2 etc. The mapping is nice of course, because then you don't have to have the same number of columns or the correct order, I just pass the tables columns names that I am loading in as an array string in the example.


4. You will notice that I switch the Schema before loading. If you don't do that, then even if you pass in the fully qualified name (ex USERS.USER) the bulk loader will still try to load into its own schema.


5. The connection string is a pretty simple string that consists of something like this:



SAP HANA Customers & Partners, join us for the 2016 SAP HANA Operations & Developers Expert Summit! on May-24th, 25th at SAP America, Newtown Square, PA. Based upon the success of 2014, 2015 Operations & Developers Expert Summit, SAP HANA Product Management team is inviting IT Architects, Administrators, DBAs, and Developers this year again to the 2016 SAP HANA Operations & Developers Expert Summit. We wanted to hear from you firsthand about your SAP HANA Platform implementation experiences, use cases, and challenges in operations and development topic areas, what is working and what is not working, etc.

So if you have worked with any of the Operations and Development topics in SAP HANA platform areas, please join us on May-24th or 25th in Newtown Square, Pennsylvania. The plan for the event is to have a quick update presentation for each topic area of the SAP HANA Platform, followed by a round of feedback sessions with the product manager responsible for a specific HANA topic and let them know what are your greatest successes, and challenges with SAP HANA Platform Operations and Development areas. This is an exclusive event to get access to the SAP HANA Product Managers who can help you to influence the product strategy and roadmap based upon your direct feedback to them. Don’t miss this opportunity to meet the experts in person at the event.

During the event, you will be interacting with a small number of experienced hands-on experts, networking together with various HANA customers and providing direct feedback to SAP HANA product managers. To this end, the number of registrations will be limited, where all the registration request will be given careful consideration and we will contact you with a confirmation email to attend the event and the next steps.

Note: Please select the right event based upon your skill set and job experience in SAP HANA Platform areas. Only 1 registration to one of the events below  will be finally approved.

For HANA Administrators, Architects, DBAs:  Register here for the Operations Track of the Summit at Newtown Square, PA on – May,24th, 2016

For HANA Application Developers:  Register here for the Developers Track of the Summit at Newtown Square, PA on – May,25th, 2016

The tentative agenda for the Operations Summit is:

8:00 am – 9:00am


9:00 am – 9:15am

Welcome & Introduction

9:15 am – 10:30am

Topic Update Presentations I






Back Up & Recovery


Virtualization & Multi Tenancy

10:30am – 10:45am


10:45 am – 12:00 noon

Round Table Feedback Sessions I

12:00 – 1:00 pm

Lunch Break

1:00 pm – 2:30 pm

Topic Update Presentations II


Workload Management 


High Availability / Disaster Recovery


Hana – Hadoop, Vora


Landscape & Deployment Options

2:30 pm – 2:45 pm

Afternoon Break

2:45 pm – 4:45 pm

Breakout Feedback Sessions II

4:45 pm – 5:00 pm

Closing Remarks

5:00 pm – 7:00 pm

Networking Reception & Dinner


The tentative agenda for the Developers Summit is:

8:00 am – 9:00am


9:00 am – 9:25am

Welcome & Introduction

9:30 am – 10:30am

Topic Update Presentations I


  Web Tooling


  Core Data Services





10:30 am – 12:10

Round Table Feedback Sessions I

12:10 – 1:00 pm

Lunch Break

1:00 pm – 2:15 pm

Topic Update Presentations II




  Application Lifecycle Managment


  HANA Vora Development


  Predictive Analysis Library


  Text Analysis

2:15 pm – 2:30 pm

Afternoon Break

2:30 pm – 4:35 pm

Breakout Feedback Sessions II

4:45 pm – 5:00 pm

Closing Remarks

5:00 pm – 7:00 pm

Networking Reception & Dinner




The SAP HANA Product Management team in EMEA are also planning an Operations & Developer Summit of their own. Please find further details here.

When you’re preparing to install dynamic tiering, make sure to review the hardware and software requirements so you can plan your system accordingly.


Software Requirements


Dynamic tiering SP 11 supports the following operating systems:


Operating Systems on Intel-based hardware platforms:

  • SUSE Linux Enterprise Server (SLES) 11 SP2, SP3
  • SUSE Linux Enterprise Server (SLES) 12
  • Red Hat Enterprise Linux (RHEL) 6.5 or higher


Operating Systems on IBM Power system platform:

  • SUSE Linux Enterprise Server (SLES) 11 SP4 for IBM Power


However, dynamic tiering needs to be installed on the same operating system as HANA. For example, you can't have HANA running on SLES and dynamic tiering on RHEL, even though both are supported.


Hardware Requirements


Dynamic Tiering doesn’t have any specific hardware requirements. However, if you want to add a standby host, it needs to have the same hardware layout as the worker host that it's providing failover support for.


To see the most up-to-date list of system requirements for dynamic tiering, check the SAP HANA Dynamic Tiering Hardware and Software Requirements topic in the SAP HANA Dynamic Tiering: Installation and Update Guide.


Related Documentation on the SAP Help Portal:

Part 1: Setup HCP account and IDE, twitter account and download first tweets

As a student in SAP HANA Product Management I had to explore what can be done on HCP trial landscape, what runs out-of-the-box, which SAP HANA features are available – and which business needs can be met, in the end? So, here are the results.

In this series of blogs I am going to explain how to build an real-time social media analytic application using the SAP HANA cloud platform (HCP). I wanted to make use of predictive algorithms. A hotspot for predictive is marketing. Therefore, I chose this topic. Of course, it is not ready-to-use, like a product. But it is “ready-to-copy-and-learn” and I named it SmartApp. While digging deeper and deeper, of course I learnt a lot through a multitude of other blogs and SAP HANA Academy videos. Thanks to all of you! (I do mention them below.)

It is an application that you can follow along yourself. However first let’s have a quick look at the overall scenario of the SmartApp. We are going to access some real-time social media information. I would like to take twitter as the example so we can get information on specific tweets that we want to look for and will pull those tweets out in real-time using the twitter API. The idea for this use was fueled by a nice blog from a Chinese colleague of mine, Wenjun Zhou: kudos to you for this work!

We will process those tweets and load them in real-time into the HANA database. However the database is hosted on HCP. It was my ambition to use an HCP developer trial edition for as long as possible (until I need a feature that is only available on a dedicated HANA instance). Once we loaded the data into the database then we can do some further analysis on how to group or segment the different types of tweeters, maybe we to group similar people together, for example the group of people expressing positive or negative sentiments towards the product or company, and it would also make sense to differentiate whether they are influential because a lot of people follow them and their tweets get retweeted. We could identify that group so we can target some education program or reach-out program to those people. So, the idea was to do some predictive analysis in HANA itself in order to segment, cluster or to group similar kinds of people sending tweets. And finally, I wanted to make the results available through web services i.e., OData, XSJS native web services within the HANA platform in order to build a HTML5 application that could run on desktop and mobile devices to look at the analysis results. That’s the overall scenario of the SmartApp.

I don't want this blog to stretch like a chewing gum, so I divided the whole story into 3 parts:


1. Setup HCP account and IDE, twitter account and download first tweets

2. Refine processing of tweets with text Analysis, building web services and setting HANA Cloud Connector – this is the “heavy lifting” on SAP HANA from my point of view!

3. Do the UX work – use Analytics tools – make it nice


In the following sections, of course there is a lot of overlap with many existing blogs out there. Thank you to everybody I learnt from. Here are just a few of them:


Before we start to build the application, we need to register for HCP trial developer account and create Hana instance within the trial account (recommended to give instance name as “dev” and Database system as HANA XS (shared)since we can refer it in many applications).

Open SAP HANA Administration console perspective by clicking top right cornerCapture 11.PNGof eclipse.

Find API keys


First, identify the schema name to put application specific artifacts like tables, views:

To Identify and set Application schema keep DEV_xxx schema selected and hit onSQLSnip.PNGSQL button.


Make sure to set the database connection by clicking ondbconnection.PNG.

Copy the following statements to the sql console in eclipse:


To Identify application schema,copy:


To Identify current user,copy:


To Verify predictive analysis library is available,copy:


To Set application schema - paste in value below,copy:



Select & Run the statement to identify application schema:

SELECT SCHEMA_NAME FROM "HCP"."HCP_DEV_METADATA";  execute by clicking executionsnip.PNG.  As a result, the schema name is shown.

Then right click on the schema name and copy the cells. To set application schema – Paste the value below as


SET SCHEMA NEO_xxxxxxxxxxx; execute it to activate the schema.

Create Tweets table executing the below code:



             "id" VARCHAR(256) NOT NULL,


              "created" TIMESTAMP,


               "text" NVARCHAR(256),


               "lang" VARCHAR(256),


               "user" VARCHAR(256),


               "replyUser" VARCHAR(256),


               "retweetedUser" VARCHAR(256),


               "lat" DOUBLE,


               "lon" DOUBLE,


                PRIMARY KEY ("id")


Now we can find the tweets table in the schema (NEO_xxxxxxx) that we set initially.


Now install node.js to access both twitter API and HANA Database very easily that connects together.

To download the code of SmartApp from Git hub please click here.

Open downloaded SmartApp -> nodejs-> app.js in the editor may be notepad++.

In the code snippet we can see a variable called twitter where we actually specify the consumer_key, consumer_secret, access_token_key, access_token_secret. If you remember we already got this information when we registered our application in twitter. Now copy and paste the consumer_key into our file. Similarly do the same for other values also as shown below.

var twitter = new twitter({

       consumer_key: 'xxxxxxxx',

      consumer_secret: 'xxxxxxxxxxx',

      access_token_key: 'xxxxxxxxxxx',

       access_token_secret: 'xxxxxxxxxxx'

We have a second file called package.json in the nodejs with the list of dependencies like express, request, oauth, twitter, hdb that are used in our application. Now we can install all these with an npm and node installation command. Now click shift+rightmouse button on the nodejs folder in the SmartApp to open a command prompt in that nodejs folder then use “npm install” command to install all the dependencies that we needed in our application.

Then Restart the system (computer) so that nodejs could run without errors.

nodejs command.PNG

npm install.PNG

Go to nodejs folder should be able to see node_modules folder which actually consists the code for the modules setup and installed.

Setup the Tunnel

Let us setup the Database tunnel to connect directly to the HANA database using p-number trial id (pxxxxtrial). In order to setup the database tunnel open a command prompt on the tools folder in neo-java-web-sdk by shift+right click mouse button. Now execute the command “neo open-db-tunnel –h –i dev –a pxxxtrial –u pxxxx” (please enter the command manually don’t copy!) by replacing the account name with your pxxxxxtrial account and username with the pxxxx number.

After executing the command enter the password that you gave during the trial account creation. The tunnel opened and gives us the properties to connect to the schema.


Now go to the app.js file, edit the port as 30115 then copy the user, password, Schema name properties and paste in the app.js file respectively as shown below.

var hdb = hdb.createClient({

            host     : 'localhost',

            port     : 30115,

            user     : 'DEV_xxxxxxxxxxxxxxxxx',

            password : 'xxxxxxxxxxxxxx'



var hdb_schema = "NEO_xxxxxxxxxxxxxx";

Now let’s start the application by simply going into command prompt running in the nodejs folder. Goto the downloaded smartApp -> nodejs folder then shift+right click then select open command window.Type “node app.js” as command, application will run and we can see two messages as shown below.


Goto the Chrome browser enter “localhost:8888/do/start?track = productname or companyname”. 

Note: Enter the product or company name that you would like to track or make analysis. 


Now we can see in nodejs command prompt that tweets are loading into HANA database. Go to the tweets table in eclipse and refresh by clicking on top right corner. We can see some tweets that are loaded in to table.


If you want to stop loading the tweets go back to chrome browser then type “localhost:8888/do/stop”.

Note: While working with tunnel it must remain open for nodejs in order to be connect to the Hana instance. Since it’s a trial account this tunnel only lasts for 24 hours at that time you need to reopen it. However it’s fine for development and testing purposes. When using Productive HCP able to let things run 24/7.

Thanks for reading. Hope you enjoyed the blog….

Next Steps

In the next blog will perform some text analysis on Tweets data, Setup SAP HANA Cloud Connector and Configurations, Building web services using HANA Native development and setup authorizations. I will insert the link in here as soon as it is available!

The UI theme designer is SAP's browser-based tool for cross-theming scenarios.Create one consistent theme that applies to various SAP UI clients and technologies.Theme designer supports only three deployment options (ABAP, Enterprise portal and HCP) but you can apply custom theme which you have developed using UI theme designer accross various UI clients and technologies.Below picture gives you an overview on supported platforms and UI technoloies.I would like to bring on more clarity on the steps being followed to apply a custom theme for any HANA XS based applications on both cloud and on premise which uses SAP UI5 as front-end technology and hence this blog.. You must also take care for the UI5 versions. They should be the same on both systems. Otherwise the theme can be broken.


Application used in this blog: SHINE Application




Create your custom theme as per your requirment. Export your theme(.zip file), the exported theme is saved on your local machine. Please follow few blog below to create custom theme,

Step1:- Create and Import Custom Themes to HANA System:-

  1. You can create custom themes using the UI theme designer and import these themes into your SAP HANA system. In the UI theme designer tool, create and export a custom SAPUI5 theme that you want to use for your XS based application. A .zip file containing the exported theme is saved on your local machine.
  2. Import the exported theme into your project in SAP HANA Studio/SAP HANA WEB IDE(Copy the contents of the .zip file into your project)
  3. To register the theme in the THEME database table using the table import mechanism, create the following files in your project:



2.png  1.jpg





   4. Activate the .hdbti and .csv files by choosing Start of the navigation path and activate it.





Step2:- Applying Theme Via  Bootstrap Configuration

Insert the following in the SAPUI5 bootstrap script tag:(index.html file)


data-sap-ui-theme:  Enter the theme ID you have provided when you exported your theme from the UI theme designer to your local hard disk.

data-sap-ui-theme-roots:  Enter the theme ID and the application path you used for your mapping in the following syntax:

{"<themeId>" : "/<application path>/UI5/"}




Now check your HANA XS based application with custom theme applied



Further Reading please refer below documentation,


Get back to me if you need more information

I was not planning on this post but I got a few questions from some colleagues as to how to expose CDS views via XSOData so here it is


As always, thank you for reading this post.


Before creating the xsodata service, let's start by assuming you have created a CDS document with a table / view and you are ready to expose your data model. If you have not done those steps, please refer to my first CDS blog to create a CDS document and build a table / view



1) The first step is to create an .xsaccess file so we can expose our service. Make sure you set the property expose: true and your also need to set its authentication method. In my case, I do not want to add authentication as I am using HCPtrial, however, you may use null (for no authentication), Basic (for username and password), Form, etc. Please use the XS Admin tool for other authentication methods such as SAML, X509 certs, etc. check out the official documentation for xsaccess files


Also create an empty file .xsapp


2) create your .xsodata file (I am exposing the view I created on my blog 2) when exposing an end point on your xodata service, make sure your endpoint contains a KEY, otherwise, you may use a dynamically generated key as I did below




3) once you created your .xsodata file and have successfully activated it, run your service by clicking on the green icon with an arrow (see screen shot above). Your service will run on a separate tab on your browser and will display the XML of your service. One thing to note here is that my endpoint on the .xsodata service is exposed as "EVENTS" and that's exactly the endpoint on my XSOData service on the browser (screenshot below)



4) your next step would be to explore the end point, and the data that will be returned, etc. I am not able to run my service due to insufficient privileges since i am running this service on my HCP trial account, but if you are able to activate your service and you can query it from your browser, then you should be able to see the records being returned by your odata service.




5) for a JSON formatted response, please include $format=json at the end of your service URL.

a). URL_HOST/path/to/service/ENDPOINT?$format=json


you can also perform other query operators by appending query string paramaters to the end of your URL endpoint.


here are a few:

$top=1  for the first record

$filter=COLUMN eq 'value for filtering results where COLUMN equals 'value'

$format=json for your JSON formatted response

$orderby=COL_A for ordering the response by COL_A


for more OData operations and syntax, please visit the odata official documentation




thanks again and hope this has been helpful for others. As always, please share your experiences and struggles and feel free to ask questions.

Hello, In this blog post, I will showcase how to create a table and a view via the Core Data Services (CDS).

In my example I will be using the HCP trial landscape and the Web Development Workbench editor.


1) Create a CDS file ( .hdbdd extension)

2) Add some (correct) content and activate the file




3) Go to your schema and open the tables folder to see the newly generated table.



4) Perform a select statement - as expected, initially your table will be empty


5) Right click on the table, select “Generate Insert”   or type it (if you want to make spelling mistakes)


6) Your generated insert statement is opened on a new tab. then, you can save it as a .sql file (selct a location on your package)


7) I opened the sql file, copied and pasted into the SQL console so i could run it. Notice console output successful insert message, "Rows affected 1" after I ran my insert statement.



8) after I was able to insert my statement, then I went back to my select statement and I ran it again to verify my record was in fact inserted. As a result, you can see my first record in the table




9) After a few inserted records, we can see the table again with more rows populated.




10) Since I have a table, then I want to create a simple view. I went back to my CDS file and I declared a view. Then I re-activated my file.




11) Once the hdbdd file was activated, then I went back to my Catalog and opened the Views folder where my newly created view was saved.




12) Once again, I did a right click on the view and generated a select statement.( I avoid typing my SQL as much as possible so that I do not end up having to type it over and over before I get the correct spelling but up to you to try to get the name right the first time.




13) My select statement was generated




14) After the statement was generated I ran it (I changed it a little but the one above should have returned the same number of records since I only have 6 on my example. See output below


* Be careful if you are doing this on a table with a lot of records as you may select *






Thins to remember about CDS:

1) Stored procedures and sequences cannot be created in CDS (as of SP10)

2) CDS files may contain one or more entities or views and when it is activated, your entities, views, etc are directly created as design time objects on your schema and are ready for use.

3) HCP trial ( in your Web Development Workbench) is very helpful for learning how to use CDS ( as you may also use HANA Studio)


Things to forget:

1) Forget to re-run SQL statements when promoting your code from one environment (such as dev) into another environment (such as QA or Prod). CDS plays very well with HALM during migrations.

2) Forget to think you are alone on this journey, you have an entire SCN group behind you

3) (I forgot 3 - i am working on the next exercise on my Workbench, aren't you?)



Again, this is a very simple document explaining how to create a table and a view via CDS. Please share your experience or struggles with CDS. Happy programming. For a more advanced scenario, please read part 2 of this blog where I showcase a view with a 1 to 1 association on a table



Here are some helpful links I used while writing this blog:


create a CDS document


create a CDS view

First of all, thank you for reading part 2 of my blog on CDS. For a basic example, please read part 1 of this blog.


On part 2, I wanted to expand into creating another table with an association. On this post, I will explain how to create a simple 1 to 1 Association, however, CDS can further support associations such as 1.. many, etc.


1) On my new table HCCAsugEvent below, I created a column EventPresenter with an association back to HCCAsugPresentations (my first entity). Notice that the key on my HCCAsugPresentations will become my key for the associations. I did not have to explicitly state it but it was is understood on my next step.




2) I went back to the workbench Catalog and opened the SQL Console. I ran a Select query to verify the table was indeed created.




3) After the table was created, I inserted a few records (no picture of the insert, but it was easily done by right clicking on the table and then selecting "Generate insert". One nice thing about the workbench I discovered during the exercise is that I can double click on the table and its definition will be shown. I knew about this feature from HANA Studio but it was also nice to see it on the workbench (Thanks SAP teams!)


My next step was to create a view for the 2 tables I created. Initially, I created a select query with an inner join. I knew this is a way to do it in SQLScript, but I wanted to also achieve this from my CDS view involving at least 2 tables.




Next is my approach to create a similar view via CDS. The great thing about creating an association via CDS and trying to do a JOIN on my view is that my entity knows about the association so that I do not need to invoke my associated table. The column that has the relationship knew about the association and it lets me callt he columns from the other table via its COLUMN_ON_TBL.COLUMN_FROM_ASSOCIATED_TBL property as shown below. I added at least one more column from my joined table just to showcase the easiness of creating views via CDS. I am becoming a fan!




Here is the output





Thank you for your time reading this blog and feel free to post your questions or share your thoughts about your CDS experiences.



Here are some helpful links I used while writing this blog:


create a CDS document


create a CDS view



as per request... someone asked me how to expose this view or data model via xsodata , here is how on my next blog post. Good luck on your scenario!

A sample example for XSOdata  with Navigation Property

Scenario of Purchase Order(PO) Header and Item: In a single service we can navigate to Item level from Header to Item with Association .

Lets start with very basic example of VBAK (PO Header), VBAP(PO Item).

Example Code in a .XSOdata file


service namespace "workshop.aisurya.XSOdata" {
"<YourSchema>"."VBAK" as "POHeader" with ("MANDT","VBELN","ERDAT") navigates ("ToPoItemAssctn" as "ItemRef");
"<YourSchema>"."VBAP" as "POItems" with ("MANDT","VBELN","POSNR","MATNR","ARKTX"); 
association "ToPoItemAssctn" principal "POHeader"("VBELN") 
multiplicity "1" dependent "POItems"("VBELN") multiplicity "*";

Below Image for reference




Activate the changes and Run As XS Service.


Lets check the Metadata of service.






As highlighted in the image(ItemRef) is a navigation property from Header to Item level.


Lets test the service only for 2 records by providing query option as (top=2).





Here ItemRef is the navigation property of corresponding PO Items with URI. We can explicitly open in new tab and get the result also .


Now lets check Expand option for Showing Header and Corresponding Items.


$Expand Feature to fetch Corresponding Item along with Header.




POHeader (MANDT='800', VBELN='0000004970')


Now this service can be consumed in different UI ‘s  depending upon the requirement.

Would you like to understand how to optimize pricing and triple your profit in just a few minutes?


Of course you would! Keep reading to learn how one organization has built an R custom component that allows business users to access a powerful optimization algorithm quickly and easily.

Input Data
For this particular use case, the pricing department for our manufacturing company would like to know how to best price each of our products for next year. For this simple case, we’ll use 9 products, and include the product name (or SKU identifier), the current annual volume sold, current price of the product, and current profit margin.


Some of the products are currently unprofitable (as evidenced by the negative profit margin), so management would like to re-set prices for next year to ensure that:

1.) No product has a price increase or decrease of over 10%

2.) Any product that is currently unprofitable has a minimum price that is break-even for profit

3.) Overall, given the same volumes as this year, total revenue will increase by 2-4%


After importing the dataset we can build some basic visualizations to help understand our current product mix. A few examples are below, showing that Wallets, Necklaces, Hats, and Jackets account for the highest volumes of products sold, while Pants and Earrings are the highest priced (per unit). The stacked bar charts show how much revenue and profit each product accounts for—consistent with the profit margins listed in the original data, several products account for negative profit.


    I came across a number of text mining examples and all them where dealing with text information that is not from my area of expertise, for example, microbiology. Such examples are not catching your attention because you can not really tell if text mining is working based on your knowledge. I created SAP TechEd 2015 Session Catalogue data to play with HANA Text Mining. Any SAP professional is familiar with this text data and can make his opinion about HANA Text Mining.

    So here is my data. As you can see there are SESSION and TITLE information fields, CATEGORY field that classifies each document and DESCRIPTION field which contains text data for text mining (full-text index is built on this field).


I prepared a number of examples. Lets see text mining in action.


Find Similar Documents

For example, find session similar to DEV260 'Building Applications with ABAP Using Code Pushdown to the Database'



Find Relevant Terms

For example, find terms relevant to DEV260 'Building Applications with ABAP Using Code Pushdown to the Database' session



Find Related Terms

For example, find terms related to 'Fiori'



Find Relevant Documents

For example, find documents relevant for term 'Fiori'



Categorize Documents

For example, you have a new session for which you have to assign to a proper category. I took SAP TechEd 2014 DEV161 'SQLScript – Push Code Down into SAP HANA to Achieve Maximum Performance' session which belongs to 'Development and Extension Platform for SAP HANA and Cloud' category and classified it using 2015 SAP TechEd catalog documents. Lets see if the document will be correctly classified.


As you can see the document was classified correctly.


You can import attacted delivery unit into you HANA system (change extenstion to *.tgz first) and play with examples and data.

Once you delivery unit is imported you will have following objects in tm_demo package created


Note: described examples are in query.sql file


Excecute install.sql script to assign proper authorizations, fix data in session table data and create full text index for text mining.

Following catalog objects will be created


Note: for text mining function to work correctly you need to be on HANA SPS10

The SAP HANA Data Warehousing Foundation (DWF) option is a series of packaged tools for large-scale SAP HANA installations, which support data management and distribution within a SAP HANA landscape. With SAP HANA Data Warehousing Foundation you can achieve smart data distribution across complex landscapes, optimize the memory footprint of data in SAP HANA and streamline administration and development, thereby reducing TCO and supporting SAP HANA administrators and data warehouse designers.


In this blog I will focus on Data Lifecycle Manager (DLM).  Please note there are other tools as part of the DWF option that we will not look at during this blog. 


SAP HANA is the core platform in any SAP Big Data target architecture. One of the challenges you will face with any Big Data Architecture is managing tiered storage.   You need to make intelligent decisions on how to optimize and balance price and performance.


For the most part in the past you had to manually manage and develop your tiered data processes including: data relocation, monitoring, logging, scheduling, testing and harmonizing tiers.   I will show how DLM addresses these challenges.  This is not a training session.  The purpose is to give you a high-level understanding of the benefits of using DLM as part of your Big Data tiering/archiving strategy.  You should get a better understanding on how this can reduce cost and complexity while simplifying administration and development.


Below is a high level view on the DLM process:



Before we get started, let’s have a look at the table/data that we will be creating a data relocation profile for.  The table contains data that is the result of a computation over the Call Detail Records (CDRs) generated by the Telecom Italia cellular network over the city of Milano. CDRs log the user activity for billing purposes and network management. Below is simply the record count by country.



1 - Once you have DLM installed and configured you can access the xs application via

http(s)://<HOST>:<HANA XS PORT/sap/hdm/dlm/index.html


Here is a screen shot of the home page.  It’s a great place to see a quick snapshot of the last data relocation runs and node utilization forecast.





2 – The first thing to take care of will be to add a storage destination. Currently DLM supports 4 different storage destinations:

  1. SAP IQ over SDA
  2. HANA Dynamic Tiering Local
  4. Deletion Bin

Each of the different storage destinations has prerequisites.  To use HANA Dynamic tiering you need to have configured extended storage for HANA.  For Spark SQL you need the Spark Controller and Hadoop configured.DLM_Destination.png

3 – Once you have your storage destination configured and activated you will be able to create Lifecycle Profiles.  The below profile is based on a source HANA table.  The destination will be spark.  All data is in the hot store(2459324).




4 – Once you have your profile created and source/targets identified you need to create a rule that DLM will use to tier the data.  As seen in the below screen shot I have defined the rule to move any record with COUNTRY_CODE = ‘40’ to cold storage.  The editor in real-time gives you information on the number of rows effected by the newly created rule.  In a production environment your rule would most likely be dynamic based on the date.




5 – Once your rule has been defined and saved DLM gives you the ability to “Simulate” the data relocation.




6 – Below you can see the simulation shows 100,740 records will move from Hot to Cold.



7 – If you are confident in the data relocation simulation you can move to scheduling the process.  For this blog I’m going to trigger the process manually.



8 -  The logs section provides an area to monitor the jobs progress in real-time as well as look at previously executed jobs and the associated steps.  Below you can see the detailed steps and associated status for the CDR job we manually executed.



9 – Now that the data has been relocated to cold storage we can take advantage of some generated objects that DLM has created to optimize access between the hot and cold data. Below you can see 2 new objects created by DLM.

                  1 – Virtual table

                  2 – Union view that combines both hot and cold data.



10 – Selecting from the Virtual table should only show the data that has been relocated to Hadoop.



11 – Selecting from the DLM generated view returns us the data located in both HANA and Hadoop.



Conclusion: As you can see managing your data across tiers becomes seamless and integrated with DLM.  You can take advantage of the DLM cockpit to create, mange and monitor all your data relocation jobs.  The generated objects should help your applications and analytics take advantage of all your enterprise data regardless of its temperature/age.


Next Steps: In part 2 of this blog series we will take a deeper look at the Data that was relocated to Hadoop and how we interact with it using VORA.

    Sentiment analysis can be fine tuned using custom dictionaries, for example, sentiment polarity changed  or additional sentiments extracted. In my blog I will explain how to archive something different - link sentiment to correct topic.

    I run Voice of Customer text analysis against some reviews with standard EXTRACTION_CORE_VOICEOFCUSTOMER configuration. What I found is that "microwave safe bowl" was incorrectly interpreted as WeakPositiveSentiment.



"safe" sentiment was linked to incorrect topic e.g. "a microwave" instead of "bowl".  I do not care about a bowl as such and the sentiment just screw up my analysis. Excluding "safe" adjective from custom dictionary looks like a simple solution, but solves one problem and creates another (what if I need to extract "microwave is safe" WeakPositive Sentiment). The right solution is to define a new neutral sentiment "microwave safe" adjective in custom dictionary. This way "safe" sentiment is linked to the right topic e.g. "bowl".


Then reference custom dictionary in custom configuration


and finally recreate text index


If I query against $TA table again, I will see that sentiments are correctly extracted


     You can try described demo in your system. Import TA_DEMO.tgz transport (rename TA_DEMO.tgz.txt to TA_DEMO.tgz first). ta_demo package will be created


and finally execute install.sql script. The latter will create REVIEW table with data, grants proper authorization to TA_DEMO schema and creates text index with standard EXTRACTION_CORE_VOICEOFCUSTOMER configuration.



Streaming Lite is a light-weight version of the Smart Data Streaming ("SDS") server designed to run on IoT gateways to process data close to the source. A great example use case for applying complex event stream processing on the edge is a Worker Safety demo that the SAP IoT team has built in cooperation with Intel. In this reference sample, data from gas, motion and heart rate sensors is continuously processed through a Streaming Lite project to detect and react to unsafe situations. The first level of alert is generated when a dangerous gas level is detected. In this case an alert is generated for both the workers to evacuate the area and for the supervisor initiate a response. The second level of alert is generated when the pattern of a dangerous gas level is detected followed by detecting from the motion sensor that the worker is now prone. This combination of events results in a more urgent alert to respond to the immediate worker safety event.

Intel has provided a solution brief titled Intel and SAP IoT Bring Actionable Insights to the Edge that describes this reference use case in more detail.

To help you learn how to start building Streaming Lite projects, we have put together a comprehensive set of tutorials available in Learning how to use Streaming Lite with HANA Smart Data Streaming.

SAP Notes are great sources of information on issues occurring in the code or documentation of a product. Use them to find out what issues impact a particular release and how you can work around them.


When can SAP notes help you?


  • You’re considering whether to upgrade your system to a newer version and you want to know what issues affect that version. Look up the SAP Notes for the product so you can plan and work around them quickly and efficiently.
  • You recently upgraded your system and you suddenly see an error message you’ve never seen before. Search for SAP Notes with that error message text to find out how to get around the issue.
  • You’re stuck on a particular task and you’re not sure what to do next. Look at the SAP Notes for the product – it’s possible that one of the notes details something missing from the documentation that can help you work around the issue.
  • You’re looking for sizing information - dynamic tiering, for example, has an SAP Note that summarizes current sizing recommendations.


Where can you find SAP notes?

  1. Go to the Support Portal.
  2. Select Knowledge Base & Incidents in the top menu, then SAP Note & Knowledge Base Article Search.
  3. Click the Launch the SAP Note and KBA search button.
  4. Enter the product name in the Search Term field to find all notes for that product.




OR, if you know the number of the note you’re looking for, simply type it in the Number field in the top right corner.


TIP: Dynamic tiering has a central SAP Note for each release that lists all notes associated with that product version. For example, if you want to find the central note for the SPS 11 release, search for "dynamic tiering” and look for the note titled “SAP HANA Dynamic Tiering SPS 11 Release Note.”




You can also find SAP Notes on the Help Portal


The Help Portal has a direct link to the current release’s central SAP Note. To find the central SAP Note for dynamic tiering:

  1. Go to the SAP HANA Dynamic Tiering documentation on the Help Portal.
  2. You can find a link to the central SAP Note for the release under the What’s New – Release Notes section. This note has direct links to all issues actively impacting this product version.

For example, for SAP HANA dynamic tiering SPS 11, the central note is


Did you know that…?


Once an issue is resolved in a particular version, the SAP Note is archived and is no longer publicly available through that version’s central SAP Note?


What has your experience been with SAP Notes?


Have you had issues finding them? Do you find the content helpful or do you think it could be improved in some way? Feel free to post any comments or feedback regarding smart data streaming SAP Notes.


[Writer's Note: Thanks to Claudia Gosav, who wrote the original version of the post for Smart Data Streaming and granted permission for me to adapt and rebrand it.]


Filter Blog

By author:
By date:
By tag: