Insight into customer sentiment of two competing brands with HANA Text Analysis

[Former Member]

There are arguably some fantastic articles out there on Text
Analysis (TA) on SAP HANA and I have been struggling to think of another piece
to contribute to the community on how customers can use TA productively in
their environment. Recently while settling an age old dispute on which
competing brands stands out between the two, it dawn onto me why not let
technology answer this question for us and where better place to stream the
voice of customers if not from Twitter.

The use cases handed down here is to understand the
sentiment of two competing brands and in this case, Honda and Toyota. Raw Information
is sourced from Twitter and we have been collecting tweets for the past month
with no geographical limitation, which means tweets can come from as remote as Tristan da Cunha to the recent post
you made on Honda or Toyota. It was an interesting excise for us and the end
result was clearly visible, though I must admit, I won’t be making any
strategic decision on the results posted here due to the limited analysis that
was performed.

This article’s intention is to state the learnings that were
achieved, summarising it and proposing some of the possibilities that can be accomplish
with SAP TA with the guidance of a mature team in areas such as predictive, digital
marketing and web mining.

Acquiring the Desirable
Data

As with any brand with direct consumer facing products,
understanding firsthand about what people are saying is crucial and the correct
insight to react appropriately without delay from external marketing research team
can be something of an added advantage.  In
the absent of linguistic behavioural observation, which is the second frequently
used form of behaviour observation, TA supplements this study. Data acquired
from Twitter is free, is you already have HANA in your environment and running
SP005 of greater, you’re good to go.

This is the part where I want to state that I might be wrong
with the following information and happy for anyone out there to correct me and
credit you for the correction made. To the best of my knowledge and SDN
searches, I was unable to conclude any standard feature within the SAP suite of
products which allows a developer to extract information from Twitter using
standard API. I did come across an article where data was fed into SAP CRM and
another one using BO Data Services albeit slightly outdate. However, my opinion
is that, whatever method or mechanism that is use, it is irrelevant as all that
is required was to stream tweets from www.twitter.com
and store it into HANA tables. I have always been a fan of using .NET for its
rich support and wide community outreached and naturally it wasn’t hand to find
a ready to use twitter application such as LINQ2Twitter by @joemayo. I recommend that you check out the list of twitter libraries from their official website to find something that is more your flavour.

Simply by doing a key word search on Honda and Toyota,
Twitter was able to return a substantial amount of tweets within a single hour
at an average of 217 tweets – including retweets.  So the decision to look back at historical
tweets was not important as new tweets on Honda and Toyota was constantly
created hance it was decided to go down the path of using the Twitter Steaming API to capture
data and it also provided us the option to perform real time analysis.

The finished product was a client application developed in
.NET with a command prompt output to display some basic information such as the
actual message, username and location.

Once the information has been successfully steamed live from
Twitter, the next step was to obviously store it into a SAP HANA table and it
was as easy as extending it with SAP’s HANA SP008 released of the .NET Client Interface.
I previously wrote an article on this and you can refer to it here.
The below code snippet is a HanaCommand() function issued to the HANA database
to perform an insert statement with some of the desired result acquired from
Twitter.

Basic Data Modelling