1 2 3 Previous Next

SAP CRM: Marketing

37 Posts

This tutorial is intended as a guide for the creation of demo/test data only. The sample script provided is not intended for use in a productive system.


The following tutorial explains a way of harvesting twitter data through GNIP. The pre-installed Python Interpreter from the SAP HANA client is used to execute a Python script from SAP HANA Studio. The script harvests the data from GNIP and extracts the useful data out of it and stores these details into Business Suite Foundation database tables SOCIAL DATA and SOCIALUSERINFO. Currently the script runs infinitely. If you want to stop harvesting the data, you can manually do it by stopping the execution of this script in the SAP HANA Studio. You can however modify the script to run for a specific period of time. To run the script, you will also need to make a few customizing and configuration settings in order to use the Pydev Plugin in SAP HANA Studio.


Make sure that the following prerequisites are met before you start out :
• Installation of SAP HANA Studio and SAP HANA Client
Install SAP HANA Studio and SAP HANA Client and apply for a HANA user with Read, Write and Update authorization for foundation database tables   SOCIALDATA and SOCIALUSERINFO

Create a GNIP account

Data Stream configuration in your GNIP account
Create a data stream for a source (like Twitter, Facebook, etc…) in your GNIP account. Remember, using a data stream you can harvest data from only a single source. So you should have different data streams for different data sources. After creating a data stream, define the rules in the ‘Rules’ tab to filter the data that you are getting from GNIP. For writing the rules refer the link : http://support.gnip.com/apis/powertrack/rules.html

1. Configuring Python in SAP HANA Studio Client
Python version 2.6 is already embedded in SAP HANA client, so you do not need to install Python from scratch. To configure Python API to connect to SAP HANA, proceed as follows.
1. Copy and paste the following files from C:\Program Files\SAP\hdbclient\hdbcli to C:\Program Files\SAP\hdbclient\Python\Lib
                a. _init_.py
                b. dbapi.py
                c. resultrow.py

2. Copy and paste the following files from C:\Program Files\SAP\hdbclient to C:\Program\Files\SAP\hdbclient\Python\Lib
                a. pyhdbcli.pdb
                b. pyhdbcli.pyd
In Windows OS, by default the installation path is C:\Program Files\SAP\.. for a 64 bit installation SAP HANA Studio and SAP HANA Database client

If you opted for a 32 bit Installation, the default path is C:\Program Files(x86)\sap\..

2. Setting up the Editor to run the file
2.1. Install Pydev plugin to use Python IDE for Eclipse
The preferred method is to use the Eclipse IDE from SAP HANA Studio. To be able to run the python script, you first need to install the Pydev plugin in SAP HANA Studio.
                    a. Open SAP HANA Studio. Click HELP on menu tab and select Install New Software
                    b. Click the button Add and enter the following information
                       Name : pydev
                       Location : http://pydev.org/updates

                   c. Select the settings as shown in this screenshot.
                       d. Press Next twice
                         e. Accept the license agreements, then press Finish.
                         f. Restart SAP HANA studio.

2.2. Configure the Python Interpreter

In SAP HANA studio, carry out the following steps:
     a. Select the menu entries Window -> Preferences
     b. Select PyDev -> Interpreters -> Python Interpreter
     c. Click New button, type in an Interpreter name. Enter in filed Interpreter Executable the following executable file C:\Program Files\hdbclient\Python\Python.exe. Press OK twice.

2.3. Create a Python project

In SAP HANA Studio, carryout the following steps:
     a. Click File -> New -> Project, then select Pydev project
     b. Type in a project name, then press Finish
     c. Right-click on your project. Click New -> File, then type your file name, press Finish.

Customizing and Running the Script

1. Customizing the python script

Copy and paste the below provided code into the newly created python file. Enter the values for the below parameters in the file.
     a. URL – unique url for the datastream you have created in your GNIP account
          (For ex : 'https://stream.gnip.com/accounts/<GNIP_USERNAME>/publishers/<STREAM>/streams/track/dev.json')
     b. username_gnip – your GNIP account username
     c. password_gnip – your GNIP account password
     d. server – HANA server name (Ex : lddbq7d.wdf.sap.corp)
     e. port – HANA server port
     f. username_hana – HANA server username
     g. password_hana – HANA server password
     h. schema – schema name
     i. client – client number

import urllib2
import base64
import zlib
import threading
from threading import Lock
import sys
import ssl
import json
from datetime import datetime
import calendar
import dbapi
from wsgiref.handlers import format_date_time
from time import mktime
CHUNKSIZE = 4*1024
NEWLINE = '\r\n'
URL = ''
username_gnip = ''
password_gnip = ''
HEADERS = { 'Accept': 'application/json',
            'Connection': 'Keep-Alive',
            'Accept-Encoding' : 'gzip',
            'Authorization' : 'Basic %s' % base64.encodestring('%s:%s' % (username_gnip, password_gnip))  }
server = ''
port =
username_hana = ''
password_hana = ''
schema = ''
client = ''
socialmediachannel = ''
print_lock = Lock()
err_lock = Lock()
class procEntry(threading.Thread):
    def __init__(self, buf):
        self.buf = buf
    def unicodeToAscii(self, word):
        return word.encode('ascii', 'ignore')
    def run(self):
        for rec in [x.strip() for x in self.buf.split(NEWLINE) if x.strip() <> '']:
                jrec = json.loads(rec.strip())
                with print_lock:
                    verb = jrec['verb']
                    verb = self.unicodeToAscii(verb)
                    # SOCIALUSERINFO DETAILS
                    socialUser = jrec['actor']['id'].split(':')[2]
                    socialUser = self.unicodeToAscii(socialUser)
                    socialUserProfileLink = jrec['actor']['link']
                    socialUserProfileLink = self.unicodeToAscii(socialUserProfileLink)
                    socialUserAccount = jrec['actor']['preferredUsername']
                    socialUserAccount = self.unicodeToAscii(socialUserAccount)
                    friendsCount = jrec['actor']['friendsCount']
                    followersCount = jrec['actor']['followersCount']
                    postedTime = jrec['postedTime']
                    postedTime = self.unicodeToAscii(postedTime)
                    displayName = jrec['actor']['displayName']
                    displayName = self.unicodeToAscii(displayName)
                    image = jrec['actor']['image']
                    image = self.unicodeToAscii(image)
                    # SOCIALDATA DETAILS
                    socialpost = jrec['id'].split(':')[2]
                    socialpost = self.unicodeToAscii(socialpost)
                    createdbyuser = socialUser
                    creationdatetime = postedTime
                    socialpostlink = jrec['link']
                    creationusername = displayName
                    socialpostsearchtermtext = jrec['gnip']['matching_rules'][0]['value']
                    socialpostsearchtermtext = self.unicodeToAscii(socialpostsearchtermtext)
                    d = datetime.utcnow()
                    time = d.strftime("%Y%m%d%H%M%S")
                    creationdatetime_utc = datetime.strptime(postedTime[:-5], "%Y-%m-%dT%H:%M:%S")
                    creationdatetime_utc = creationdatetime_utc.strftime(("%Y%m%d%H%M%S"))
                    stamp = calendar.timegm(datetime.strptime(creationdatetime[:-5], "%Y-%m-%dT%H:%M:%S").timetuple())
                    creationdatetime = format_date_time(stamp)
                    creationdatetime = creationdatetime[:-4] + ' +0000'
                    if verb == 'post':
                        socialdatauuid = jrec['object']['id'].split(':')[2]
                        socialdatauuid = self.unicodeToAscii(socialdatauuid)
                        socialposttext = jrec['object']['summary']
                        socialposttext = self.unicodeToAscii(socialposttext)
                        res = client + '\t' + socialmediachannel + '\t' + socialUser + '\t'  + socialUserAccount + '\t' + str(friendsCount) + '\t' + str
(followersCount) + '\t' + postedTime + '\t' + displayName + '\t' + displayName.upper() + '\t' + socialUserProfileLink + '\t' +image
                    elif verb == 'share':
                        socialdatauuid = jrec['object']['object']['id'].split(':')[2]
                        socialdatauuid = self.unicodeToAscii(socialdatauuid)
                        socialposttext = jrec['object']['object']['summary']
                        socialposttext = self.unicodeToAscii(socialposttext)
                        res = client + '\t' + socialmediachannel + '\t' + socialUser + '\t'  + socialUserAccount + '\t' + str(friendsCount) + '\t' + str
(followersCount) + '\t' + postedTime + '\t' + displayName + '\t' + displayName.upper() + '\t' + socialUserProfileLink + '\t' +image
                    hdb_target = dbapi.connect(server, port, username_hana, password_hana)
                    cursor_target = hdb_target.cursor()
(?,?,?,?,?,?,?,?,?,?,?,?) with primary key'
                    cursor_target.execute(sql, (client, socialmediachannel, socialUser, socialUserProfileLink, socialUserAccount, friendsCount,
followersCount, creationdatetime, displayName, displayName.upper(), image, time))
                    cursor_target.execute(sql, (client, socialdatauuid, socialpost, socialmediachannel, createdbyuser, creationdatetime, socialpostlink,
creationusername, socialpostsearchtermtext, socialposttext, time, creationdatetime_utc))
            except ValueError, e:
                with err_lock:
                    sys.stderr.write("Error processing JSON: %s (%s)\n"%(str(e), rec))
def getStream():
    proxy = urllib2.ProxyHandler({'http': 'http://proxy:8080', 'https': 'https://proxy:8080'})
    opener = urllib2.build_opener(proxy)
    req = urllib2.Request(URL, headers=HEADERS)
    response = urllib2.urlopen(req, timeout=(1+GNIPKEEPALIVE))
    decompressor = zlib.decompressobj(16+zlib.MAX_WBITS)
    remainder = ''
    while True:
        tmp = decompressor.decompress(response.read(CHUNKSIZE))
        if tmp == '':
        [records, remainder] = ''.join([remainder, tmp]).rsplit(NEWLINE,1)
if __name__ == "__main__":
    while True:
        except ssl.SSLError, e:
            with err_lock:
                sys.stderr.write("Connection failed: %s\n"%(str(e)))

2. Run the script from your editor

3. Checking the Results in the database tables SOCIALDATA and SOCIALUSERINFO.

Other blog posts on connecting Social Channels: 


Twitter connector to harvest tweets into Social Intelligence tables using Python script.


(If you find any mistakes or if you have any doubts in this blog please leave a comment)

I noticed very high interest in all topics that are related to marketing prospect functionality in SAP CRM.

On the other hand I have the impression that only a few companies actually started using marketing prospects in SAP CRM. With this they get the chance to measure the real value of marketing prospects for their business.


Perhaps using marketing prospects is considered as a big topic, or even a real mind shift? You go away from starting with prospect data that is already quite complete, in the sense of typically having a name and at least parts of the address, instead of starting typically with only an e-mail address.


I would like to encourage you to stop hesitating.

For sure you are infected by the market trends to bring your prospects as early as possible on a great "customer journey". Otherwise competitors could get them.

The pity is that most companies have a lot ot prospect data but don't use it yet. They don't address these prospects with marketing activities. As said such prospect data is far from being complete. That's why - from my perspective too often - it is just stored but never used.


What about starting with a first small set of such prospect data? Bring this data in the SAP CRM system, and define exactly the kind of lifecycle for them. - After a while measure your success with them, and then in case you are successful you can go ahead with the next set of prospect data, and increase the amount step-by-step.


Keeping my fingers crossed for your success!


Some additional information that could help to get this started:

For getting an overview of what is new in the marketing prospect area you can read SAP Note 1896854.

For understanding how to measure your success with marketing prospects you can read http://scn.sap.com/community/crm/marketing/blog/2014/03/07/how-to-measure-the-effectiveness-of-your-marketing-activities-with-prospects

The below configuration settings are required for a loyalty program to work in SAP CRM:


Create Loyalty Type

Transaction Path - IMG >> Customer Relationship Management >> Marketing >>
Loyalty Management >> Basic settings >> Define Loyalty Type


2 Assign Number Ranges for Loyalty Objects

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Basic Settings >> Assign Number Ranges


3. Loyalty Program Types Profile Definition

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Loyalty Programs >> Define Profiles for Loyalty Program Types


4. Partner Determination Procedure

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Basic Settings >> Assign Partner Determination Procedures


5. Date Calculation Procedure

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Basic Settings >> Define Date Calculation Procedures


6 Status Profile Assignment

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Basic Settings >> Define Status Profiles


7 Reward Rule Maintenance Templates

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management >> Basic Settings >> Define Templates for Reward Rule Maintenance


8 Condition Groups Mapping Definition

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Loyalty Programs >> Define Mapping for Condition Groups


9. Membership Settings


Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Memberships >> Define Membership Settings


10. Definition point Qualification Type

Transaction Path – IMG >> Customer Relationship Management >> Marketing >> Loyalty Management
>> Membership Cards >> Define Point Qualification Type

There are some known issues related to survey transformation. This blog post should give an idea about how the transformation is working and contains a collection of common issues together with documented solutions. Any known issues are highlighted with the alert icon (), whereas each solution is highlighted with the information icon ().


The main settings and the related XML and XSLT files can be found in the survey repository in SAP GUI. This can be accessed in from the survey suite (transaction CRM_SURVEY_SUITE) only, using the survey repository button:


survey repository1.jpg

The survey repository contains the files for the style sheets (cascading style sheets), the static survey XSLTs, the parameter files (XML), as well as layout XMLs for the survey print function:

survey repository2.jpg


  • Cascading Style Sheets (CSS): defines survey format - colors, background, style, etc
  • Static Survey XSLTs: required for rendering the survey from the survey XML to the HTML
  • Parameter XMLs: contain parameters for URL scenario
  • Layout XMLs: contains layouts for print scenario


For the survey rendering the CSS style sheets and the Static Survey XSLTs are required. The transformation works the following way. The survey is stored as an XML. While presenting the survey, XML goes through the XSLT transformation as a result of which the survey HTML gets generated - this is happens in 2 steps:

  • Survey preparation (Build Time): the preparation step is done once for every survey. The first stylesheet (GenerateValues), extracts the default values from the survey and stores them in the survey vales template file. The second stylesheet (GenerateTemplate) generates the survey template.
  • Survey Execution (Run Time): The Survey Template processes the Survey Values Templates and generates the Survey HTML with the default values.


The rendering happens with the so called CATServer. The CATServer is responsible for transforming the survey XMLs to the HTML using the XSLT files. The CATServer is to be maintained in the Survey Repository:cat server admin.jpg

There are two versions of the CATServer available in the system - the active one is highlighted in the CATServer administration:


  • Internal CATServer(ABAP Based): This version makes use of the ABAP XSLT processor. Therefore, this version of the CATServer has some limitations with respect to functionality, performance and resource consumption (especially memory). Hence the JTS is the recommended CATServer with the performance point of view, however since the ABAP based CATServer is much more stable always recommend this one.
  • JTS: Java Based Version of the CATServer JTS is Java based and provides the same functional scope than the external Java version. This requires no external installation. In addition, it is low in resource consumption and shows a much better performance than the internal ABAP-based CATServer.                


Switching between the ABAP and the JTS CATServer can be done at runtime, there is no system restart required.


There are some known issues related to the CATServer used. If the survey can't be activated or can't be displayed due to the following error message raised in SAPGUI, this is usually a CATServer issues:

cat server transforermfactory.jpg


As mentioned the ABAP based CATServer is much more stable, therefore the ABAP based CATServer is definitly the one to recommend.


To understand how to switch between the CATServer versions consider the following SAP note:


857535 - internal CATServer: Setup Instructions


Using class CL_SURVEY_CATSERVER you can either execute the method SET_CATSERVER_ABAP or SET_CATSERVER_JTS. The parameter user must be left empty.




There was a change done to the framework recently. The BSP scenario is now migrated to the THTMLB framework.


There may be some survey transformation issues. When accessing the survey at runtime you may get the following error:


bee xml2.jpg

bee xml error.jpg

This error should be solved by implementing the following SAP notes:


1817152 - Migration to THTMLB framework for BSP scenarios

1842704 - Error: <bsp:bee>: (BEE XML) BSP extension <:*> is unknown


There may be various layout issues. The style sheet is not considered or issues with allignment of radio buttons.


Those issues should be solved with the following SAP notes - the solution includes 2 steps:


For all corrections that affect the XSLT file it is required to generate the affected survey again. This can either be manually or by using report CRM_GENERATE_SURVEYS. The report is delivered with the following SAP note:


1835143 - Delivery of report CRM_GENERATE_SURVEYS

As a general recommendation for any survey issues happening in the WEB UI I would suggest to search for the latest notes correcting any XSLT file. There are some known issues, since most of them are related to the XSLT file the issue may be solved with the latest corrections.


I want to share some ideas and a brief overview about how the locking mechanism for the BPS Planning is supposed to work, how this may be analyzed and how to tackle possible locking conflicts. Solutions for known issues are labeled with . This blog is related to locking conflicts only so the BPS planning integration to CRM should already be set up properly.


The BPS Planning is integrated in CRM Marketing as key figure planning application. This provides a planning functionality that allows to measure the performance of an organization, a department, a project, or other by setting plan values for important business key figures. The planning is happens in the BW system using BPS Planning.


Whenever the CRM Marketing object such as marketing plans, campaigns or trade promotions are edited, the planning cube in BW sets a lock based on the defined characteristics selection. The idea of this blog post is to provide an overview about the locking mechanism.


The locking of the planning cube is required when either accessing the planning layout or at the time of saving the marketing plan object. Once the CRM Marketing Object is edited the lock on the planning is set immediately. If the planning in BW cannot be locked, the CRM Marketing object cannot be locked either. This design is to protect data integrity.


When accessing the planning layout this is opened in edit mode per default:


doc bps lock edit mode.jpg


Once the planning layout is opened, the planning cube is locked based on the selected characteristics. If any other user tries to access the same object, the planning layout is opened in display mode – hence no synchronization happens. The user is informed with the following error message:


Planning object is locked in BI; changes cannot be saved [CRM_KFP 009]


doc bps lock msg1.jpg

doc bps lock msg2.jpg


The lock entries are created for the characteristic selection only. When a user edits the planning layout for marketing object A, another user should be able to edit the planning for marketing object B.


There is a known error in BPS with BW Release 7.40. If a user wants to plan data for an info cube, no other user can access the same cube. This is caused by the fact that the system locks the entire cube, ignoring the characteristics selection. This locking conflict affects SAP_BW 7.40 from Support Package 2 to Support Package 5 inclusively. The locking conflict is solved either with SP 6 or with implementing the following SAP note:

1926227 - Lock conflicts in BW-BPS from Release 7.40 [https://service.sap.com/sap/support/notes/1926227]



How is the locking mechanism working?



When accessing the CRM Marketing Object CRM calls the BW interface with the OBJECT_ENQUEUE command. Using the RFC call to the BW function module UPX_KPI_API_XML the BW system is accessed:

doc bps lock1.jpg

CRM --> BW

doc bps lock debugging bw.jpg

If the object_enqueue call is successfully locking the characteristics selection in the planning cube the following lock entries are generated. Those can be tracked using the following path in the Easy Access menu in the BW system (transaction SM12):


SAP Menu => Tools => Administration => Monitor => Lock Entries => Header 1


doc bps lock lockentries.jpg


The lock entries are created for the RFC user that is defined in SM59 for the BW remote connection, so this is not necessarily the CRM online user.


There is no locking entry in SM12 but still planning is not possible due to an error – where is this coming from?



There is another known situation, where the user gets the following errors:


RFC error: Planning cannot be performed for real-time InfoCube 0CP_SLSCA; see long text. [CRM_KPI_PLA 205]
Planning Services Error [CRM_KPI_PLA 203]


doc bps lock msg 3.jpg

This is not a locking conflict but is caused by the definition of the InfoCube. The cube is obviously defined as exclusive read cube – once the cube is in read mode planning cannot be performed. The cube needs to be in write mode so planning can be performed. This is further documented in the following SAP note:

555849 - Definition of exclusive read cube [https://service.sap.com/sap/support/notes/555849]


This is one of the weakness in the design of the CRM planning integration, caused by the very way BPS handles locking. The locks set by BPS will prevent to lock the same data in another BPS, or BW-IP session. Locks set by BPS will not prevent locking or updating of the data by core BW processes. For example, nothing prevents a BW administrator to change the mode of the InfoCube from Planning to Loading, while a user locks the same data by editing the Marketing Object.



There is a locking happening due to a data slice – what does this mean?



When accessing the planning layout the user gets the following errors:


RFC error: Data record is locked by data slice. [CRM_KPI_PLA 205]
Planning Services Error [CRM_KPI_PLA 203]

doc pbs lock data slice.jpg


The data slice is defined to the planning area, and is used to protect data of the InfoCube against changes. If the data slice overlap with the characteristic selection of the CRM Marketing Object planning cannot be performed. In general Data Slices are not supported from CRM planning integration perspective.


doc bps lock  data slice2.jpg


Further information about data slices can be found in the online.help:



If the data slice overlaps with the planning selection there is no other way than deactivating the same to enable BPS planning for the CRM Marketing object.

What if you could target each and every customer individually?


„Marketing today is working totally different compared to the last years“. Those were the opening words from Karina Herrmann on the SAP stage at CeBIT 2014. The SAP product manager outlined the challenge very nicely: “Many companies today have millions of customers. Capturing all interactions can lead to billions of records”. (source)


SAP’s answer for the future marketing is Customer Engagement Intelligence, a real-time solution that enables companies to instantly unlock sentiment and contact insights from both social media channels and company-internal sources to better target and influence prospects and customers in a variety of ways. With SAP Customer Engagement Intelligence you can analyse those billions of records in seconds and gain powerful insights into the customer’s behaviour.


Let’s have a look at a concrete example. A marketing manager wants to feel the pulse of the market regarding a new idea discussed over lunch with a colleague: a hybrid SUV. By using SAP Social Contact Intelligence, our marketing manager can drill down on the interests of millions of contacts, being customers, prospects or even anonymous contacts. Filters can be applied to limit the scope of the analysis to the US market, for example, and to the automotive industry only.


Segment-of-one Marketing Image1.png


The interests of all selected contacts are nicely displayed as a tag cloud (s. above figure). By selecting “SUV” and “Hybrid Car” we can focus our analysis only on those interests. The end result is a tailored target group that could be used in further marketing activities, like an email campaign. But we can go even further. By switching to the Sentiment Engagement tool, we can have a deep-drill on the qualitative dimension. We can understand what those people were doing on the social networks and on the company-owned forums. The solution allows our marketing manager to answer questions like: what is the overall sentiment on hybrid SUVs? what are people saying about hybrid cars? And the loop can be closed right here: you can pick individual posts and instantly react to them.


Segment-of-one Marketing Image2.png


But it is not only about the possibility of instantly reacting to a post.  SAP Customer Engagement Intelligence offers all the required information in order to know how to react. You can have a deep insight into each and every contact before engaging with them. Information such as contact level, company, role, activities and interests can be accessed directly within the tool.


Segment-of-one Marketing Image3.png


And it can become even more interesting: build-in predictive analytics algorithms can compute the buying propensity and assert how a target group will react to a particular product. The analysis can be run by marketing employees directly, without any support from IT.


If you want to learn more about SAP Customer Engagement Intelligence you can check out the solution overview page here or you can start a free cloud trial here.


Marketers are already familiar with dividing customers into target groups. Now it’s time for unique and individualized targeting.

Welcome to the future of marketing!

The Big Data Visualization Conundrum

This glorious age of big data is creating incredible opportunities for businesses to glean deeper and faster insights for more accurate and timely decision-making, thereby leading to improved customer experience and greater innovation.


Concomitant with this are several challenges. Organisations are overwhelmed by the volume, variety, and velocity (do check out Doug Laney’s original research note on the 3Vs of big data) of the data pouring into and across their operations. Businesses are barely able to store big data, leave alone understand it, or present it meaningfully. Traditional reporting-based BI tools are insufficient to unlock the value that big data represents, partly because they were never designed to analyse semi-structured or unstructured data in the first place.


Data visualization enables organisations to assimilate raw data and present it in a way that generates the most value. I'm proposing 3Cs that good data visualization should empower viewers with - coherence, context, and cognition. (Consequently, I hope that someday I'll be as famous as Doug Laney! I also thought about correlation and causation, but there seems to be a raging debate regarding the relevance of those two). Pairing big data with data visualization discovery tools empowers business users to be self-reliant and not depend on enterprise IT to mine data, perform ad-hoc analysis, or create one-off reports, for them. Going ahead, this democratisation of BI will serve real-time insights to business users directly, leveraging the growing abundance of mobile devices, and bypassing the conventional batch-processed-reporting route.


For those interested in knowing more about making big data more meaningful, I would recommend these articles on Wired and Forbes magazines.

Introducing Pixelplots

Pixelplots, a data visualization technique, are high-density multivariate landscapes of big data that empower the discovery of insights, without any aggregation of data. Simple analytics (bar and pie charts) are easy-to-use (as long as one isn't using a 3-dimensional pie chart, for example, or using a format that is incongruous to the objective of the presentation) but present highly aggregated data, with a limited number of data values. Pixelplots do have a learning curve (just like Treemaps), but are invaluable when it comes to visualizing the big picture without forfeiting granularity - almost like a multi-focal lens. Their fundamental premise is to represent as many data objects as possible on an electronic display at the same time, by mapping each data object to a pixel. The number of pixels mapped is therefore the number of data objects being considered. Key attributes of a data object can be mapped to its corresponding pixel’s colour, or horizontal and vertical axis ordering.


There has been some academic interest in pixel-oriented visualization techniques in the past, but I am yet to hear about an actual implementation of a Pixelplot in any commercially available data visualization / BI discovery tool. The reason for my fervent interest in this is twofold. Firstly, being a data visualization buff, I am fascinated by how much the Pixelplot actually accomplishes by visualizing a huge set - while simultaneously representing multiple attributes - of data objects. Secondly, I believe that Pixelplots perfectly complement SAP HANA, and they address big data’s “volume” problem more effectively than any other visualization technique in existence today. Keep in mind that almost all analytics on conventional dashboards aggregate, sample, or sort and selectively pick out the data they represent, and never represent the entire data set on a single screen.


Moreover, Pixelplots leverage the ever-increasing pixel densities of modern electronic displays. Apple’s “Retina displays”, for example, already pack up to 5 million pixels into a 15” laptop screen. On regular desktop displays, a Pixelplot measuring just 960 x 600 pixels can represent 576,000 unique data objects. Mobile device pixel densities are typically even higher than desktops, and by this virtue, the Pixelplot is mobile-ready. I am hoping that you are sensing my excitement!


Visualizing Consumer Engagement

To understand Pixeplots better, let's meet our primary user persona, Cari Smith. Cari is an Online Marketer with a consumer electronics company called Cool Electronics (fictitious). Do note that this use-case for the Pixelplot focusses on Marketing within CRM - based on the choice of KPIs, they can be used in any industry or line of business.


The Consumer Life Cycle

Nate Elliot from Forrester authored this magnificent blog post on the “Marketing RaDaR” , where he presents a powerful alternative to Elias St. Elmo Lewis’ AIDA (Awareness - Interest - Desire - Action) funnel model, which has been used for years as a tool to structure an organisation's sales. He proposes a model based on a four-stage consumer life cycle (rather than a funnel) – consumers first discover a product or service, then explore it greater detail; next they buy the product or service, and after purchase they engage the company from which they bought, as well as with other consumers. Based on my own interactions with Marketing Analysts (through user interviews while working on a next generation consumer engagement innovations powered by SAP HANA), this resonates perfectly with their mental model and their abstracted perception of their consumer base.


The Top Marketing KPIs

What are the KPIs that are of interest to Cari Smith (our primary persona, just clarifying as I've been bandying around several names in this post)? While there are several interesting articles talking about the most important marketing KPIs, Avinash Kaushik’s article lists out a ladder of marketing metrics, with Customer Lifetime Value at the very top. By definition, CLV is the amount of revenue or profit a consumer generates over his or her entire lifetime. To be truly insightful, CLV should not be merely historical (summing up revenue earned from a consumer till date), but be predictive (project how much revenue can be realised from a consumer over their lifetime). As consumers become more digitally networked and businesses move towards a single system of record for all consumer data, another (orthogonal) metric that could add tremendous value is an aggregated social activity score, something like the Klout Score. Another important dimension could be the time spent – how much time have consumers been in a certain lifecycle stage?


Based on all that has been discussed above, here is (finally!) a mock-up of a Pixelplot:



  • At the highest level, Cari sees how many consumers are in each of the four life cycle stages. Note that these life cycle stages are customizable - this could be substituted with stages of a customer loyalty program, for example, or need not be a progression at all (simple categories).

  • Every pixel represents a unique consumer, and every consumer at any given point is in some stage of the life cycle.


  • The colour of every pixel lies along a monotonic (blue) range of shades, and represents Customer Lifetime Value. The darker the shade, the higher the CLV.


  • The pixel x-ordering maps to the time spent by consumers in that particular life cycle stage. The farther they are to the right, the longer time they have spent.


  • The pixel y-ordering maps to the social score of the consumers - the higher they are, the more social activity they've recorded.


  • A compact Frequency Distribution of the Customer Lifetime Value (click on a bar in the stacked chart to filter) is available at the top, and the Conversion Rate from one life cycle stage to the other (over a default interval which can be altered) is displayed below the Pixelplot.


  • This is a mock-up created using Adobe Illustrator - the Pixelplot may not look this "artistic" actually!

At a glance, Cari can view her entire consumer base and see how they are divided into life cycle stages - this is the big picture. She can instantly identify consumer clusters, for example, those who have a high lifetime value and are more engaged socially - now this is all about pattern recognition, a task we homosapiens naturally excel at (although the machines are catching up!). This kind of insight is very relevant for businesses to satisfy the digitally connected and socially networked consumers of today.


Here is an example of an insight that Cari might astutely glean from the Pixelplot:

"Aha! Here is a large group of consumers who have a high Customer Lifetime Value, are significantly engaged socially, and have been in the Explore phase for a while. I should create a Facebook or Twitter promotion to get them to buy!"


Focus & Context

We talked about the big picture, but the USP of the Pixeplot is that it visualizes data at the atomic level (sans aggregation), in this case, value-by-value, at the individual consumer level. Cari can click on an individual pixel (since this would test anybody's psycho-motor coordination, as pixels are fast becoming invisible in modern displays, I am proposing a focus+context interaction that converts the cursor into a zoomed-in matrix of 9 x 9 pixels) to go into the details of specific consumers. Our old friend Tom Whitman (from the Teched Demo we ran in 2013) makes a reappearance in the screen below. This ability to instantly drill-down to the atomic level helps Cari plan and run 1:1 marketing campaigns, or simply to understand who some of her typical consumers are:


Filtering the data in the Pixelplot enables Cari to "thin" information effectively. She can choose what KPIs she wants to visualize, and also restrict the data set based on other attributes (demographics, channels, or loyalty). Altering the filters instantly reveals how many consumers match the filtered criteria:



Visual Segmentation

The idea of the Pixelplot isn't unique, but using it to demarcate market segments through direct manipulation, potentially is! Using algorithms like support vector machines we could automatically discover consumer segments and visualize them graphically, layered atop the Pixelplot. Alternatively, Cari could draw her own segments based on the insights she derives from the Pixelplot, either by using a pointing device or a stylus. Do note that this approach is very different from the traditional (rule-based) methods used to define segments - hence the term "visual segmentation". Cari sees her entire consumer base in one screen and is also able to identify patterns that either automatically emerge (based on the orthogonal metrics that are simultaneously visualized, like in the example above), or are arrived at by slicing and dicing (using the filters we talked about). For every segment (either suggested or defined), we could surface additional details through microcharts, all for better decision making. Cari could edit segments, or add a title/description for those that she wishes to retain. These are still early days - I am confident that the Pixelplot lends itself to several other exciting possibilities!



This was an introduction to Pixelplots and how they could be applied to visualize big consumer data, conduct analysis through slicing and dicing, and to define market segments visually (and directly!). Like I pointed out earlier, this was just one specific illustration and there is a lot more that can be done with them:


  • By incorporating panning and zooming, the Pixelplot can leverage the much-loved design principle of progressive disclosure -  zooming in reveals additional levels of detail about consumers progressively, akin to how online map applications (Google Maps, for example) work.


  • Using linking and brushing, selecting a certain consumer can reveal others who have similar behaviours and attributes - enabling a "look-alike" discovery of target consumers


A parting note - Pixelplots are not easy to implement, as pixels need to be ordered (they are not positioned absolutely as there might be instances of overlapping) in horizontal and vertical axes simultaneously, which needs a robust rendering algorithm to work efficiently behind the scenes. Also there might be performance issues at the UI layer to render such a vast data set. There is a workaround to this that I can think of - populate the pixels that are likely to correspond to higher / important values first. In the example above, this would mean that the darkest blue pixels (the consumers with the highest Customer Lifetime Value) appear first, followed subsequently by the lighter shades.


Thanks for reading, and do share your feedback and comments. I'd love to hear from you!

Dealing with prospects is a real challenge:

  • You often have only fragmented data of them
  • You often don’t know their interests, or you only have a little bit of information that can easily mislead you
  • Nowadays people quickly “run away” when feeling pushed - or even when feeling watched

On the other hand you have a lot of prospect data records available:

  • Prospects show interest on your website
  • They are active on Facebook, Twitter & Co

So certainly you want to reach them with the products of your company and win them as customers.

But how to measure the success of your marketing activities? How to really be able to follow-up on your actions you have taken so that you can improve them further?

First of all I would recommend to get the data of your prospects into SAP CRM

See also some blogs related to this topic:




Now I would like to explain how you can measure your success in a very simple way:

Phase 1: Set up

  1. Make sure that the responses of your prospects are tracked: Use the OData Service CRM_MKT_PROSPECT_ODATA to create
    interactions for your prospects per response.
  2. Make sure that for each of these responses the correct marketing campaign gets referenced in the response: Use the BAdI CRM_MKT_INTERACTION_OBJECT to implement the correct logic for referencing always the right campaign.
  3. Make sure that for each kind of response a score value is defined: Define this score value in Customizing -> Customer Relationship Management -> Master Data -> Business Partner -> Marketing Prospects -> Define Score for Interaction Objects. – Each response will then increase the score of the prospect.
  4. Decide at which score level you want to convert a prospect to an account (BP). - This is the time when you consider a prospect being converted to a customer.
  5. Decide also when you delete (= “give up”) a marketing prospect, for example after which time, after how many or which marketing campaigns at which score level.

Phase 2: Execution

  1. When evaluating the right target groups for your campaigns consider also the interactions (including the ones that reflect the responses). With doing this you can be more specific, and you can address really the right people in the right way. Because you not only consider their  more or less static master data but also their behavior and their reactions to your marketing activities.

Phase 3: Measurement

  1. Analyze based on your campaigns (now with detailed data about the number and kind of responses) and based on the “prospect-to-customer” conversion rate the overall effectiveness.

More details about what is offered in the area of nurturing and targeting of marketing prospects can be found in SAP Note 1896854.

Many companies not only analyze what people talk about their products, but started also engaging in a more interactive way with them. A conversational style has evolved. Even one-to-one interactions with the persons active on the corporate Facebook, Twitter & Co pages become normal, especially in the B2B business.


It is good to ideally have a friendly atmosphere and let your followers and social contacts communicate with you in the more modern pull-mode.


However you should still decide at the right point of time to get the data of your social contacts into SAP CRM. Otherwise you cannot easily track your marketing activities with them, neither you can measure your success.


How should you then be able to decide if your activities with your social contacts have a measurable effect at all?


Of course it doesn’t make sense to upload all social contacts into SAP CRM:

  • Not all of them are relevant for you
  • You are not sure if they are open for being addressed by additional marketing activities


Why not using the principles you already apply on the company’s web site also in the social world?


Let your social contacts decide at what point of time their data gets stored in the SAP CRM system; for example as soon as they show interest in a newsletter, demo or event. When taking this active step they can verify for themselves if they actually want to provide their data to you, and if they accept that a more focused marketing will probably start.


With this you can keep the friendly and interactive communication style you started with.


In any case you can benefit from the following when storing social contact data in SAP CRM: You can connect the activities of your social contacts in the social world with the (internal) data in SAP CRM, and by doing this you can get a more holistic view on them.


Go ahead as follows:

  • Get the data into SAP CRM, including their social user data, for example their Facebook accounts.
Tipp: You can create marketing prospects based on the social contact data, using the OData service CRM_MKT_PROSPECT_ODATA.
  • Get the social user IDs from the respective social network(s). These are technical IDs that identify uniquely your social contacts per network, and can be used to connect data from the social world with the internal CRM world.
Tipp: Use the report CRM_MKT_PROSPECT_SMI_UPDATE to update the social user data in SAP CRM with the IDs from the social network(s).

Recently in a customer project we got the requirement that customer needs the assignment block "Membership Activities" could only be visible for some user under a given period. The authorization for that user must be explicitly assigned by administrator with period clearly defined. For example, the user XXX could only be allowed to see that assignment block between 10:00AM ~ 10:30AM this Friday.


Since the SAP standard authorization concept could only support time based condition, we have to do some custom development:


1. We create a new UI component and put it to a new work center "Authorization Center".


We put this new work center to business role LOY_ADMIN so that only Loyalty administrator could be able to assign / delete time based authorization.



Admin could choose the user via search help and click Assign button to grant authorization.


we use a custom table to store the authorization detail. This is ok since in customer company, normal user could only use webclient UI to access CRM with no SAPGUI installed. The authorization could be deleted by admin at any time if needed.


2. for UI component LOY102H_MSH, we enhance the view controller below, add a post exit on method DETACH_STATIC_OVW_VIEWS to filter the view

CUMSHMA.LOY102H_MSH/MSHMemberActivities by checking the authorization. The technical implementation could be found here.


The current user and current time is compared with authorization detail stored in custom table. If no authorization,

the assignment block will be hidden with a warning message displayed.


This question is very typical, and the decision that needs to be taken is fundamental.


Following you find some hints that hopefully help to take this decision:


A) In case you deal with prospect data that contains only a few attributes, for example the e-mail address only, it is often not allowed to create such prospects as business partners in the CRM system. The CRM system itself can store such data, and it is also possible to configure the data exchange via CRM Middleware in a way that such prospects are not replicated to a connected SAP ERP. But usually organizational guidelines exist that need to be followed for creating a business partner in the CRM system. With an e-mail address only it is often not allowed to create a business partner.

In this case I would recommend to create such prospect data as marketing prospects in SAP CRM.


B) Besides it can make sense to explicitely separate the data of prospects where you don't know at all yet if you can start a real business relation with them; if they are interested at all. They perhaps only registered on the company's web site or talked about the company's products on Facebook or Twitter. But this doesn't mean that they have a quite serious interest in the products, or that they would even buy some of them at some point of time. In many countries the data privacy regulations are quite strict: It is not allowed to store data from persons, with which no business relation exist.

In this case I would recommend to create such prospect data as marketing prospects in SAP CRM, and convert them to business partners as soon as a real business relation starts.


C) In case the company deals with prospects in a very close and extensive way from beginning on, somehow not dealing with prospects that are not considered as being quite serious about their company, creating them as business partners in the CRM system can make more sense. Only business partners can be used in activities, leads, and opportunities, to mention some examples for business transactions that are based on the one-order framework of SAP CRM. So if such business transactions need to be created in SAP CRM for the prospects from beginning on you have to create such prospects as business partners in SAP CRM.


Some final remarks:


If the decision is taken to use marketing prospects for storing prospect data in SAP CRM, be aware of their short life-cycle: Either convert marketing prospects as soon as possible to business partners, or delete them if you can't establish a real business relation with them.

The purpose of marketing prospects is to support you in a very defined and controlled phase of time in your marketing activities.


I would recommend to define exactly what needs to be reached by marketing, how this can be reached, and in which time frame this needs to be reached.

This helps you to always have an overview of the prospects, and the company's success with these prospects.

I regularly present the new SAP HANA based application “SAP Customer Engagement Intelligence” (CEI) to customers
and partners. One application of the CEI suite is called “Audience Discovery and Targeting” (ADT) and used for prospect and customer segmentation.


The question that arises in the audience is typically:


“In which cases do you recommend utilizing the new SAP HANA based application CEI-ADT rather than the High Volume Segmentation capabilities of a SAP CRM on HANA when it comes to large numbers of segmentation objects (i.e. several millions of customer records)?”


Below table should provide you with some insights into the key differentiators of the two solutions to help you deciding the best fitting solution approach for your requirements.



One of the most compelling arguments for HANA based applications is certainly performance related. To get a feel for numbers and the throughput achievable with the new solution, product development group did an E2E performance test with a sample of 15 million customer records.


The process starts with loading the data with SAP SLT from the source system to the HANA DB. Based on that sample a customer segmentation process is
initiated using Country/ City as filter criteria. Next steps are target group creation from the segmentation model and target group member replication to a
connected SAP CRM system.




Impressive, isn't it?

Companies increasingly want to reach ALL their prospects with marketing.

Often they get new prospect data through web site registrations, or increasingly as followers on Twitter & Co, as soon as they enter the social media space more seriously. Such prospect data is usually of quite low data quality. Only a little bit of data – often only the e-mail address - is available.

Many companies hesitate to create business partners in the CRM system based on such low quality data. But still they would like to benefit from that data and reach such prospects with their marketing campaigns.


Starting with CRM 7.0 for EHP3 SP02 functionality is provided that allows to easily get such marketing prospects into SAP CRM, and nurture and target them with marketing campaigns.


With SP03 (available now) additional features are provided.



  • Calculating a score based on the type of response from the prospect   
  • Creating and changing of prospect data from any data source with using an OData service; a duplicate check can be incorporated


Check out SAP Note 1896854 for details.

There are 2 different tracking options(++ or ##), tracking via target site and tracking via intermediate site, if you use ## as the tracking option, the actual URL which is found after the first attribute '?' along with attributes and tracking variables will be encoded with escape characters. That is, the sepcial characters in

URL will be replaced with escape characters(like %2F, %26...)). This is the standard way of URL generation.


An URL contains multiple parts.
        a.) Actual URL
        b.) Attributes
        c.) Tracking variables.


You can find the below link (after encoding) as an example for tracking URL generation:

<a href="http://website.xyz.com/site/TEST/tracking?openagent&target=https:%2F%2Fxyz.com%2Fxyz%2Fk2%2Fj.php%3FAT=down%26ED=121273952%26EF=MA%26UUID=1128228232%26SHA2=UUfdo-bx5x0lCEwI3hVlvBJWkkUMGci8K86N76ebMVA%3D%26RT=MiM3
&amp;MIG=6A13EE4A0305E701E1000000C0A80A60&amp;URLGUID=07D1B64AEA6DC167E100000011A80A60" target="_blank">Session


The first part is not encoded. The first variable is added with '?'. The remaining parts of the URL(Actual URL, Variables and Tracking Guids)
are encoded. The first part is not encoded because, when the users click on URL, they will be taken to this site. From this site, the redirection
will happen, which is why all the special characters are replaced with escape characters.


(Just to confirm this, type gmail.com in your browser, you will get a new URL with two parts:

a.) Google default site, where there is no encoding done.

b.) The second part that starts after account. The special characters in the second part will be replaced with encoded values.

This is just an example of how the URL has to be generated. This is the universal standard of URL generation.)

Configuring SAP CRM TFM (Trade Fund Management)

Trade Funds Management enables planning, tracking, accruing and controlling of Trade funds. All trade funds are processed through fund Management via linking between the TF’s (Trade funds) and TP’s(Trade promotions/Trade activities). Funds are used to control trade spends at different level of customer, product and spend type. This helps manufactures to manage distribution and consumption of Funds.

Funds Management is tightly integrated with other modules like

1.       TFM- Set up Funds plans and Funds

2.       TPM- Create TP’s with valid fund plans

3.       ERP Integration- Processing of orders against TP’s

4.       FI posting – Accrual run and accrual integration back to TP through Check book

5.       Claims Integration – All types of claim process

Funds Plan:- It’s a framework for the funds setup and defines Validity period, currency and other attributes that all funds have in common.  You can create Funds Plan as shown below through web UI.



SPRO->CRM-> Funds Management->Funds Plans and Funds ->Define Funds plan types



Assign Fund types as below:-



Define Fund Transaction type as below.



Fund:- A fund is an object that allocates budgets for a specific customer-product-sales org(Expense type combination)

A fund is identified by: Funds Plan, Fund Type and Attributes (Account, product, individual hierarchies and territory).



Budget: It represents an initial fund amount which can use or consumer by TP’s.

Budget Distribution is done via Budget posting, budget update or budget transfer.

You can update the budget as using the transaction type: Budget update



Budget Posting is done with status – Approved. Otherwise it won’t appear in the Funds checkbook.





Similarly you can do Budget Transfer to exchange budget between funds.

Generally these funds are created on Business partner/ Business partner hierarchy / Product / Product Hierarchy which can be consumed in the relevant TP’s. This will be determined by fund attributes which are assigned to Fund type in the configurations.

For example: Fund type- Off Invoice, Fund attributes are Account and product category.



Finally you can link Fund plan to Trade promotion.



Filter Blog

By author:
By date:
By tag: