Introducing an Add-on for Cleansing Character Strings

[sander_vanwilligen]

As you might know SAP BW does not allow every possible character in character strings. An upload or activation can terminate in such cases. It occurs frequently with character strings where users (or interfaces) can freely enter text.

SAP delivered various Function Modules for validation of character strings. Unfortunately cleansing of character strings is not provided by SAP. It would be very helpful to have a comprehensive ABAP Add-on for cleansing of character strings.

In this blog I would like to introduce such an ABAP Add-on which I developed. Please refer to my document Implementing an Add-on for Cleansing Character Strings for detailed implementation instructions.

Issues with Character Strings

Let’s have a look to a typical example: a source field contains “forbidden hexadecimal characters” (between HEX00 and HEX1F) which are usually not permitted in BW.

Figure 1: Example error message in log

Figure 2: Detailed error message

The error message shows that the character at position 11 (hexadecimal HEX0D) is not permitted. The error message does not show that also the character at position 12 (hexadecimal HEX0A) is not permitted.

Please note by the way that ‘HEX23’ is a documentation error in message BRAIN 315.

Other typical examples are a source field that contains lowercase but the Characteristic only accepts uppercase, or a source field that contains “special characters” which are not allowed.

Allowed and Unallowed Characters

In SAP BW the default allowed characters are:

• !"%&''()*+,-./:;<=>?_0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ
• SPACE symbol

You can maintain permitted extra characters with t/code RSKC. You can also find it in the SAP Customizing Implementation Guide (IMG): SAP NetWeaver > Business Warehouse > General Settings > Maintain Permitted Extra Characters.

Figure 3: Maintenance of permitted extra characters

You can choose between the following three options:

• ALL_CAPITAL (and nothing else): characters which are uppercase letters in the code page are permitted for data loading;
• ALL_CAPITAL_PLUS_HEX (and nothing else): equal to ALL_CAPITAL pus hexadecimal HEX00 to HEX1F;
• Specify “special characters” individually.

Option ALL_CAPITAL is mostly used and can be considered as the “best practice”. SAP does not recommend using ALL_CAPITAL_PLUS_HEX.

Validation of uppercase and lowercase are handled differently. The validation rules which were presented above are applicable for uppercase Characteristic values. The lowercase validation is less strict; only hexadecimal HEX00 to HEX1F are never allowed.

There are two additional exceptions:

• A Characteristic values that only consists of the single character ‘#’ is not allowed;
• A Characteristic value starting with ‘!’ is not allowed.

Cleansing Add-on

The Cleansing Add-on is an ABAP Objects Class and is intended to be used in a Transformation Rule. The logic is based on two standard SAP Function Modules in Function Group RSKC:

• RSKC_CHAVL_CHECK - Check if characteristic value is permitted (uppercase)
• RSKC_LOWCHAVL_CHECK - Check if characteristic value (with lowercase letters) is permitted

Those Function Modules only contain “validation” functionality. The “cleansing” functionality is added in an RSKC compliant way. The Cleansing Add-on has an easy-to-use interface and can be implemented in any Transformation Rule for a character string using a Routine.

Related Documentation

If you would like to read more about this topic, I can recommend reading the following documentation:

• SAP Note 173241 - Allowed characters in the BW System;
• SAP Note 1075403 - Unallowed characters and ALL_CAPITAL;
• SAP Note 1373402 - Issues with invalid characters or SID Generation;
• Special Characters in BI.

Conclusion

In this blog we discussed the possible issues with character strings. We had a look to the allowed and unallowed characters in the context of SAP BW. Furthermore, I introduced briefly the Cleansing Add-on. Please refer to my document Implementing an Add-on for Cleansing Character Strings for more information re. implementing this Cleansing Add-on and how to use it in Transformation Rules.