Value conversion in SAP interface communication – architectural thinking
This article discusses the SAP technologies and key points that need to be addressed when an SAP solution is needed for value conversion in business process integration scenarios across systems. The article is based on SAP recommendations which have been enriched with additional architectural key points. The first part of the article explains value conversion in general. The next chapter discusses SAP technologies for value conversion. The third chapter analyzes architectural key points of value conversion that need to be taken into consideration. In the last chapter, the SAP technologies are explained in different architecture scenarios which are based on SAP’s recommendations and the key points discussed in chapter three.
Value conversion basics
Many different types of IT systems can still be found in many organizations. Each system contains its own data and semantics, especially in enterprises that are spread across multiple companies and geographies. Problems are likely to occur when data is exchanged between these systems via interfaces.
A simple example of a value conversion issue could be the representation of country codes in different systems. One system could use the letter “D” for Germany and another uses the common notation “GER”. When data including country codes is exchanged between these systems, the operations will fail because the systems do not work with different representations except their own. A mechanism is needed to convert the sender value to the expected receiver value. This kind of value mapping is classified as a functional value conversion, because the content is critical for correct transactional processing inside each system.
Other value conversions, such as the conversion of system names, are needed only for technical reasons. This type of mapping can be technically necessary to enable communication between the sender and receiver system. The functional departments are not concerned about these types of technical details. They will also not feel responsible if errors occur in this type of mapping. The separation between technical and functional value mapping is important when we discuss different value mapping approaches later on.
The definition of one single representation of a value (e.g. unique customer numbers) in the whole company could be considered a solution from an architectural point of view which would make value conversion unnecessary. In practice, this is often not applicable, however, because each business unit is interested in keeping its own representation in its system. Making changes would also incur additional costs and the users would need to get used to the new representation. In a worst case scenario, it would be impossible to make any changes for certain terms as they are hard-coded by a third-party vendor. In such cases, another solution will be needed for value conversions.
This section briefly discussed what “value conversion” is about and why a concept is needed to handle it. The next chapter explains the technologies that SAP recommends.
SAP Technologies
In this section, we discuss the SAP Process Integration (PI) and SAP Application Interface Framework (AIF). Both are intended to connect interfaces with each other and both explicitly provide value conversion functionality. Other SAP middleware products, such as SAP gateway, are not discussed here because they do not focus mainly on interface integration and value mapping.
PI is the middleware which is able to deal with different technologies (like file, SOAP, RFC, etc.) and the mapping of different interfaces on message type level. The middleware seems to be designated for message transformation and value conversion. The source system sends data to the middleware, maps the values (e.g. from “D” to “GER”), and delivers the message to the receiver.
SAP PI provides different standard concepts that can be used to map values between interfaces. These concepts, which are “Fix values”, “Value Mapping”, “Value Mapping Mass Replication” and “RFC look-Up”, will be discussed briefly.
Figure 1: SAP PI as a middleware
The most basic option in PI is “Fix Values”. The value pair is entered and used for the specific target field mapping only within the enterprise service repository (design time). The value pair cannot be re-used in other message mappings or target field mappings. Maintenance and look-up of the value pairs are done in PI because no mass upload of values is possible. You can only enter source and target values and there is a 1:1 relationship between them. This option is recommended if you only want to map very few values by hand and the mapping is not to be re-used in other message mappings. The following figure shows a “Fix Value” example in PI.
Figure 2: Fix value in SAP PI
“Value Mapping” is another more complex option in PI. The value pair is entered in the integration directory (configuration time) and not directly in the target field mapping, like fix values. It can be re-used in all message mappings. Maintenance and look-up of the value pairs are done in PI. The creation of value mapping pairs is done manually and you need to specify additional meta-data, such as scheme, agency, and a context, for each individual value, which requires a sophisticated naming convention. Inside the Enterprise Service Repository (ESR) in the target field mapping, you must enter the source scheme and source agency of the source value and the target scheme and target agency for the target value. You must also define the context.
All values that belong together are classified as a “Value Mapping group.” In this group, there is no predefined source and target value. You can convert values in both directions based on the semantics that you use for source and target scheme and agency in the target field mapping. Compared to fix values, you can have more than two values in one group.
The following table is an example of a representation of the same person, Mr. Smith, in different systems:
Context | Agency | Scheme | Value |
Smith | System A | Employee | Mr Smith |
Smith | System B | Customer | 0175965 |
Smith | System C | Member | 3885 |
Table 1: Entity representation with value mapping
Based on the determination for context, agency, scheme and source value, the looked-up target value may differ.
“Value Mapping Mass Replication” is an extension of “Value Mapping”. Mass upload of value pairs is done via java inbound interfaces on PI. This allows any back-end system to store and maintain value pairs which can then be replicated to PI.
It is impossible to replicate values from PI Cache to the back-end. Only back-end to PI replication is possible.
Caution: Back-end and PI Cache should always be in sync. If the conversion rule base is updated in the back-end, the information needs to be replicated to PI. In the meantime, the value mapping cache in PI will not be up-to-date. If interfaces are running during this timeframe, inconsistency and false mappings may occur.
When using “RFC Look-up”, the value mapping pairs are saved in a back-end system.
During run-time, the value mapping pairs are looked up dynamically in the back-end.
The look-up API supports RFC, SOAP und JDBC.
The inbound and outbound interfaces of the look-up need to be configured in the ESR.
The communication channels also need to be set up in the Integration Directory.
PI is one solution in which data conversion can be converted. SAP also introduced AIF as another technology which is an add-on that is to be installed on each SAP back-end system. It enables interface development, monitoring and error handling. The functionality of the framework is performed before the actual inbound or outbound interface is executed in the SAP back-end system during run-time.
Figure 3: SAP AIF combined with SAP PI
This SAP framework provides functionality to implement value conversion in interfaces without or with very little coding effort via customizing transactions. These value mapping objects point to different SAP tables which are to be looked-up during run-time to retrieve the value needed. If a more complex mapping rule is needed, a custom function module can be developed and called by AIF to do the conversion. Another advantage is the re-use of value mapping objects in several interfaces. AIF even provides advanced error handling capabilities such as automated notifications and mass correction, which are also useful in correcting value conversion errors. The error notifications can be directed to different users based on their roles to speed up error correction and reduce coordination effort. That allows the functional end-user to do the correction directly. Mass correction of the same type of error in several messages can be done to make things easier. AIF can be combined with SAP PI, but it can also be used alone. Because AIF is only to be installed on each SAP back-end system, AIF cannot be used in pure non-SAP scenarios that only involve legacy systems. The AIF interface development, functional mapping and monitoring functionality complements the interface transformation, technical mapping and routing functionality of PI.
Requirements for a value mapping architecture
When developing a solution for value conversion, the following points need to be taken into consideration before choosing the appropriate SAP technology:
System landscape: Are SAP systems involved? AIF cannot be used in a scenario that involves only non-SAP systems. Nevertheless, PI can always be used as a middleware technology. If value conversion has already been implemented in PI for such scenarios, it might be appropriate to re-use it in scenarios where SAP systems are involved instead of re-implementing value mapping in AIF, if it is introduced afterwards. On the other hand, if there is at least one SAP system involved per scenario, you can consider moving functional value mapping to AIF.
SAP recommends that AIF be designed for functional value mapping, functional error correction and user notification. PI is designed for message transformation of different interfaces, message conversion of different protocols and routing of messages to different target systems. Technical interface monitoring is also to be done in PI.
This recommendation makes sense as both technologies have strengths and weaknesses and can be combined in an effective manner.
Maintenance: Who should maintain the value conversions? The technical team is responsible for IT-systems, like the middleware. Typically, IT teams do not update functional conversion rules. They expect the functional departments to maintain these values because it is their area of responsibility. However, the functional departments do not want to work with any other system besides their own. They consider another system, such as a middleware engine, to be an additional burden and double work. A solution is needed which is acceptable for both departments, functional and IT. This again emphasizes the split between technical and functional mapping rules for which each department is responsible. If AIF and PI are combined, then PI should attend to technical conversions and AIF functional.
Possession of data: As described, an easy solution would be to save the value conversions directly in the middleware. Other solutions would be to save the conversions rules in the sender or receiving system. It could also be possible to replicate conversion rules between several systems. In the end, the concept must take into consideration who the owner of the conversion rules is and who is responsible for updating them. A master system needs to be defined. The master system should be the system that is to be used over the long-term.
Interface dependencies: There can be situations in which dependencies between interfaces must be considered. Let’s say there are two interfaces between two ERP systems. One interface replicates vendor master data and the other vendor account payable bookings. Critical for the target system is that the vendor number of the sender is always converted to the vendor number of the receiver system, as they have different number ranges. If a booking is passed before the vendor master data was replicated, the booking might fail because the vendor number does not exist yet in the target system. Even if the vendor master data was replicated before, the conversion rule must be updated first, before the other interface is executed. This means that if PI contains the conversion rule, it needs to be updated with the new vendor master data by the receiving ERP system before it can process the interface.
Timeliness of data: The example involving the vendor master data demonstrates that the conversion rule base must always be up-to-date in order to be able to execute other interfaces successfully. On the other hand, country codes are rather static as they do not change very often. In this case, a solution might be sufficient by which the conversion rule base is not updated that often.
Performance: Performance is very important in cases where the end-user expects real-time processing. Other scenarios do not demand high performance, e. g. when data is replicated at night and the user expects the updates the next day. A middleware usually leads to more processing overhead and less performance. The performance drops even more when the middleware also needs to do look-ups in other systems, e.g. to find a conversion rule for country codes.
Re-usability: When the conversion rule base is replicated between several systems there is redundancy and always a risk in terms of obsolete data. Implementing a central conversion rule base eliminates redundancy and increases re-usability. However, it might lead to worse performance because each conversion request needs to be routed to the central conversion rule base. The non-functional requirements need to be taken into consideration here and the right trade-off between performance and redundancy needs to be evaluated.
Error handling: How errors are handled represents an important aspect. A distinction is made between functional errors (wrong values) and technical errors (incompatible message types). A decision must be made as to who is responsible for certain types of errors and how they should be corrected. Audit requirements, e.g. for financial bookings, demand transparency in this process which usually results in a copy of the original broken message. Especially when it comes to defining role-based error handling, one must distinguish between technical and functional errors.
Development effort: Different solutions for value conversions also mean different degrees of effort for implementation. This must be taken into consideration during evaluation to find a solution that is worthy of being implemented. The more SAP technologies are used and combined, the higher the costs for implementation, licenses and operations.
SAP architectures
When considering the architectures of several systems that communicate with each other, we must distinguish between a middleware and a point-to-point architecture.
In a point-to-point connection, the sender and the receiver systems communicate with each other directly, without PI. If the number of interfaces increases, point-to-point connections will be difficult to maintain. A middleware often reduces both the effort and the costs. Nevertheless, point-to-point connections still often exist because they do not generate additional overhead compared to a middleware. Point-to-point connections usually result in very good performance. In terms of conversion rules, either one or both systems may establish a conversion rule base. As for communication that involves SAP, the AIF is designated for such scenarios. It still allows for high performance and conversion for inbound and outbound interfaces. When communication between two SAP systems takes place, one system needs to be defined as the master and do the value mapping. The systems also remain autonomous and the users do not have to work with additional systems. To reduce redundancy, as few as possible systems should contain the conversion rule base.
Figure 4: Point-to-point connections
By using PI as a middleware, the central mapping instance is highly re-usable, but the performance is worse compared to point-to-point. The middleware needs to be maintained with the conversion rules, which can lead to more effort compared to local implementation with AIF, for example. When PI is used and SAP back-ends are involved, AIF can also be used, which is what SAP recommends. As both technologies enable interface development and operations, one needs to discuss whether tasks can be split usefully, especially because value mapping is available in both products. The technical IT is usually aware of technical mappings and they also maintain the technical system details. It makes sense to assign responsibility for the technical mapping to them and execute it in PI directly. On the other hand, functional mapping rules are usually defined in the back-end systems that the functional departments responsible use.
Figure 5: SAP PI as central middleware
Conclusion
Implementing a value mapping concept in SAP is a complex topic that needs to take several key points into consideration. Different scenarios are possible depending on the requirements and the usage of SAP PI and/or AIF. Ideally, SAP PI and AIF are combined to differentiate between technical and functional value conversion and to allow for role-based error handling. If only one of them is used, another approach will be needed. Also, different value mapping concepts can be applied within PI. Each one has strengths and weaknesses which need to be evaluated for the problem statement. The following table shows where specific types of mapping should take place.
PI only | AIF only | PI + AIF | |
Functional mapping | PI | AIF | AIF |
Technical mapping | PI | AIF | PI |
Table 2: Usage of SAP PI and AIF
The following table also shows the difference between PI and AIF in terms of value conversion as well as the advantages and disadvantages of each mapping concept in PI:
Mapping with AIF only | Fix Values in PI | Value Mapping in PI | Value Mapping Replication in PI | RFC Look-up in PI | |
SAP Standard | Yes | Yes | Yes | PI only Back-end custom development | PI only Back-end custom development |
Performance | Good | Good | Good | Good | Poor |
Development effort | Low - Middle | Very low | low | Middle | Middle |
Re-usability of values in message mapping | Limited – non-central approach | No | Yes | Yes | Yes |
Amount of pairs to be handled | High | Low | Low-Middle | Middle-High | High |
Default exception handling when no matching value was found | No restrictions |
| |||
Accuracy of data | Always up-to-date | Need to be updated by hand | Depending on replication interval, data can be outdated | Always up-to-date | |
Conversion rule maintenance | Back-end | PI | PI | Back-end | Back-end |
Table 3: Overall recommendation
- SAP Managed Tags:
1 Comment
