I recently ran into a fellow in a forum who was soliciting opinions from SDN folk about EDI subsystems. So I responded to his post with a little information about Sterling Integrator (SI, formerly GIS) from Sterling Commerce, which I have some experience with.
I also added some general bullet-point thoughts about what I believe an EDI system needs to have in order to be useful in the enterprise, based on my experience. Since I've seen the question asked in one form or another many times before, I thought it might make an interesting topic for a blog posting. These are general points that are valid for EDI systems whether or not SAP is part of the enterprise landscape.
Which brings me to another, often over-looked, thought: an EDI system is an enterprise application. This should be obvious to everyone. But my experience is that in the real world, this fact is often not fully appreciated.
I've seen EDI systems that process hundreds of millions of dollars worth of transactions every year built in a slipshod manner with no thought for trackable integration into SAP or any other business system, without even considering such basic requirements as being able to trace the process flow of an interchange from beginning to end or to identify a file in the archive if something goes wrong.
Vision and Planning
This speaks to proper implementation planning and architecture, to having an integration vision. This is standard stuff in the SAP world where there are strict methodologies, structured projects, and lots of controls.
As critical as EDI is the success of the business, and to the integrity of the data in SAP, the same care is not always taken to implement an EDI system and to run it after go-live, especially in smaller companies. This discrepancy also crops up during SAP implementation projects that include streamlining, upgrading, or replacing an EDI system.
This can lead to project delays and increased costs of as much as $45,000 a day and more in multi-million dollar ERP implementations, according to a study released on September 21, 2009 by AMR Research. The study was funded by GXS, which runs a major B2B exchange, a VAN (value added network), and sells EDI outsourcing services.
Yes, GXS has a really big dog in this fight and that's why they sponsored the AMR research. But these problems are real. I've seen them first-hand on large projects and in mid-size businesses that implemented SAP and EDI.
Implementation and running of the EDI system needs to be handled with the same care and devotion as the SAP system. The professionals who manage the EDI project and who design and build the EDI system should have the same level of skill as the professionals who build and maintain the SAP system.
That's my little sermon for today. Consider it a statement of principles that should guide the joint SAP-EDI implementation. But it's about more than just EDI. We're concerned with all integration requirements, with external trading partners and between internal systems. Integration is difficult and complex and it needs to be approached with vision and good planning.
So What Do You Need in an EDI System?
For now, I just want to talk about the basic requirements for an EDI system. What do we need at a minimum to make EDI work with SAP and other internal systems such as CRM or VMI or data warehouses or whatever?
I'm not advocating any particular approach to EDI here nor am I favoring building and maintaining an in-house over an outsourced EDI system, although I do love to build systems. And I'm not distinguishing between standalone EDI systems and EDI adapters for SAP XI/PI. I'm just outlining what I believe any EDI system must have before it's ready for prime-time, regardless of how it's built or where it's hosted.
These opinions are based on my experience with practical EDI: with evaluating EDI vendor offerings for purchase, designing and building end-to-end SAP EDI systems, and with running them in a daily production environment.
If I were responsible for selecting an EDI system for my employer I would look for these features and evaluate how the vendor handles them.
1. A Complete Library of EDI Standards
To me this is probably the most important requirement. EDI is all about standards, after all. I would not even consider an EDI system that did not include as standard all transactions and messages from all major EDI standards including X12, HIPPA, EDIFACT, TRADACOMS, VDA, and so on in all versions.
I would also insist on regular upgrade of these standards by the vendor. Software vendors routinely force their customers to upgrade even if an older version of their product serves the customer well. If the customer is forced into the expense of this upgrade cycle, the very least that the vendor can do is include the latest changes to all EDI standards with the upgrade.
It's not enough to support the EDI standards. They must be included complete in the package before you buy or the vendor must be prepared to provide them, including all upgrades to the standards. If you take on a new trading partner you don't want to find yourself scrambling to buy an EDI transaction or message or version because it's not part of your library. You need to be prepared for whatever your trading partners may throw at you.
I would consider this a deal breaker and could not recommend a vendor that does not provide the complete library of usable EDI standards with an upgrade path as part of the delivered package.
2. Trading Partner Management
This is about managing the relationship with your EDI Trading Partners. An EDI transaction or message is a legal contract between two parties that have a trading relationship that is defined by an agreement between them.
Trading partner management in the EDI system defines the technical details that governs this relationship, including:
- EDI trading partner IDs and qualifiers used by both parties in the relationship.
- Contact information.
- EDI standards, transactions or messages, and versions used to exchange data between the partners.
- Enveloping and deenveloping requirements for the trading partner
- Acknowledgement requirements.
- Communication protocols and connection parameters.
- Security requirements, encryption protocols, certificate requirements and so on.
3. Enveloping and Deenveloping
All EDI transmissions contain envelopes, regardless of the standard. Envelopes contain identifying data used by the EDI system to recognize the source and nature of the transmission and to make decisions about what to do with them.
Enveloping packages an outbound interchange while deenveloping unravels an inbound transmission.
An envelope is more than just a thing, an identifier. It's a complex process of data mapping and reformatting and the best EDI systems hide this complexity. In other words, you should not have to develop custom processes or write custom code to handle standard enveloping and deenveloping.
There are times, of course, when trading partners have funky requirements, like a need for an "intelligent" (imagine my finger quotes here) interchange control number that would probably require custom coding.
Which leads to another point for consideration that is shared by all of our other development requirements: you want robust and easy to use custom development tools.
4. Acknowledgements
This is a much bigger issue in North America than Europe. But North America — the United States and Canada — is still by far the greatest consumer of EDI in the world. Acknowledgements are required for every EDI transmission in North America.
This is handled by the X12 997, in its many versions, generally at the group, or functional, level, or the EDIFACT CONTRL message. This is why the EDIFACT group level is mandatory for transmissions to and from North America. In Europe it's optional although there are some EDI consumers who do acknowledgements at the interchange level using the CONTRL message.
Any EDI system that is going to be used for North American traffic must include the ability to automatically generate acknowledgements, whether at the functional group (GS or UNH) or interchange (ISA or UNB) levels. The level is determined by the trading partner agreement.
Like enveloping and deenveloping, this should be a standard feature that does not require custom development. I would not consider an EDI system that does not include automatic generation of acknowledgements as a standard feature.
5. Communications
The EDI system should include adapters and services to support all forms of communications including AS1, AS2, AS3, HTTP, HTTP/S, FTP, SFTP, file transfer, and so on.
You'll also need an SAP adapter that supports JCo (Java Connector) or XI/PI connections to move IDocs — ASCII through JCo and XML through XI/PI — into and out of SAP.
On the inbound, IDoc adapters use the Java connection classes in JCo to make RFC calls into SAP to log in and call function module EDI_DATA_INCOMING to kick off IDoc processing in SAP.
On the outbound, IDoc adapters receive RFC calls from SAP through a registered TCP/IP type RFC destination set up in SM59. The RFC call triggers a registered program or process in the EDI system to kick off processing and transmission of the outbound IDoc.
6. A Good Mapping Program
This goes without saying. You need to be able to convert EDI transactions or messages to IDocs and IDocs to EDI. You also need to be able to convert many other standards and custom flat file formats to and from many other formats.
The mapping program should be able to import metadata structures and qualifiers for use in maps in a number of standard formats including XSD, DTD, DDF, and so on.
It should also have a graphical interface that allows drag and drop linking of data elements on both sides, an ability to handle mapping of looping groups and levels within those groups to different looping levels on the output, and a flexible and powerful rules or programming language to accommodate data conversion and special processing requirements.
I do most of my mapping in code in my EDI mapping tool. I quickly realized when I first started to build maps that there are a lot of issues that can only be handled in code. You want the ability to write your own code in your mapping tool and to be able to create and use custom Java objects or to call Java customer exits to handle processing that can't be done in the mapper's rules or programming language.
7. A Graphical Business Process Modeling Tool
Business Process Modeling (BPM) is supported by SAP and pretty well all major EDI vendors on the market. It's a convenient and relatively easy way to build interfaces, allows for standardization of processes, and is practically self-documenting.
A graphical BPM modelling tool provides a workspace for linking together interface program objects into a processing chain that generates BPML (Business Process Modeling Language) or BPEL (Business Process Execution Language) for execution at run-time. BPML and BPEL are both dialects of XML used to define and run BPMs.
Typically, the objects in a graphical BPM tool are dropped onto a workspace and linked together with connectors. Objects are configured with parameters and provide services such as a call to a map or a timing event or routing to a communications adapter and so on.
Conditional processing is applied through rules, in XPath or some other language, at the object or connector level. These rules are evaluated during execution and control the process flow of the interface based on data that is only available at run-time.
This is all straightforward programming stuff, with a different face perhaps. And different systems implement BP modeling in different ways, but most do it. You want to be comfortable with how your vendor handles it, which also means that you should consider the skills within your organization, consulting and training requirements, before you make a decision.
People Matter
So now we come down to the crux of it all. Technology is only as good as the people who implement and run it. No matter how you cut it, you're going to need expensive consulting support to get up and running, at least until you can train your own people or outsource running your system to an external provider.
Even if you outsource, you need your own people to be involved, to understand how the whole thing works and to know when something is not right. You have a personal stake in this: it's your data and your business, after all, and nobody cares about it as much as you do.
The people issue comes down to this: the availability and quality of the training and the consultants that the vendor can provide. You need to be an educated consumer. What are your expectations? How can the vendor fulfill them?
Ask the questions and demand the answers. Accountability is your right as a consumer. Cut through the vendor's sales fog and focus on the needs of your own business. Consultants and the software publishers and service providers that employ them all too often forget that customer service — not selling software and services — is their job one. Don't let them forget.
4 Comments
