Every day businesses make payments to other businesses that are processed by banks. These payments are made by check, electronic funds transfers (EFT), ACH (Automated Clearning House network), or by other related means.
They include, or are followed separately by, a reconcilition report to the bank — a remittance or payment advice — that identifies key information about the payment including bank number and account, check or transfer number, amount, invoice number, reason and date of voiding if the check was voided, and so on.
The remittance is reconciled to the actual payment, allowing the bank to apply it properly, whether it was made by check or electronic transfer. The amounts reported in the remittance must equal the amounts in the payment in order for the payment to clear and your vendor to get his money.
There are standard IDocs and EDI transactions to handle the electronic transmission of remittances, including:
- IDoc: Message type REMADV with basic type PEXR2002
- EDIFACT: Message REMADV
- X12: Transaction Set 820
There are also standard flat file formats used by banks and other institutions for electronic transmission of remittances and payments, some of which predate EDI. In the United States there's NACHA, administered by the ACH and the Federal Reserve, which also includes a network and business rules to manage and reconcile funds transfers.
In Europe SEPA (Single Euro Payments Area) manages all electronic funds transfers as domestic payments for countries within the Euro Zone.
And then of course there's SWIFT (Society for Worldwide Interbank Financial Telecommunications) that manages electronic funds transfers on a global basis.
SAP provides standard payment programs, in Accounts Payable in FI, to generate bank payments in a number formats including IDocs. For example, report RFFOEDI1 can be used to generate a payment advice IDoc for transmission by ALE or EDI from a payment run. The IDoc can then be translated into electronic payment format supported by any bank or institution.
All these standard formats are widely used and supported. But the problem is that many large banks and financial institutions (such as American Express) also have their own custom flat file formats for payment reconciliation data. These custom formats have been used for years and even decades and are still being pushed out to thousands of bank customers around the world.
The sheer volume and immense value of transactions carried in these custom formats every day make it highly unlikely that they will be replaced by standard file formats any time soon. The custom payment reconciliation formats used by banks are working, have worked for years, and the costs of replacing them are higher than any benefits that might accrue from using a standard format.
And let's face it: the banks are so big that they can tell their customers how to send almost anything, including their reconciliation data.
One Off Thinking
So it's not an unusual task to build custom payment interfaces in SAP. I've had to build my fair share, even in the days before I became invovled with EDI and still saw integration as a question of developing file extracts that got picked up by scheduled FTP processes running in scripts. It was all a black box for me once the file had been written by my custom ABAP extract to a target directory on the application server.
The actual file transmissions to the banks are by a number of protocols. In the past, it was mostly through proprietary modem transfers. Today, AS2 and SFTP both provide cheap transmissions and plenty of security. SFTP (Secure File Transport Protocol) is especially popular because of its native support for PGP encryption and public key validation. But it's not always easy to set up if you don't have experience with transmitting data to trading partners.
Recently I've had the opportunity to revisit this issue. As at many other sites that I've worked in the past, payment extracts were handled as one-off flat file interfaces that were transmitted by calling a script that did the SFTP call at the operating system level of an application server. A number of interfaces were built in this way by ABAP consultants with little experience of SFTP or integration.
In the beginning, they worked ... or at least they seemed to work, although there was no way of knowing if the SFTP transfer was successful except by checking with the bank after the transfer. The consultants left, the interfaces failed, and it was difficult to get them working again. It was difficult to even recognize that some weren't working ... out of sight out of mind.
Well, not totally out of sight. The extract programs generated a payment reconcilation report — a check register — that was emailed to Account Payable (AP) after each successful run. But the extract programs couldn't generate an error if the SFTP script failed to send the file to the bank.
When it comes to integration, visibility is everything. After all, you don't know what you don't know. If an SFTP transfer fails in the night and there's no process to trap that failure but there is a report to AP telling them that the payments were successfully extracted, who's going to know that the bank didn't receive its reconciliation file?
This is the kind of challenge that I love. And luckily I had a starting point, a framework within which I could consider the problem: the EDI system.
Because I worked in both SAP and Sterling Integrator (SI), the EDI system I designed was built with maximum visibility of EDI processes in SAP. This was achieved in a number of ways.
- Inbound: Translation BP ID and EDI data such as trading partner ID, interchange and group control numbers passed to the control segment of the translated IDoc and sent into SAP.
- Outbound: STATUS IDocs generated and sent into SAP to update the outbound IDoc after translation (with status 06 for success or 05 for failure) and after EDI acknowledgement received from trading partner (with status 16 for OK or 17 for validation issue in partner's EDI system).
EDI Milestones In IDoc Control Segments
So information about key milestones in the EDI processing cycle was available in the control segment of each EDI IDoc.
Without giving away the store and without getting into the nitty gritty details, the system was also designed for generic processing of EDI interfaces. It features separate processes for communications, both with SAP through JCo RFC and the trading partner through AS2 or FTP; and translation and enveloping or de-enveloping, including STATUS reporting triggered by translation.
It included a cross-reference table in SI used by inbound and outbound transmissions that mapped SAP partners to EDI trading partner IDs by EDI transaction. For outbound transmissions, the table provided data for each trading partner transaction that allowed SI to identify the envelope and through that the map, the communications protocol, and the next business process (BP) to call to trigger transmission of the translated EDI file.
A BP in SI is defined with BPML (Business Process Modeling Language), a dialect of XML similar to the BPEL used by SAP in PI and in the workflow system through BPMs.
In SI, a BP can work much like an SAP function module. It can be developed to process one task, such as kick of an AS2 connection to a trading partner, that can be called by another BP that translates an IDoc to an EDIFACT or X12 interchange file.
The next BP call can be dynamic and can be determined by reading a table or pulling values out of process data with an XPath statement. Process data is XML formatted data available to the BP at run-time. It's put there by the system or the BP developer with BPML calls to services or adapters. It can include adapter configuration data, the results of a JDBC SQL read to the database, a document, or information that you know will be usedful to drive processing of the BP.
So the trick was to identify pieces of the existing EDI architecture that could be used to support the bank interface. And that was a pretty straightforward exercise but it also had an impact on the design of the overall bank interface approach.
The pieces of the existing EDI system that I was able to use included:
- In SAP the RFC Destination with its registered program and the RFC connection to EDI through the JCo connector.
- The receiving BP in SI that corresponds to the registered program in the SAP RFC Destination.
- The cross reference table in SI that maps SAP to EDI data to support outbound processing and transmission.
- The STATUS IDoc interface into SAP that reports on the results of translation.
The bank interfaces are substantially different from EDI interfaces, however, and they needed a new architecture designed to handle them. The bank files did not use EDI standard formats and did not support enveloping. EDI was mostly AS2 with some FTP, while the banks wanted SFTP transmissions with public key certification.
But the bank interfaces had enough similarities between them that a generic architecture could be built to accomodate them all.
I know that architecture is an over-used word. But in this context it fits. It describes a unified processing approach that completes a well-defined business process ... a payment reconciliation to the bank from SAP ... using two systems, two communications protocols, and a number of different development objects in both systems.
But all the stuff is invisible to the user. When everything is put together, the pieces disappear and what's left is a business process that kicks off automatically, sends itself, and then communicates back to SAP and the EDI team that the file transfer succeeded, or not.
The AP users see only the payment report. And at that point they know that the payment file was sent. They don't even have to think about it. If there are any transmission issues, they are immediately fixed by the EDI team without any user involvement or anxiety.
For all intents and purposes SAP and SI, and each of their objects and processes, morph into one seamless operation.
Stay tuned ... We'll see how that's done in my next posting.
Comments