Introduction to DI Construction Kit

[Former Member]

h2. What is DI Construction Kit ?

DI Construction Kit is a versatile, lightweight open source toolkit that aims to make integration and migration tasks easier in a SAP B1 environment. In addition to integration and data migration/update scenarios, it can even be used as a building block in custom application development.

This is the first in a series of blogs explaining the purpose and use of DI Construction Kit. This blog serves as an introduction to the tool and the following blogs will dive more deeply in details of use.

You can download DI Construction Kit from here  (http://tinyurl.com/4pcsccr).

A related article about configuring DI Construction Kit is found in

SDN B1 Articles section

.</p>h2. **Why yet another integration tool ?*

In my opinion, the B1 market has been lacking an integration tool that has the following properties:

• Fits to the budget of SMBs (no expensive license costs, simple scenarios must be fast to implement)

• Enables doing small changes in the configuration quickly and easily

• Is extensible so that the feature set of the toolkit does not become a bottleneck

• Can execute jobs both from the command line (enabling scheduled and event-driven operation) and from a “traditional” visual UI.

• Supports code reuse and transfer of knowledge in the form of “packaged” sample solutions

• Comes with source code so that the tool can be extended, debugged and self-supported if needed.

The above is what DI Construction Kit is all about. There is some overlap with +Data Transfer Workbench +and Copy Express, as well as with the Electronic File Manager that is to be delivered with B1 8.8.1. There is also some overlap with the B1i integration toolkit. However, DI Construction Kit takes a very different approach to integration. There are lots of cases where the mentioned tools either cannot be used at all or they have serious limitations.

There are also commercial 3rd party integration solutions for B1 (such as iBOLT), but at least the ones I know are simply too costly for most B1 customers. 

*Where does DI Construction Kit come from? *

* * The history of DI Construction Kit dates back to 2003, when I started working in SAP B1 integration and implementation projects. Back then, I realized that most of the code we wrote was “plumbing” code. A lot of it was almost identical from project to project, as a result of heavy copy-paste operations. I had the wish that we could separate the most repeated parts of those one-off solutions and distill them into a common platform. This would enable us to concentrate on the actual “business logic” part of the implementation in each project. I also wanted to be able to do simple but significant changes via configuration, without having to write any code at all. These configurable elements would include such things as connection parameters, path information of files to read/write, character set information, SQL queries used for selection etc.

**

 *****The dream*

*** *</strong>The “execution engine” would only need to be installed once on the customer site. Any changes could either be done directly in the customer’s system or prepared and tested offsite and then simply copying the modified scripts to the customer site. Anyone who’s ever done a few rounds of the develop-build-test-deploy cycle with “normal” development tools can imagine what this would do to your productivity.</div><p><p>Having the business logic defined separately would result in the essence of each project being squeezed into a very small amount of compact code that is easy to read. This would not only make the solution easier to maintain and reuse, but it would also help to see the purpose and logic of the solution directly from the code. Another benefit of separating the plumbing code from the business logic would be that even when there’s a new B1 version (with new DI API), you would only need to recompile the execution engine and deploy it instead of having to recompile each of the projects separately. There would be no need to do anything for the configuration files that define the business logic.</p></p><p>**h3. First steps towards the goal

</strong></strong></p><p><p>So much for wishful thinking. In 2004, electronic invoicing was a hot topic in Finland. The local banks had recently agreed on a common invoicing message specification and were pushing this standard as a replacement for the vendor-specific, proprietary “standards”. Being in the need of a fun and meaningful project for my Master’s thesis in Computer Science, I saw that this was my chance of killing two birds with one stone: get my thesis done and put the fantasy of a common integration platform into practice. During the summer, I was in contact with SAP Finland and we were able to strike a deal with them. SAP would pay us for implementing a Finvoice export interface that would be made available to all the B1 customers in Finland.</p><p>After some hectic hacking, the first official version of ++DocEngine ++saw the light of day on December 15, 2004. Compared with the DI Construction Kit of today, the first generation DocEngine was very limited. Still, the most essential parts were already there. There was the export feature based on the NVelocity template engine that enabled you to separate the static and dynamic parts of a document. There was even a limited import feature that used an XML-based instruction syntax to express the import commands (see sample below) and a “conversion” feature that allowed you to call predefined SQL queries for value translation. </p><pre><Import></pre><pre><header></pre><pre><Instruction></pre><pre><memberName>CardCode</memberName></pre><pre><memberDataType>string</memberDataType></pre><pre><xPath>SellerPartyDetails/SellerPartyIdentifier</xPath></pre><pre><conversion>Supplier_CardCode_from_TaxID</conversion></pre><pre><mandatory>true</mandatory></pre><pre></Instruction></pre><pre><Instruction></pre><pre><memberName>DocDueDate</memberName></pre><pre><memberDataType>date</memberDataType></pre><pre><xPath>InvoiceDetails/PaymentTermsDetails/InvoiceDueDate</xPath></pre><pre><mandatory>true</mandatory></pre><pre></Instruction></pre><pre><Instruction></pre><pre><memberName>Comments</memberName></pre><pre><memberDataType>string</memberDataType></pre><pre><xPath>SellerInformationDetails/SellerWebaddressIdentifier</xPath></pre><pre><mandatory>false</mandatory></pre><pre></Instruction></pre><pre></header></pre><pre><rows></pre><pre><rowPath>InvoiceRow</rowPath></pre><pre><Instruction></pre><pre><memberName>ItemCode</memberName></pre><pre><memberDataType>string</memberDataType></pre><pre><xPath>BuyerArticleIdentifier</xPath></pre><pre><mandatory>true</mandatory></pre><pre></Instruction></pre><pre><Instruction></pre><pre><memberName>Price</memberName></pre><pre><memberDataType>double</memberDataType></pre><pre><xPath>UnitPriceAmount</xPath></pre><pre><mandatory>true</mandatory></pre><pre></Instruction></pre><pre><Instruction></pre><pre><memberName>Quantity</memberName></pre><pre><memberDataType>double</memberDataType></pre><pre><xPath>DeliveredQuantity</xPath></pre><pre><mandatory>true</mandatory></pre><pre></Instruction></pre><pre></rows></pre><pre></Import></pre><p>The connection parameters, path information, selection queries and SQL transformation functions were stored in a configuration file, so that you could change the behavior of the application without the need to modify application code. At that time, the tool could only export and import DI API objects of type “SAPbobsCOM.Documents” – in other words sales and purchase documents. Hence the name “DocEngine” seemed appropriate.</p><p>Compared with today, perhaps the most important missing piece in DocEngine was that there was no real support for scripting. You had to get by with the limited – although Turing complete – programming language features of NVelocity templates for export. The Import feature was even more limited with nothing but the XML-based “language” for imports. The XML instruction code was very painful to use, as you had to write a whole lot of “code” to do simple things. In this sense, it was much like working with XSLT transformations. An additional burden was that there was no support for proper looping, branching or conditional operations. </p><p>Later on, I also experimented with using the NVelocity template language for use in imports, ending up with syntax such as: </p><pre>$doc.set("header","NumAtCard",$imp,"MsgHeader/MsgNumber","string")</pre><pre>#set($header=$imp.SelectSingleNode("MsgHeader"))</pre><pre>#foreach($date in $header.selectNodes("MsgDate"))</pre><pre>#set($attr=$util.readAttribute($date,".","DateQualifier"))</pre><pre>#if($attr == "4")</pre><pre>$doc.set("header","DocDate",$date,".","date")</pre><pre>#elseif($attr == "2")</pre><pre>$doc.set("header","DocDueDate",$date,".","date")</pre><pre>#end</pre><pre>#end</pre><pre>#set($freetext = $header.selectNodes("Freetext"))</pre><p>While this was a big step ahead from the Xml-based “language”, I was still not happy with the power of expression that it provided.</p></p><p>**h3. Checking out the alternatives

</strong></strong></p><p><p>Somewhere around 2005, I needed to implement a rather demanding XML-oriented EDI solution for SAP B1. As part of the preparations for this project, I attended the course TB1400 (SAP B1 Integration Toolkit, aka ITK) at SAP headquarters in Walldorf, Germany. I wanted to know what integration solutions SAP provided for B1 and if they would be of any benefit in my project: I did not want to reinvent the wheel.</p><p>Being totally R/3 centered - not to mention outrageously confusing - the Integration Toolkit did not prove to be of any help in the kinds of scenarios I was dealing with. On the other hand, I was still happy to attend the workshop. It helped me understand that I should not stand around waiting for a decent integration solution from SAP. Instead, I should go ahead and build it myself. Had I decided to wait, I’m afraid I would still be waiting.</p></p><p>**h3. Scripting done right

</strong></strong></p><p><p>I kept being haunted by the idea of a scripting solution that would let me combine and harness the full power of the .NET framework and the DI API. Luckily, it turned out that there was a runtime compilation feature in .NET – that was just what I needed. Once the runtime compilation feature was embedded in DocEngine, I was finally able to do what I intended to in the first place: express business logic with compact scripts that run on a platform which is under my own control.</p></p><p>**h3. Maturing up

</strong></strong></p><p><p>The EDI project proved out to be a fantastic test bench for the concept of DocEngine. Needless to say, a lot of new features were added during that project. Overall, I had the feeling that DocEngine was making my work easier and much more fun. Whenever I faced a need for a new feature, I could simply add it myself. I have always hated those black box solutions that are hard to debug and impossible to fix. I believe most developers would agree with this.</p><p>Since then, the changes in DocEngine became somewhat less dramatic. More and more projects were implemented without touching the source code of DocEngine, which proved that the concept was working. However, I kept refactoring and adding new features every time I realized I need one, simultaneously trying to make these new features as flexible and open-ended as possible. In addition to incremental changes, I also made a couple of more dramatic rounds of refactoring: basically broke it all to pieces and put it back together in a more meaningful and better organized constellation.</p></p><p>**h3. Going public

</strong></strong></p><p><p>In 2007, the full source code of the toolkit was handed out to all the Finnish B1 resellers. Very little came out of this publication, apart from a copycat Finvoice implementation (which I appreciate a lot, as imitation is the sincerest form of flattery). Back then, I was already planning to release the tool on the SDN, but SAP had some legal worries about publishing 3rd party binaries and the whole thing lost momentum (the funny thing is that they did not have any such worries about DI Commander, which is still available for download from the SDN). Since then, I always felt that before going public, I need to add this feature, fix that part of the code, write some more documentation, prepare more sample solutions - the list goes on and on.</p><p>In late 2010, I attended another SAP workshop which was about the B1i integration framework. I had very little previous knowledge about B1i before the course and I wanted to see what it was all about. Live and learn, B1i was essentially a polished-up version of the “good old” Integration Toolkit. This revelation gave me that final boost I needed to go and publish DI Construction Kit.</p><p>During the workshop, I was kindly given the possibility to demonstrate DI Construction Kit to other participants and the feedback from them was very positive. I also noticed that some of the participants had come to the same conclusion as I did before: they had started building their own little “DocEngines”.</p></p><p>I came to the conclusion that even with all the flaws and missing pieces in DI Construction Kit, it still is a nice little toolkit that deserves to be published. With all the effort I put into it, it would have been a shame not to share it.</p><p>**h3. Was it all worth it?

***

I haven’t really kept track on the hours I’ve spent developing and using DI Construction Kit, but I’m sure it’s a four-digit figure.

The aim of the tool was to make my own job easier and more fun, which it did indeed. Additionally, it has also made it possible for me to implement demanding projects within a very short timeframe. How about implementing a fully working group consolidation scenario in half a day? An import interface from a salary system in a couple of hours? I’ve been able to offer my customers working solutions with a very attractive price tag and delivery time. Many of these solutions would’ve never seen daylight if I hadn’t had this tool in my back pocket. Perhaps one of the main strengths of DI Construction Kit is that it’s been built and enhanced by someone who’s actually been using it in real-life scenarios. Eat your own medicine!

My sincere hope is that you will download the tool, experiment and tinker with it and hopefully find it useful. Whichever way it turns out, I would like to hear what  you think about it.

How can DI Construction Kit be used ?

There are three distinctive ways in which you can use DI Construction Kit:

0.1. As an interactive Windows Forms application

0.2. As a commandline tool that can be executed in a scheduled or event-driven manner

0.3. As a DLL library that you can embed in your own applications

***

Interactive tool

</strong></strong></p><p>The most typical way to use DI Construction Kit is via the Windows client. You can use the client to execute predefined jobs that either import or export data to/from SAP B1. Additionally, the Windows client includes a tool called Python Console, that essentially replaces the DI Commander tool (mmthat was released in 2007.</p><p> </p></p><p>!https://weblogs.sdn.sap.com/weblogs/images/32520/login_winclient.jpg|height=450|alt=DI Construction Kit Winclient|width=595|src=https://weblogs.sdn.sap.com/weblogs/images/32520/login_winclient.jpg!</p><p><p>**h3. Commandline tool

***

Using the commandline tool, you can execute the same jobs as with the Windows client. However, the commandline tool makes it easy to define scheduled or event-based batch jobs that run in the background.

| !https://weblogs.sdn.sap.com/weblogs/images/32520/commandline.jpg|height=168|alt=image|width=557|src=...!</body>