Back to the Future - Part 02

hardyp180 · ‎05-09-2012

Before I start, I would like to say that if this blog cuts off in the middle of a sentence, or even a word, that is down to the blog deciding to publish itself, without me going anywhere near the "publish" button. I discovered this "feature" of SCN a few days ago. If that happens I will just keep adding to the blog until it is complete.

OK, in my last blog http://scn.sap.com/community/abap/blog/2012/05/01/back-to-the-future--part-01 I talked about what I am trying to achieve and the historical background which caused me to want to re-write an application in a modern way.

In this blog I am going to discuss steps I have already taken to try and make custom programs more "portable" between SAP systems. This is going to be mostly technically but at one point I am going to go off on a flight of fantasy about my thoughts on documentating programs an repository objects.

In subsequent blogs I will try detailing my progress on creating the new version of the application - I have already started coding - and get into a detailed discussion of the "new" concepts and technologies involved.

At the start of each blog I am going to acknowledge and talk about the feedback that I got at the end of the prior blog. I had two examples - one to do with OO inhertitance, one to do with the best way to handle really complicated rules as to the field order in an ALV grid.

Rob Burbank mentioned that he had tried to put complicated rules into a table and it just made things worse. This sort of feedback is actually really useful i.e. "do not go there, I have done this, it did not work so well". In the UK the phrase would be "I would not touch that with a ten foot barge pole".

Our head office developer chief has an aversion to people creating loads of new Z tables all the time, so the Germans created a sort of dynamic application which was years before BRF plus, but serves the same sort of function as the BRF plus decision table, as far as I can tell i.e. A + B + C returns D as the result.

So, new Z tables are most likley not the way forward here.

Grhama Robbo suggested I use the Enhancement Framework to achieve my goals. I had not considered this; I had only viewed this as a way to replace "repairs" in standard SAP code. James Geddes wondered how this was better than using the native "polymorphism" (I hope I have got the term right) that comes with OO languages, as both approaches have the same goal. This is always the problem - you have multiple options open to you and it is hard to now which one to choose. More on this in a minute, but as one of my aims is to improve my personal knowledge, I think I will pick a particular case and code both approaches. then I will have a minor amount of practical experience in both and it might be more obvious which is better. I will put the code of both my attempts on the SCN for (hopefully) constructive criticism.

Wolfgang Doerner rasied several points.

Fisrtly, UML. I have come across this in several SAP books and I think I have got my head around the basic concept. My understaing that in Java (and indedd all OO languages) you start by creating the UML diagram and there are even some tools where you can graphically draw a UML model and then auto generate the Java classes and what have you, as skeletons for you to add the flesh to the bones. I suppose the point is that you do the UML diagram first, so you have the design right before you start coding.

In the book "Design Patterns in Object Orientated ABAP" by Igor Barbaric, which is great by the way, he shows the reader a UML diagram of his demo application and reccommends they print it out and refer to it often whilst reading the book to help understand what is going on. I went on the internet and got some UML stencils for Visio, some of which look like fish, but thus far I have only got a few boxes in my diagram. I am working from the above mentioned book plus the ABAP programming book by Horst Keller.

As I said, I am starting from the theories metioned in such books, and seeing how they bear up in real life, or indeed if I have understood what I was reading at all. For example my undertsanding is that you have to isolate the database access in its own class so you can use unit testing.

The second point was about using redefined methods in subclasses. I may not have been clear enough here, so I will give a more concrete example. The difference between country A and country B is not down to legal rules (it never is) but that the project team in country B wanted to re-invent the wheel and achieve the exact same outcome using a different string of SAP transactions.

In country B when you cancel a delivery, the desire is to keep the original delivery totally unchanged, but to create a return orders and return delivery, do a goods movement on the return delivery and there we go. In country A the desire is to reverse the goods issue of the original delivery, then set the quantity to zero, and set a billing block. In both cases the end result is more or less the same.

So, I would want the core program to say LO_DELIVERY->CANCEL( ) and dpending on what country the user is in, a different string of SAP transactions are executed programatically.

The third point was about ALV. My understanding is that in the latest version of SAP there is a new model for the ALV classes, so you no longer have to create a big empty screen filled with a component controller, or whatever it is called, you just call a factory method and you can even pass in an internal table rather than having to build up a field catalogue. That all sounds wonderful.

I had a look at the dynamic internal table wiki, I have seen something similar before, in a program i got from the SDN about how to do a table edit on any custom DDIC table.

In my case, I always want to pass in the internal table with all the columns, it is just depending on assorted logic, only a subset of columns are actually displayed, in a different order, and the rest are there but hidden. I could use the logic to build up the table dynamically and then pass that into the CL_SALV_FACTORY, but wouldn;t that need just as many lines of code as defining the field catalogue manually? Forgive me if I am missing the obvious.

My gut feeling is that I am going to need a field catalogue, as I am changing lengths and adding hotspots etc, so just passing the internal table into the CL_SALV_TABLE=>FACTORY is most likely not going to work for me. Naturally if there is a way to combine the best of both worlds, then I am all ears.

The last bit of the comment by James Geddes dovetails nicely into something I was going to talk about anyway.

Hammer To Fall

James made a reference to the famous proverb "when you have a hammer, every problem becomes a nail". this was in regard to me wodnering if BRF plus was appropriate in a certain situation as I am not sure yet here and when BRF plus should be used. I do know it is being used at the Australian Tax Office, so I hopem along with all other Australian taxpayers, that the tool works properly. Or that it doesn't work at all and gives me a 100% refund.

M.C.Hammer

the problem I face is bigger than this. As time has gone by, and I had to sit on a 4.7 system whilst new things get invented around me, every time I read a presentation on a new feature in ABAP, I added it to my list of new hammers. So now I have a whole shed full of hammers, all of which are looking for nails. Here is my list of hammers, each followed by a random thought or two.

Hammer House of Horrors

Oh look, the blog just published itself. This was in response to me pressing the button to have a list of bullet points. Oh well, one day the SCN will work properly, I am not holding my breath.

Code Completion - in ECC 6.0 you get more options. I will cover this in the next blog, but any further tips would be welcome.
Pragmas - this enables better documentation of extended syntax check warning situtaions. Again more on this in the next blog.
ABAP Unit - I need to use Test Driven development to put ABAP unit tests in new programs, and retrofit them to major developments (the difficult bit). once again, I give an example in the next blog.
Class Based Exceptions - not really new, I think I see an application for this when handling the user command aspect of the model
Enhancement Framework - to convert all repairs into user exits, and as mentioned above GR says this has a place in Z programs also. I talk baout this in blog number five.
BRF Plus - I know HOW you use this, I need to get my head around WHERE this should be used. An example of me playing with this will be in the fourth blog in this series.
Shared Objects - once again where is this applicable, and will HANA render this obsolete? An example is in blog number five.
Web Dynpro / Floorpan Manager - they speak for themselves
Internal Tables - where some internal tables are duplicated so then can be read two different ways, we can now use secondary indexes on internal tables. I think you can have dynamic where clauses also
Helper Variables - the ABAP language will now accept statements like IF ( A + B ) > ( C + D) like BASIC did in 1981, so many helper variables can be eliminated. I usedto code such statements on my ZX81 when I was14. I have missed them.
ALV - new classes are supposed to simplify all ALV reports, dramatically reducing the code needed, and improving memory management. Again, I read an article on this once, but now cannot find it for love nor money, so any pointers would be welcome. In my fourth blog I will give code samples of some test I have done in this area.
Persistent Services - supposed to help with databases updates _ i don;t really understand how, even after reading the SAP Press book on the subject. the whole concept seems over-complicated and full of glaring functional gaps e.g. fields in a structure have to be in alphabetcial order
Class Builder - you can define internal types as public, ths negating the need for creating tons of repository objects
Message Mapping - can this be usedto replace stupid SAP standard messages with ones that a human can understand?
ABAP to XLS - an open source project from the SCN code exchange - in my custom programs I want to add an option to download the data to EXCEL properly and to put this functionality into a generic re-usable repository class. Users love EXCEL. They always will. I talk about this in blog number five.
Now you can say READ TABLE ITAB INDEX LINES( ITAB[] ).
Workflow BOR - make sure note 1639167 is implemented to bring the ABAP editor in SWO1 up to date. this cannot be downloaded via SNOTE, it comes with support stack 11 for BASIS / ABAP on Netweaver 7.02
Netweaver Business Client - I understand that one day everyone using the SAP GUI is going to have the desktop NWBC version, whether they like it or not, so getting a grip on this seems like a worthwhile thing to do.
SALT - this is free, my company has installed it, RSC has a great track record, but can this product help in real life? More on this in a further blog, we have installed it in my company, after ten minutes I decided this was really useful.

Not all of this is relevant for the task at hand, but I thought I would include everything I could think of. If there are any new hammers I have missed, could you please ring them in the morning, and ring them in the evening, all over this land, and then tell me about them.

Seconds Out - Round Two

Now I will start on the second part of my blog. This involves what I have done so far, which I hope may prove of interest to some people, and once again, I invite comments like "you shouldn't have done it like THAT, you should have done it like THIS, you foolish fool".

As mentioned before, I had to copy over a large number of programs from one SAP system to another. As we all know, when you look at a given companies SAP system everything looks different from the last one you worked on - customing, number range assignment, the list is never ending.

When I looked at the custom programs in my system I discovered they had bucket loads of "dependencies" - that is assumptions that the program will always run in a systems just like the one in which it was written e.g. hard coding of number ranges. So, the aim of the game is to break all such dependencies.

The most obvious is language dependence - we need to use field symbols whenever you have text, but I had already done that, as you get prompted to do so by the extended syntax check.

Now, as it turns out I had already had some positive experence of transplating programs. A year before I went to Europe one of our divisions in Australia was sold off to a competitor and they were going to use the custom software I had written for their business line. As might be imagined some people in the new owner were really worried that major changes were going to be needed due to the above factors - customising settings etc so they arranged to hire me out as many times as was needed. I only went once as it turned out, as the programs worked more or less out of the box in a totally different system withonly a few changes. Just as well, as this was in Adelaide, and when I went the temperature was 47 degrees centigrade, which is just ridiculous.

Anyway, how can this be? How do you design a program so that it can be used in different systems if need be? Soo I will explain some technical methods I have used, but the first problem I had when I looked at the programs that were to be copied over was the complete and utter lack of documentation.

We all know why this happens - a mixture of time pressure and "can't be bothered". I have found that if I change a Z table, cutting and pasting the reason for the change from the ticket into the documentation within SAP and then changing it a bit ofr readability adds about three minutes to the development process, but this never seems to happen anywhere.

The Obvious - or is it?

As we know, the vasrmajority of custom programs get corrected and enhanced many times throughout their life cycle. The biggest risk is that when a develope other than the one who origianlly created the program - or even the same developer after a long enough gap - finds it very difficult to work out what the program is trying to do from looking at the code, given the total lack of documentation.

This leads to increased time to make changes in the best case and increased likelihood of errors and unforseen behaviour by the program in the worst case. This is all obvious, is it not? However, if it is so obvious, why do so few people seem to care?

Some ways to alleviate the probelm are as follows....

Documentation!

As I said just now, generally because of perceived time constraints, custom objects are not documented AT ALL. One of the first tasks I did was to look at all the custom objects from ten years ago and document what they did. This is a good way of finding rubbish to clean up. If I could not work out what a given object did then it was most likley not being used (and indeed this was usually the case) and once we were sure of ths the object could be deleted. At the very minimum this makes the drop down lists shorter when seraching for Z objects, and it also helped a lot during the upgrade i.e. less things to check.

Oh What a Tangled Web We Weave

So, I had two related tasks. To see what Z objects were actually being used, so I could delete the ones that were not, and to make sure all the ones that remained were documented. How does any programmer solve a problem? They write a program!

So I created a local program in the source developement system called YBC_DEPENDENCIES in order to automatically determine all needed objects for a target program or group of target programs. this also helps to arrange things into a package hierarchy i.e. several applications which live in their own ackage depend on utility objects which should live in a "helper" package.

This program also shows documentation for each custom object, where such documentation exists in SAP by the user pressing on the question mark next to each object. Within this report you can even get documentation on objects where documentation is not usually supported with SAP such as for Parameter ID's. I used general texts created via SE61 for this, with names programatically built up from the object type concatenated with the object name.

I start with inputting the program name on the selection screen, and then every custom object needed forits' existence pops up, using a recursive formula i.e. Z programs needs a Z function module, function module needs a database table, Z table needs a Z data element, Z data element needs a Z domian, Z domain needs a value table, farmer needs a wife etc. Thus this is a souped up versions of the standard SAP where-used list using the same database tables to get the relationships.

If a yellow question mark is mising then the object has not yet been documented so I press the pencil and the appropriate tool pops up to let me enter the documentation. When everything has q yelloq question mark, my work is done.

The "gotchas" is that if a data element is used in an IDOC segment definition then the standard SAP "where-used" list will not show this, thus SAP lets you delete the data element in question. I found this out the hard way.

Dixon of Documentation Green

Anyway, as we know, programs should be documented at several levels. At the header level (i.e. at the level of the Z object itself as in Goto > Documentation), at the start of subroutines (methods), and before blocks of processing code withinh subroutines. In SE24 you can stick documentation all over the place e.g. at method level. I wonder if anyone ever does?

Documentation should focus on WHY the code was written, not HOW it achieves this goal, though you may also need the latter if it is not obvious.

Examle - in some of my code I change the "material freight group" in a delivery under certain circumstances. I could write a comment saying "I am changing the material freight group". What value does that add? Anyone could tell that from the code. It would be better to write a comment such as "the business rule is xYZ and this means we have to change the material frieght group when Jupiter is in conjunction with Saturn, otherwise the driver is paid the wrong amount".

Magic Numbers

I found a comment saying "we add 12 to the maximum allowed value, and subtract 12 to get the minimum allowed value". You could tell that from the code, but the question is WHY do we need a range of values? Where are those values used? Even worse, at some point in the past "the business" had decided that actually 10 was a better increment than 12, and so the code was changed but the comments were not. Oh dear. I understand this concept is known as "Magic Numbers" when someone looks at some code and sees a number and has no idea of the context.

As we will see later I would change:-

LD_THING = LD_THING + 3.

To

LD_THING = LD_THING + GC_DEFAULT_BONUS or some such.

Variable and Subroutine Naming

It has been claimed that if you use meaningul names for variables then the code becmes largely self documenting.

For example

A = B + C

Becomes

LD_TOTAL_LOADING_TIME = LD_PLANT_LOADING_TIME + LD_MATERIAL_LOADING_TIME

It might be longer, but the casual reader can tell what is happening.

I have adopted a naming convention for variables from a blog on SCN by Uwe Schieferstein. The letters at the front are to tell a developer the nature of the variable e.g. if it is local to the subroutine, or available throughout the program (evil in OO terms, but remember I was changing a very old non-object set of programs, and futhermore the devlopers in my companies German office looked on the idea of OO in horror). The developer therefore knows to take extra care when modifying a global variable within a subroutine as this could have effects elsewhere. Here is the link to the blog:-

uwe.schieferstein/blog/2009/08/31/nomen-est-omen--abap-naming-conventions

The jump into the new SCN has RUINED the above blog in terms of formatting but the principle still stands. You will see me putting my two pence worth into the discussion.

In essence I have adopted the naming conventions from the above blog so that at least two ABAP developers in the world will name things the same, which is one more than before. Given that lots of developers will work on one program you will get a pot noodle of different naming conventions in different places. Standard SAP code is no exception, if anything it is worse. Here comes a great example.

I Know This Much is True

Generally you would expect a Boolean variable to have two possible values 'X' = TRUE, space = FALSE.

However, when looking over our custom programs, and even, horrifyingly, standard SAP functions, there was no consistency whatsoever. I found the following possibilities.....

TRUE FALSE

X space

space X

T F (this was a standard SAP function module for comparing internal tables)

Y N

J N

A B

I decided that withing the area of my control I would standardise on ABAP_TRUE and ABAP_FALSE and when a standard SAP function came back with "T" or "J" or something to instantly convert the value to ABAP_TRUE or ABAP_FALSE, in case that value was needed further on down the code.

This is just an example of what happens when you have ten dozen programmers all working on the same thing over a long period of time.

What do we call a subroutine? How about F09654325?

Naming is even more important for subroutine names. I found a subroutine called "CHECK_DELETE_USER_SELECTION" which did about ten different things, none of them connected to the name. This needed to be split into several smaller subroutines with names reflecting their purpose, before any new developer would have the slighest chance of working out what was going on.

I have seen lots of examples of using numbers for subroutine names. That is a good way of hiding what the routine does. With OO I don;t think I have ever seen someone use a number as a method name, though it would not shock me.

Removal of Unused Variables and Code

The extended syntax checks avalable in SAP tell you when variables are declared but never used, and also which subroutines are never called.

removing both of these has miniscule performance benefits, the real gain is that the program becomes easier to read, and less time is needed to make any changes. A possible example is that during an upgrade you could take many hours correcting the syntax of a subroutine which is never called.

In addition, programs can fill up with "dead code". Sometimes when you change a program you leave the old code there commented out to make what you changed obvious. This can lead to litterally hundreds of lines of "dead" code making the program impossible to read. An old custom function module I found used to be like this, it consisted of 90%+ commented out lines with the odd active statement sitting on it's own on a desert island surrounded by oceans of dead code.

You can always tell recent changes by using the "version management: utility in the ABAP ediotr. You would think this was not a secret, but looking at huge chunks of dead code, going back years, I wonder.

Modularisation

ABAP lets you nest IF statements and the like up to a depth of 256. However anything with a nesting level over about 5 becomes impossible to understand without a great deal of effort.

So I split up really long sections of code into smaller subroutines.

e.g.

LOOP AT INTERNAL_TABLE.

IF CONDITION = 1.

... 300 lines of code

ELSEIF CONDITION = 2.

... 200 lines of code

ENDIF.

ENDLOOP.

Becomes....

LOOP AT INTERNAL_TABLE.

IF CONDITION = 1.

PERFORM SOMETHING.

ELSEIF CONDITION = 2.

PERFORM SOMETHING_ELSE.

ENDIF.

ENDLOOP.

Time was on my side

Luckily, I had a number of months whilst the project documentation was being prepared during which I was not allowed to copy over the programs. I spent this time changing our existing system from one which was not documented at all to one which was almost totally documented.

For example at the end of the process, if you were on a custom Z screen and pressed F1 on any field you would get a "war and peace" description of why the field was created and what impact the value in that field had on the business process.

To Err is Human

Whilst I was going through the code with a fine tooth comb, I also looked at all the Z error messages. At that stage, they were all just like standard SAP messages i.e. a cryptic statement saying something was wrong with the "self-explanatory" box ticked.

So, each time I would get my head around the code to see what was causing the rror message to be issued, and added in a long text saying what had happened, why it was a Bad Thing, and most importantly, what to do about it.

Moreover, as you can put trasaction links into the long text of an error message, often in the code I would do a "SET PARAMETER ID 'THING' FIELD "SOMETHING"' just before calling the error message, and then in the long text in the "what to do" section I would say "check the schedule agreement to se if the delivery schedule has been mainatined" and then provide the "call transaction and skip first screen" link, and when the user clicked on the link in the message they would be in the transaction to have a look.

Even better, in the following example, the error is down to a missing customising entry and you can provide a link straight to the missing customising table. the user is unlikley to be authorised to correct or add the entry themselves but it provides useful information to the service desk.

SET PARAMETER ID 'DVI' FIELD (Name of Custom Table)

MESSAGE e123 WITH etc etc etc and the message long text has a link to SM30

Item Category ZXZ6 - Everyone Knows what this means

To state the so-called obvious once again, the reason that SAP programs, such as sales order entry, can be used by many different organisations worldwide is that they contain no hard code values but instaed use many hundreds of customising tables where business rules can be set up by each organisation.

In contrast, many companies that use SAP tend to fill their custom programs with hard coded values that relate to the customising entries they have created specifically.

As an example, this is something that was in a custom program:-

This has the following problems

- the causal reader of the program has no idea what values like 'ZNOR' mean

- if we add a a new item category which should be treated in the same manner as the existing ones we have to modify the program

- if the program is transported into a new system the customising values will 99% likley be different. There may already be a ZNOR value which might mean something completely different. Thus you cannot predict the behaviour of the program in this new environment.

In the Australian system a list was drawn up by me of all such hard coded values in all of the major developements earmarked for inclusion in the German system. There were MILLIONS.

A "general purpose customising" table was then created to store such values, alongside their business meaning.

Values can be specific to a given country, or more fine grained e.g. specific to a sales organisation or company code or even to a given plant. In the example above if we had a new material group set up for concrete we would add it to this table and then the programs that read this table would not need to be changed.

It has been argued, by the ultra-catious Germans, that such an approach is dangerous, as when you change a value here you do not know what you will be affecting. to mitigate this, we have added a "where used" button by the side of each entry, which gives you a list of all programs that use this value.

You can then double click on a line and see the source code that makes use of the value. We would also say this makes this safer than normal SAP customising. When you change a normal SAP configuration table it can also have far reaching consequeneces, and there is no real "where used" list for the EFFECTS of SAP customising tables.

The Australian custom programs were then changed so that the hard coded values were replaced by single value variables (where you are assigning a value) or ranges (where you are reading a vlue) that get populated based upon the global customising table as soon as the program knows what organisational elements are being processed. You generally want a range if there is the slighest possibility another entry could be added at a future date with a similar meaning.

The example from above becomes...

The constants now have more meaningful names - we now know that a ZNOR is a service item which would not have have been immedaietly apparent before. Looking back I would have expanded the name further to get rid of the cryptic "R" at the end, and say what it actually means.

A better example is as follows:-

This is the case where we currently have two concrete material groups, and now we can add another one without having to change the program. In the screenshot above we have kept the term HAWA hard coded as this is a standard SAP value with the same meaning in thousands of other SAP systems.

This also means that in each custom program there is an area which lists all customising values that influence the behaviour of the program.

I knew there would be cases where the whole concept would be different and would have to deal with such exceptions on a case by case basis.

That was That

So, I did all of the above, and then copied the programs over (via SAPLINK) in a state where they were documented and free of hard coding, with meaningful variable names. At that stage SAPLINK didn;t actually copy over documentation, but someone on the SAPLINK site had posted how to do this, so I enhnaced all the methods. I think this feature is standard now.

Ghost of Christmas Yet to Come

So, the first blog was The Ghost of Christmas Past, explaining the historical background of what I am trying to acheive, this blog is The Ghost of Christmas Present, explaining what has been done so far and the current state of things, and in subsequenet blogs I will move on to the creation of a new version of the example application, one small step at a time, at every step talking about the "new" concepts and technology involved and inviting a debate each time.

Cheersy Cheers from Madrid

Paul Hardy

Subsequent Blogs:-

http://scn.sap.com/community/abap/blog/2012/05/30/back-to-the-future--part-03

http://scn.sap.com/community/abap/blog/2012/06/16/back-to-the-future--part-04

http://scn.sap.com/community/abap/blog/2012/08/06/back-to-the-future--part-05

http://scn.sap.com/community/abap/blog/2012/08/17/back-to-the-future--part-06

http://scn.sap.com/community/abap/blog/2012/08/31/back-to-the-future--part-06

Other OO Blogs:-

http://scn.sap.com/community/abap/blog/2012/09/08/design-by-contract-in-abap

http://scn.sap.com/community/abap/blog/2012/10/27/crossing-the-great-divide--procedural-vs-oo-abap-p...

http://scn.sap.com/community/abap/blog/2013/01/08/domain-specific-language-in-abap