ABAP Objects is the Way Forward – I Want to Believe – Part Two
http://scn.sap.com/community/abap/blog/2013/07/04/abap-objects--i-want-to-believe
The above blog – first part of this series - can be summarised as follows:-
· I want to prove that writing new programs in OO really is better than the “old” way
· I have a real life project which allows me to do just that
· I am going to try to document how I go on this step by step
· The example will be silly, but based on the real project
· I am using test driven development, so I am starting with the tests
I got as far as writing one test, which does not even compile yet. The next steps are as follows:-
· Try and create my unit tests properly
· See if I can convince my colleagues that unit testing is a Good Thing
· Develop the program with regular user feedback
· See if anything I develop in this project can be re-used in subsequent projects
I apologise I advance, this is far too long, as usual, and I go off topic all the time, also as usual.
Why am I doing this in the first place?
I am now stuffed full of new concepts from reading assorted OO programming books, and am only half way through Robert Martins “Clean Code”, so a nice big new programming project gives me ample chance to try and see if any of this can be applied in real SAP life.
If I write blogs all through writing this monster application, then I am forced to explain what I am doing, and often if you try to explain what you are doing, it makes you think about what you are doing, and that in turn sometimes leads you to think “WHY am I doing that? I must be a madman”, and then a better way may suggest itself.
If I publish my reasoning on the SCN then chances are a better programmer than me will read this and tell me where I am going wrong and point me in the right direction. Conversely if a newcomer reads this then they might get some helpful hints on how to things, and as I said even if I am wrong then the comments should hopefully correct the myriad mistakes I make.
As anyone who has read any of my prior blogs will know, I am also going to continually wander off the subject and then slap myself round the face with a fish to get back on topic.
So, off we go, let’s talk about UNIT TESTS.
UNIT Tests
Figure 2.1 Unit Tests
It is time to talk about the scope of the unit tests. When I published the last blog, lots of people said you should not test private methods. If you look on the internet you can see a lot of debate about this with people very passionate about this, some say yes, some say no, though I get the feeling the consensus is “no”.
The argument seems to be that if you test the public method, then if one of the underlying private methods breaks then that failure will propagate itself to the public method, so your test of the public method will then fail.
That way you still know instantly if your change has broken anything, and then you can change the implementation of the public method (i.e. refactor it) as much as you want without having to change lots of low level tests. That logic sounds good I have to agree.
I am not trying to give away too much about what I am doing in real life, but it is going to become obvious it is all to do with the SAP Variant Configuration module. In my example I am building a monster for Baron Frankenstein based on various desired monster attributes, in real life my application is going to be a bit of a monster also.
I have one massive unit test, what I call a user acceptance test. This takes the desired characteristics and the output is a bill of materials from which to build the monster.
The basic structure is one public method, which under the hood will call about a dozen private methods, each of which represents a square on a VISIO flowchart. Each “square” in turn will call about ten other private methods.
A Finger of Fudge is Just Enough
From reading the assorted literature about test driven development it seems there are three rules, and you are supposed to go around in a really small circle, a circle Robert Martin says should take only a few minutes.
· No code can be written until you have a failing Unit Test.
· Only write enough of the Unit Test to fail.
· Write just enough code to make the Unit Test pass.
OK, so I have a failing unit test, my very high level monster making user acceptance test. I know the inputs and expected outputs.
I need to actually define the implementation of the class under test and the main method which is being tested. I can create an empty implementation of this main method, and then in my test class I have a test method which sets up the inputs (GIVEN) calls the main method of the class under test with the correct signature (WHEN) and then does some assertions to make sure the output is correct (THEN).
Everything now compiles, I can get as far as doing the TEST -> UNIT_TEST from the ABAP editor, and get a screen showing me a red light because all my assertions fail. I have now finished the section part of the TDD cycle, I have written JUST ENOUGH for the test to fail.
I now need to write just enough production code to make the test pass. Hang on, that is ALL OF IT, all one hundred odd private methods. Somehow I don't think I am going to be able to do this all in one go, and not in under a minute.
The only way I am going to be able to do this in a way that attempts to follow those TDD rules is to start with tests for the smaller methods – which are all private – and work my way up.
As I say repeatedly I have a big VISIO process diagram, and for each square I have an equivalent part of a spreadsheet, and I know what values go in and what values are supposed to come out. I will start doing a unit test for each square, and after creating each test I will write the production code. “That’s the way you do it!” as Mr.Punch would say. He was a big fan of test driven development, when he wasn't stealing sausages.
ABAP Unit – Sharing is Caring
The other day in my company we had our annual programmers get together where we take a day away from sorting out the users problems and focus on technical things. Ten years ago when we started this we mainly focused on what SAP would describe as “customer program optimisation” where you look at all your Z programs for performance problems.
As the years go by our relentless focus on this matter means this is no longer a problem in the slightest, so now I have moved the focus to looking at the myriad new technologies available to us with the advent of ECC 6.0 and similar.
Since I had done quite a few programs with unit tests in them by this stage I thought I would explain to my colleagues what this was all about, as I knew for a fact I was the only one using this tool (ABAP Unit).
I had been so impressed by the – to me – obvious benefits of an automated testing framework that I thought this would sell itself. However this was the reaction:-
Figure 2.2. Unit Test Reaction
I had, in my audience a wide variety of people, quite a mixed bag, all of whom had at least ten years SAP experience. None had heard of ABAP unit.
I like to think I can explain something well when I put my mind to it, and I showed various programs I had written with unit tests, and wrote some lovely pretty pictures on the whiteboard like the consultants do. My boss said I had explained it well, so I thought this was going swimmingly.
Then the questions started. No-one expects the Spanish Inquisition!
So…..
· No, this test driven development / unit test business wasn’t something I had invented myself
· No, this wasn’t something I had got off the internet like ABAP2XLS
· Yes, it’s actually a part of standard SAP, a part of SE80
· Yes, it does mean you write twice as much code, takes twice as long to write something
I thought I could get around that last one, by doing another consultant trick and drawing a pie chart. Consultants can convince anybody of anything my drawing a pie hart or dividing a white board into four and writing the option they want into the top right hand square.
Figure 2.3 Consultant Diagram 1
I have seen that technique used millions of times, it usually works, it didn’t work for me - so I tried a pie chart. This time I actually had a bit of logic on my side.
Figure 2.4 Consultant Diagram 2
The point I tried to make is that since 95% of the effort expended on a program is on maintenance, then doubling the time spent on development does not matter that much in the grand scheme of things, if it makes the other 95% of the life cycle faster and more stable.
I am so lucky where I work. My boss has an open mind. Initially the idea that most of the effort on a program was maintenance came like a bolt out of the blue to him, but when he asked the other programmers if this could possibly be true, and they all nodded, he got it straight away.
However he raised the obvious point that how can you convince a business manager that something they want right now will take twice as long to delivery initially, because in the long run they will be better off. Most human beings do not care at all about the long run, just what they can get right now, right this second.
There was a cartoon where Charlie Brown had to go away for a few days and he left Snoopy with three bowls of dog food, and told him not to eat them all at once, but one per day, otherwise he would go hungry. Snoopy ate the first one, and then got more and more agitated and the dive on and ate the other two, and then led back and said “I’m glad I did that – what if tomorrow never came?”
So, I don’t think management are going to go for this in a big way, and I am fairly certain none of my colleagues are going to start using unit tests. Worse, in nine months time, when they change one or my programs that has loads of unit tests, they won't run them after the change to see if anything has broken.
However, my boss asked me “have you any proof that this works?”.I replied that I had stuffed a procedural program full of unit tests, and once the program was in production the requesting manager had never once complained, and for the last ten years he had ALWAYS complained that what we wrote was never good enough and always came back half a billion times with corrections to be made.
Maybe if I make a go of this, it might encourage others….
Waterfall Am I
Talking about end users, I have a highly nervous user base, some of whom have grave doubts whether this project (which I am working on and is the subject of this blog, not that you can tell this as I keep diving off down rabbit holes) can be achieved at all. The traditional method of dealing with an enormously complicated SAP project like this this was to take the original user requirements, go and lock yourself in a room for two years to build something, and then give the finished product to the user who wrote the original requirements and be really shocked when they say “this is not what I wanted at all” and then you have to tell them "sorry this is what you’re getting, the projects run out of money, we’re doing a new one next week”.
This is the “waterfall” method and it is how most of the major SAP implementations I have observed have worked. A few years either side of the year 2000 a lot of consulting companies made all the money in the world using this method. The secret to their success was they could hand over the finished product and then run out the door.
Figure 2.5 Waterfall Projects
If you look on the internet you would think that this is a thing of the past – nowadays everyone uses extreme programming / agile / scrum” type methodologies in the same way all ABAP programmers use OO programming. Now, is that true?
Just two weeks ago a colleague of mine from America asked me what in the world “scrum” meant in an IT context, and I gave a simplified explanation which revolved around showing a working prototype to the users every two weeks which just the functions you have built thus far, as opposed to showing them everything after two years.
He said that sounded great but it “probably wouldn’t work on a massive SAP project”. “Plus Ca Change” as the French say. I wonder how many “waterfall” projects are being done on SAP systems worldwide at this very instant. Probably enough to make the authors of the “agile manifesto” wonder why they bothered.
The End User is Nigh
I was dancing around the room with joy today when in the weekly ABAP meeting one of my colleagues said that Michael Bubble was really happy with my colleague’s new SAP custom development. I thought that was great that he could take time off singing to test our programs, but it turns out this was a different Michael Bubble, this one was an end user.
It turns out the decision had been made to go live with this development but only let one or two users have access to it, and then they would come back with a non-stop stream of bug fixes and enhancements until the development was fit for a human to use. Then everyone else could have it. You might think one or two users is not enough of a broad base to be representative, but it is a hell of a lot better than zero users looking at it before it went live for the whole company, as appears to be generally the case according to anecdotal evidence from all over the world.
The method above is still a waterfall pig with some agile lipstick on, but how much better would it be to be able to show the end users something every two weeks, or more likely to be able to sow the constant stream of visitors from assorted areas of the business what we have so far, as and when they arrive. I am going to be allowed to do this, it helps that I know a load of the end users, having been in the company for a long time.
As I am going to break down the development into bite size chunks, one VISIO square at a time, it is likely that each week I will have one or two squares each week finished. However a unit test is a bit of an esoteric thing to show somebody, especially given that if the test passes you see nothing at all except a blue line at the bottom of the screen. Strangely enough, that does not blow the socks off the end users and make them say “my god I am impressed, all my worries have melted away”.
Simulate, late, breakfast show
My idea was to have an executable program that showed the sample input data on a selection-screen, and then call all squares thus far written one after another and use the application log to tell the user in exquisite detail what is going on, so they can see the logic for themselves and if they don’t like it, start arguing with the guy who wrote the specification, literally over a year before it is likely to reach its final state and be foisted upon them.
Here is a great SDN article about using the application log to display complicated information:-
http://wiki.sdn.sap.com/wiki/display/profile/2007/07/09/Message+Handling+-+Finding+the+Needle+in+the...
I could not find a practical application for this until now, this is a perfect fit for what I want.
Break Down and Cry
I am going to break down my monster making process as follows:-
· Each square is a “main process” – it will have it’s own unit test
· That square is usually a series of sub-processes. If that process is not utterly basic (like adding two values together) than that gets its own method
· Each sub process is a series of process steps, in which you either get some data from the database, or do some sort of calculation on that data. Since I am just working towards building a BOM based on a set of characteristics those are the only two things I have to worry about.
My simulator program will just call all the squares thus far written one after the other and then display an application log in a nice tree format, so a user can drill into the bits they are interested in.
How you break up methods seems to be the subject of a fierce debate in the OO world, with people burning each other at the stake for heresy if they have an opposing view on this subject. What is this argument?
You’ve got that ONE THING
Robert C.Martin says that a method should do one thing only and do it well. In procedural programming it is not uncommon to see a FORM routine ask the user something, then go off to the database and fill up an internal table, and then loop over that table doing assorted things to the data. If you are REALLY lucky the call to the ALV (or WRITE statements if you are where I used to work) is in another FORM routine.
How often have you seen this:-
START-OF-SELECTION.
PERFORM GET_AND_TRANSFORM_DATA.
END-OF-SELECTION.
PERFORM DISPLAY_DATA.
It’s a good job that virtually nobody understood what END-OF-SELECTION really meant, otherwise instead of two FORMS there would be just the one (in case you are wondering END-OF-SELECTION is only meaningful if you are using a logical database in your program).
I’ve noted before that many people have not even made the leap to procedural programming yet; on the 13/09/2013 I just looked at a new program written on the 01/09/2013 and there were not even any procedures - everything in one big list after START-OF-SELECTION. That is the opposite of what Robert Martin is looking for.
He once wrote a blog called “extract till you drop” where he took a Java program and broke it down into smaller and smaller chunks. That got a load of arguments from both sides as to whether this is a good thing or not.
The ABAP programming guidelines book from SAP Press says “modularise don’t atomise: and when the following blog was published:-
http://scn.sap.com/community/abap/blog/2013/07/09/why-should-we-write-clean-code
… you will note the very first comment says that “light weight” classes i.e. classes or methods that don’t do very much, are more suited for Java than for ABAP.
Well I don’t know the answer, I am still a beginner, I am going to split things up in a way that seems right. Everything is going to be a local class / method inside one Z program to start off with, hopefully the more I get into this the more obvious the correct way to split things will become.
It has been said that is one of the benefits of test driven development – it forces you into having the correct structure. We shall see….
Last of the Cohesions
I find that often when I read the reams of material available about how to write object orientated programming you get seemingly contradictory instructions. Whilst that is frustrating on the face of it, the good thing is it forces you to think about both ways to do something and make your own mind up. It is better to have two options than to have no clue at all as to what to do.
As an example, I mentioned earlier the idea of “high cohesion” which says that if a variable gets used in all the methods of a class then you should make it a “member” variable (sort of like a global variable but only within one instance of a class) as opposed to constantly passing it backwards and forward in the signatures of all the methods. This would make the signature simpler.
My gut feeling however was to stick with the big signature as it made it obvious exactly what a given method needed to do its work, and what values it defined or changed.
There is a saying in some places in the world that if you don’t like the weather just wait a few hours until it changes. It seems to be like that with the OO books I read, if I decide to do something then sooner or later I come across an argument which justifies my opinion.
In this case we have the idea of “temporal coupling”. This says that it is a Bad Thing if you have two methods, and if you change the order in which they are called then the program behaviour changes unexpectedly in a way that is difficult to debug.
I am making no secret that the area I am working in at the moment is variant configuration, and I have a series of methods which correspond to VISIO squares and the point is that it is highly likely the flow chart will change and then the squares will get called in a different order. They say that a lot of the art of programming I making a guess as to what my change in the future so you can prepare for it, and to my mind the order of the squares in the flowchart changing is a virtual certainty. In a previous project I am aware of a similar flow diagram changed twenty seven times before everyone was happy with it.
In conclusion, I think I am going to stick with having big signatures in my methods until such time as I find a reason to reverse my position even stronger than the argument that made me choose that position. This might well happen, that is what is so good about learning new things, you can’t be like the sheep described by the Housemartins – “they never questioned anything, they never disagreed, sometimes I think they must have wool in their ears”.
In unit tests however, I have almost everything as member variables, so I can make the code look like English sentences with virtually no signatures at all. Those methods always run in the same order GIVEN – WHEN – THEN so there is much less danger of “temporal coupling” errors.
The Revenge of Domain Specific Languages
Figure 2.6 Domain Specific Language
Here’s a little blog I wrote, you might want to sing it note for note:-
http://scn.sap.com/community/abap/blog/2013/01/08/domain-specific-language-in-abap
The idea of a “Domain Specific Language” is that a business user could look at your code and understand what was going on at first glance, because everything was couched in business terms.
I was going all out to try and achieve this, and had extracted methods for each line in the spreadsheet so that the sub process methods consisted of lines like DO_SUCH_AND_SUCH( ).
I copied the names from his spreadsheet. Then when he asked to look at the code I thought he was going to be pleasantly surprised but he looked and said “I don’t understand a word of this”.
That is because I had still kept some ABAP type words in the code e.g. INITIAL which no-one would understand, and also because I had used the same abbreviations as he had in his spreadsheet. I should really have expanded them back out again to their full English words, we have 30 characters in ABAP - I like to use as many as I can.
So, I went back and tried again. It can be done – here is a great blog on the use of macros…
http://scn.sap.com/community/abap/blog/2012/10/28/use-macros-use-them-wisely
… and with a mixture of them and small methods named like English phrases, and then you can have code that a business user can read.
You might think that is a pointless goal, but I have seen my fair share of business analysts debugging, and one (in Germany) even told me “I can follow whatever you write as long as it is not that object orientated ****”.
Just as an acid test, if the business user who wrote the specification could read your code, and understand it, and say “yes, that it just what I meant” that would be a good thing, would it not?
I am sure some people may say “I don’t want users being able to understand my code, if they understand it, then I am out of a job, that is why it is called CODE so no-one can understand it!”.
Hoots Mon! There is Re-Use, Loose, about this Hoose!
Figure 2.7 – da da da da da, da dada da dada etc
I keep plugging the SAP Press book by Igor Barbaric called “Design Patterns in Object Orientated ABAP” because that is what got me going into this area in the first place. He said that the first time he did a major application in an OO fashion he ended up with a whole bunch of components that could be re-used in subsequent projects.
A cynic would say you could do that with function modules, but I am trying to keep an open mind. Also, this is rather like the Moon Walking Bear – you don’t look for it unless you know it could be there. I have to say that even after a short while I am finding things that could be potentially split off into their own Z classes and then re-used elsewhere.
Since OO design, and unit testing in particular, forces you into the “separation of concerns” where different things are in their own classes, it soon becomes apparent that some classes are clearly specific to the application at hand whereas others are far more generic.
I never knew there was so much in it
Naturally if I can find an existing artifact to do want I want I can re-use it. However, one of the “problems” of SAP is that if you have a programming problem to solve, it is almost certain someone has been there and done that and got the t-shirt, and that there is a standard class / method / function module that already does what you want.
The problem is finding it.
There are more function modules in the standard SAP system than there are stars in the sky, and more recently a lot of classes / methods as well, though it will take time for them to catch up numerically due to the vast amount of time when OO things were not available in ABAP.
It is my horrible suspicion that hundreds if not thousands of ABAP programmers spend their day writing functions that are exact copies of things that already exists in the standard system, or creating user exits which achieve functionality which either exists in enhancement pack business functions which have not been activated in their system, or even for things that can be achieved by customising.
Even worse I think the same thing happens within SAP itself. When I have searched for standard function modules to achieve XYZ I usually get no results or five or six functions doing exactly the same thing.
When I was reading the book “Thinking in Java” http://en.wikipedia.org/wiki/Bruce_Eckel said the if you did not know the name of a method in the Java GUI “swing” framework you could guess what it was called because the naming conventions were so strongly followed. That does not seem to be case in SAP, if anything there is an utter lack of naming conventions in standard SAP repository objects.
Even the BAPI naming is all over the place, one of the four principals for BAPIS was supposed to be consistent naming, along with meaningful documentation, he he he.
You end up with names containing assorted naming conventions for creating a new business object of some sort.
With other functions, even if the function at hand is the sort of thing that is re-usable in many contexts, the SAP programmer names it after the project they are currently working on, so you get class names like JAPANESE_MICE_RACING_MATHS_FUNCTIONS which contain many useful mathematical functions which could be used in many applications which are unrelated to mice, or racing, or Japan.
I don’t like to call a function module with a name like that, as it would confuse the hell out of any casual reader of the code.
In the end very often we create our own function module or class, and then later on we might accidentally stumble on an identical one in standard SAP.
Going back to my example, in the spreadsheet was the EXCEL function MAX. I thought there must be one in standard SAP that does the same thing – as indeed there is – but I couldn’t find it for love nor money. A load of Google searching did not help, I searched for function modules and found BELGIAN_TRUFFLE_GROWERS_MAX_VALUE which was almost OK but had a few odd mandatory import parameters.
In the end I wrote my own method to do this, and then every so often I did another Google search using different parameters – the term “BUILT IN FUNCTION” did not help much for example – and in the end I discovered the standard function is NMAX.
You are probably thinking “oh my god, what a whinger and complainer he is, always talking about how hard things are to do in SAP”.
Well, I’ll tell you, I never wanted to be an ABAP programmer in the first place!
I... I wanted to be... A LUMBERJACK!
(piano vamp)
Leaping from tree to tree! As they float down the mighty rivers of British Columbia!
With my best girl by my side!The Larch!The Pine! The Giant Redwood tree! The Sequoia! The Little Whopping Rule Tree! We'd Sing! Sing! Sing! I’m a Lumberjack and I’m OK
As mentioned earlier, when it comes to logging I want to make re-use of CL_RECA_MESSAGE_LISTto handle showing error messages to the user and the application log in general. Great as this is, I am never happy, so naturally I sub classed this and added assorted things of my own.
For example, sometimes I just want to add a free text message and do not want to have to clutter up the code passing in the message number and type and what have you. This is like the ABAP statement MESSAGE ‘something’ TYPE ‘I’. So I redefine the ADD method and let it accept just a text string and the message type and then pass in suitable generic values for the message class and number.
This is just an example, and probably not a very good one, the point is every time I feel the lack of some sort of function to do with logging I will add it to be Z message handling class, and this is bound to be re-usable in future applications. OO people keep on and on about how you will end up with a library of re-usable components you can share with your colleagues – or the SCN in general – and maybe they are right. That is after all the purpose of the code exchange, with ABAP2XLS being the best example.
Local Hero
That logging functionality will clearly be a Z class as I can be virtually certain myself or someone else will make use of it in the future in another application. However generally, most of the code in a given application will be specific to the application itself.
So the question is – and this will be the subject of the next blog – do you do your prototype mainly as a Z class, or as a local class, and what are the drawbacks and advantages of both approaches?
Cheersy Cheers
Paul
Links to subsequent blogs:-
http://scn.sap.com/community/abap/blog/2013/10/03/abap-objects-is-the-way-forward--part-03
16/08/2013 - I have just been informed that I will be speaking at the Australian SAP user Group Conference in Sydney on the 11th/12th of September 2013, about Unit Testing in SAP.
P.S. Further to some earlier comments about software which promises "not one line of code" and how it can become more complicated than actual ABAP programs. Please find attached a nice simple graphical mapping that i found in PI, where assorted squares in fact contain little self-defined Java programs.
- SAP Managed Tags:
27 Comments
