Journey to setup best possible Continuous Integrat...

former_member205561 · ‎11-27-2015

Background

After working for several years in the nowadays so called "DevOps" area, developing test automation tools, simulation frameworks, automated performance & stability tests and rampup and maintaining quite complex build, test and delivery pipelines (CI's/CD's) for both Java and .NET stack projects i joined this year SAP. Into an purely ABAP based project.

Idea

Setup CI. Simple. Or ?

Beside normal CI requirements we have additional test activities to do before release

Compatibility to all versions between 7.02 and 7.5x, S4H*
Performance KPI's, especially from customers with huge (x hundred's Mio contracts like Vodafone)
Test with specific customizations (using one config as blueprint for whole industry sector)

so, first sketch (translated Dev/Integration/Release branch model to ABAP development landscape)

ATC

So easy to start, but not all developers consistently run check and remove - at least Prio1&2 - before triggering transport. For *some* machines (which are connected ) it is possible to use the ATC gadget on the Bridge Dashboard.

Follow this guide to setup HTTP IF for ATC Checks.

AUnit

Same. Failing Unit tests do not block any transport. TDD? Yes, most heard about it, but really doing in daily life ? Good practice is to plan as task immediately for each Item from backlog and add to teams DoD a minimum of code coverage for newly created code.

We found it very effective to setup at least nightly runs of our unit tests, each team got an own variant, an own Jenkins Job for the trend charts and results(errors) discussed in each teams daily standup.

Step-By-Step Guide Setup Jenkins Instance wit AUnit Test Jobs

Transport Events

Yeah! After I first heard about it I immediately thought that is the solution to slackers which ignore both ATC and AUnit. But setup is a bit more complicated as thought. First of all - check version of SUT. Write DLM ticket to update to latest version (as of today - 11/2015 - this is 2.12). Use Transport Requests with latest version of SUT Core as reference within the DLM ticket.

Note: It will not work for 7.02.

Now you can configure your systems to run both ATC & AUnit triggered by Import/Export Events (which sends EMail to the author of the change).

ITF - Instant Test Framework

During research I found some references to the ITF, which was developed within ByD landscape and never ported to NW.

The idea is close to CI - first run a "trial" transport into a shadow system, and only if all tests passed there activate the transport for next stage.

See some more ITF details.

eCATT with START

In our project we have SAPUI & WebDynPro, which means for E2E tests we have to use eCATTs (or manual testing). eCATTs are quite expensive (around 2 days per TC) and generate a heavy maintenance load (test scripts need adaptation, test data might need to be changed, errors itself are hard to find cause for some types of crashes there is no way to get a detailed trace). All these reasons have led to running them as less as possible. And of course no test trend chart available too.

Best possible solution: Trigger eCATT test plans (which so far must be configured in ECA system cause START cannot run test plans from GTP) via START command line from Jenkins. Meanwhile START team extended functionality so that results can be retrieved in JUnit format and displayed quite nicely (but without possibility to directly jump to detailed traces).

Code Coverage Trends

Within one of the latest versions of SUT a new feature was introduced to generate Code Coverage Trend Charts.

And again, not availble for NW 7.02.

ABAP code duplication check (Simian, conQAT, …)

http://scn.sap.com/community/abap/blog/2015/02/17/static-abap-code-analysis-using-conqat-and-of-cour...

I tried to configure conQAT in eclipse but gave up after one day without the expected results.