Support Pipeline scenario
Introduction
In this article I am explaining actual upgrade scenarios proposed by me in year 2008 for the upgrade part of a larger optimization, roll-out and upgrade project of SAP ERP landscape (ECC5 to ECC6 Ehp3). In the end, final scenario with all consolidation options was chosen, and upgrade was finished with a minum development downtime (development freeze) of less than one week (instead of two months or more). Whole upgrade lasted almost six months, including all preparations, sandbox installations and tests, official development system upgrade was started in February 2009, and Go Live was completely successfully finished by the end of last weekend in May, 2009.
Basic Scenario - SCENARIO 1
Official (ASAP) Upgrade Roadmap document available on Marketplace (service.sap.com/upgrade) and in Solution Manager, Upgrade Master guide, SDN and other official SAP sources offer many upgrade scenarios - from which the most simple one is a scenario which I will call SCENARIO 1 ("by the book"), while there are more sophisticated ones (like CBU, switch scenarios, etc.) which are not discussed here. I will not discuss here all the technical details of an upgrade process, only those which affect the decissions about project timeline, development freeze duration and possible hardware purchase or h/w leasing. What is here discussed is the possibility of variants of SCENARIO 1 by using parallel Transport Management routes (or so called Support Pipelines, Transport Lines, or just Lines, [2]) with some degree of additional hardware usage or consolidation in order to make development freeze as small as possible, and for other support purposes during the upgrade project.
In almost any standard upgrade procedure scenario we must have a test SANDBOX system (as described in references). By a standard upgrade SCENARIO 1, a sandbox system as a homogeneous copy of the production system is needed for:
- Initial technical upgrade (from ECC5 to ECC6): prepare, start of upgrade, SPAU/SPDD, modifications, support packs, etc.
- Functional tests needed to improve procedures, correct problems, create and plan all necessary activites - this can be iterated until achiving satisfying results
- Generating upgrade script (a detailed set of upgrade activites, participants and dependencies e.g. as given in )
In business environments without so strict validation practices it is possible to use this system as a future upgraded D (development) system (as proposed, if I have understood well as I wasn't present on his presentation/workshop), or it could be used as a place for formal functional tests (as proposed in references, upgrade/business case) and a future Q (Test and QA) system - thow they also propose creating a new (P) production system. Company needs three-tired (D-Q-P) lines in the landscape for validation purposes. In all cases, without taking into account which servers are physically used for ECC6, three-tiered systems (D-Q-P) and any upgrade scenario need this series of events (as given in references, [4]) - the UPGRADE LINE:
Fig. 1 - upgrade steps in the basic scenario - scenario 1
Parallel Hardware Infrastructure - SCENARIO 1A
In a variant of such a scenario (one of many which involve system copies or original systems and landscapes), whether we could use a parallel h/w infrastructure for ECC6 - let us call it SCENARIO 1A:
Fig. 2 - „parallel" h/w landscape
Arrows in fig. 2 show direction of system copy, D5-Q5-P5 is the existing ECC5 line (landscape) of SAP systems (D5 stands for dev elopment, Q5 for test system and P5 for production on ECC5, while D6 is development on ECC6, and so on) which is both the maintenance and the upgrade line later on, and D'6-Q'6-P'6 is the ECC6 sandbox line (though, only one sandbox system is normally sufficient for "rehearsal" purposes and for upgrade script generation) made by heterogeneous system copy of each system established as a separate line in TMS on new set of servers. On the other hand, as in SCENARIO 1, there is only D5-Q5-P5 line. This scenario does not differ much from basic scenario (development freeze starts as soon as the upgrade of development system starts).
At one point, the following series of events (*) is inevitable in any of the previous scenarios (SCENARIO 1 and SCENARIO 1A) - the place of the beginning of development freeze is same:
- 1. Development freeze
- 2. 😧 Upgrade D5 to D6, corrections, formal unit testing, possible repeating step 2.
- 3. Q: Transports generated by upgrade in step 2 from D5 upgrade. Including corrections (somewhat faster than complete upgrade of Q5 system instead of using transports of changes), formal integration tests, possible repeating step (2. and) 3. If successful, we have Q6 as result.
- 4. P: optional testing upgrade on renewed sandbox and possible repeating steps 2. or 3.
- 5. Transports from D5/Q5 upgrade, Final upgrade of P5 to P6 and GoLive
The thing here is that integration tests should take at least few weeks (with all the performance impact on new Q6 system), without the days needed for the pure technical upgrade itself. In all scenarios here mentioned the D'6-Q'6-P'6 line plays actaully the role of the sandbox system expanded to whole landscape, which might give some downtime decrease in the development freeze phaze - but whith extra hardware and effort which is not justified (at least I seriously doubt so, if h/w not borrowed I am certain). This downtime is not important if it is not influencing Development Freeze, as given and explained in the next scenario - SCENARIO 1B - and this is the only important difference between these two scenarios. Hardware can be even more consolidated as in the scenario to be proposed here bellow:
Parallel Hardware Infrastructure - SCENARIO 1B
An alternative solution (with variants) is given in where an additional support line made of heterogeneous system copies of D and Q (or renewed copies from the SCENARIO 1A) is used:
Fig. 3a - Temporary Support (pipe)line scenarios (as in [2])
In that manner, production system stays intact until the previously described step 5 - let us call this SCENARIO 1B. Formal validation (unit and integration) tests can be done without development freeze (**). In this case we have hardware performance-ready with optionally minimal additional hardware needed for D and Q copies instead of whole parallel h/w infrastructure as in SCENARIO 1A. Therefore, I propose for SCENARIO 1B.
SCENARIO 1B could be OPTIONALLY more comfortably realized with addtional hardware (but this is not a mandatory requirement) which could be leased/borrowed (or bought) during the initial upgrade phase as project support resources.
In this case it is even possible to make several levels of h/w consolidations with existing hardware infrastructure and other business requirements (and our future SAP systems if bought, depending on business needs, e.g. additional D/Q Solution Manager system, BI requirements, additional archiving solution with SAP Content Server, etc) - our existing SAP hardware was recently upgraded with additional RAM.So BI test system BWQ needs just one physical node, other node could be used in the support line of servers (and might even be temporarily left on only one cluster node with functionally intact). Sandbox system can be established on one of our integrity servers (also upgraded with RAM, like SAP5 in fig. 3c). MSCS switchover tests are necessary only on new (ECC6) ERQ servers.
If possible, borrowing servers or some RAM from our hardware vendor (HP, Belgrade) or his partners could significantly simplify this scenario. This would save a lot of time, effort and minimize the risk about these activities. Nevertheless, it is possible to make sufficient hardware consoloditions by replacing existing RAM modules from our existing servers which are less loaded in order to improve performance on critical servers (e.g. Q5 systems for testing, D5 for upgrade to D6), with insignificant risk of performance issues on old development and sandbox systems. But these actions have to be planned on time and carried out with devoted attention.
Possible h/w Consolidations with SCENARIO 1B
The SCENARIO 1B could be compared with SCENARIO 1A with a diagram on Fig. 3b similiar to Fig. 3a:
Fig. 3b - Upgrade and Temporary Support (Production Maintenance) lines
But here D5 and Q5 are copied into D'5 and Q'5 (and so is P5 into SANDBOX system), which together with P5 form the new Temporary Support (Production Maintenance) line: D'5-Q'5-P5 - which is used for urgent changes in the system (so called „fast-tracks", minimized in number as much as possible). These additional changes need to be taken care of in the post-GoLive phaze or during upgrade. Physically same servers D5 and Q5 are then upgraded to D6 and Q6 by the upgrade script generated on SANDBOX system. Development freeze is smaller than in SCENARIO 1A.
The bottom line here is that it is possible not to buy NEW HARDWARE for project support (no servers at least) with only potential performance issues with new ERD (and sandbox) consolidated on one of our existing servers which could be handled successfully. In the Fig. 5 bellow is a diagram of SAP System Landscape in our company with systems, transport lines and physical server names. The idea is to „borrow" two servers from ERQ and BWQ servers, using existing clusters and leaving temporary support line out of the cluster (which should not be an issue). In the fig. 3c bellow are is given a proposal of hardware consolidations using existing hardware landscape and only one available additional server (SAP5, rx4640 with 1 CPU and 4G RAM). Physical servers are greyed-out, clustered nodes are grouped (MSCS, Microsoft Cluster Services):
Fig 3c - Consolidation h/w maps |
Better consolidation could be achieved by using MCOD for D'5 and Q'5 systems on one server (two systems with one database). This also enables using existing snapshot EVA Storage functionality for creating instant copies of database VLUNs while saving space (which is very important) and Oracle transportable tablespaces for binding D5 and Q5 databases into one database. But in this scenario 8G RAM is mandatory for optimal performance. Using virtualization technology would enable more options and more optimal consolidation in all scenarios.
All urgent changes would be treated as fast tracks (urgent corrections as covered by standard change management procedures in the existing "normal" landscape). After the production upgrad and Go Live (but before any new change requests in the upgraded landscape), these would be implemented. There are two possibilites for their handling:
- each development in D'5 should be repeated in D6 (transports are only possible to D5, e.g. before upgrade to D6), and transported to Q6 and P6 later
- after the upgrade, D'5 (or a system copy of D'5) could be upgraded, and all fast tracks could be transported to D6 after that
Upgrade steps in SCENARIO 1B in a skecth could be compared to steps (*) described earlier, which now need no Development Freeze in step 1, but only during the final step 5 - upgrade of production:
- 1-3. Same as steps (*) 2-4.
- 4. Development Freeze (**)
- 5. Transports from D5/Q5 upgrade, Final upgrade of P5 to P6 and GoLive
- 6. Copying fast-tracks back upgraded D and Q systems.
Hardware Requirements and Sizing Estimations Concerning ECC6 Upgrade
In all these scenarios I find that pure technical upgrade of ECC systems can be estimated as follows (by Note 901070, SAP benchmark information, and Oracle documents given all in references) - by upgrading from ECC5 to ECC6 (and from Oracle 9.2 to 10.2) we have only a slight increase in need for the h/w resources:
System Component | CPU | RAM | DISK |
SAP INSTANCE | 0-5% | 5-10% | insignificant |
ORACLE DB | 0-5% / insignificant | 0-5% / insignificant | 5% |
Checking necessary disk space and datafile consumption by the guide is necessary. Some indications in references show that it is possible to have up to 25% DB growth during upgrade, but this is not an issue for ERP system as it has already more than 50% free space in the database files, and sandbox system didn't show such behaviour.
The only important issue about all these scenarios is the currently available Storage space for Temporary Support hardware (in any scenario, but especially for SCENARIO 1A) and during project implementation. With some additional effort and incoming resources for backup disks this might be solved, too, but this needs to be carefully estimated. But in any case, Storage space has to be well planned for the future.
Hardware Sizing and Other Project Requirements - Quicksizing
On the other hand, system sizing and Storage space is also significantly influenced by Sizing produced by the Rollout & Optimizitaion activities and appropriate parts of the project. This has to be discussed separately, BC needs input from each team about estimated number of users and their types, significant changes of system usage (numbers of document generated, etc) and other information needed for the quicksizing utility which should be part of the regular project business blueprint activities. These are planned to be until 01.09.2008, but we need this info ASAP in order to inform out h/w vendor (HP) on time and to place orders as needed. Timelines are still not cleared. I expect that it would be mostly about additional RAM, CPUs and Storage, and optionally about borrowing servers. HP also offered us help from his SAP experts at least on this topic, which could be very useful for the whole project.
BI hardware landscape and system sizing for it's upgrade is unknown in terms of project scope (to be defined in the project scope). For Solution Manager, company has requirements for only one upgraded system (as seen from workshops, SolMan landscape with development and production is not in the scope of this project). Using Web GUI for BI and Java instances for BI should also be considered in system sizing.
Borrowing servers as temporary resources and support during the project is also a good option which influences mainly the project timeline, but are not important after GoLive if no User Requirements emerge during BBP (e.g. BI systems and Solution Manager usage).
References:
[1] Road Map V3.2 (available in Solution Manager, or Upgrade media)
[2] http://www.thespot4sap.com/upgrade_guide_v2.pdf
[3] http://www.sap-press.de/download/dateien/1324/sappress_netweaver_application_server.pdf
[4] ADM326 Upgrade Course
[5] http://www.sapdb.info/sap-ecc6-upgrade/
[6] Upgrade Master (service.sap.com/instguides)
[7] http://service.sap.com/benchmark
http://www.sap.com/solutions/benchmark/index.epx
http://www.sap.com/solutions/benchmark/sd3tier.epx?num=200
http://www.sap.com/solutions/benchmark/sd2tier.epx
[9] Keter case study: Keter_ECC6_Upgrade_Presentation_170107.ppt
[10] http://service.sap.com/quicksize
