The world’s largest corporations have dozens or even hundreds of SAP systems with various functions. Their availability for users is usually a priority, and the unavailability may have various consequences, including the suspension of key business processes in case of failure. One such corporations is Bombardier, which has built its own competent team of administrators and SAP solutions architects to manage and maintain its systems landscape. The majority of this team is located in Montreal, Canada.
IT departments with specialized teams of administrators, Basis specialists and ABAP developers often use external support. The reasons may be different and may result from the strategic objectives of the company, increased intensity of work, a large amount of simple, repetitive work that is relatively easy to outsource, or assignment of major tasks to an external company. Often, an invitation to cooperation involves the need to take advantage of the expertise in narrow, rarely used areas. Usually, it is uneconomic to maintain such competences in-house.
The cooperation between Bombardier and BCC (now All for One Poland) began in December 2010. The group of SAP Basis experts mutually agreed that the rational thing to do would be to conduct a technical audit of selected systems from the SAP landscape. This approach allowed the BCC consultants to obtain the necessary knowledge of the company’s systems. For the customer, it provided an independent picture of their systems, including a full set of recommendations regarding their maintenance and configuration.
The subsequent work contracted to BCC involved, among others, the participation in a large SAP upgrade project.
Technical audit of SAP systems
The design of an SAP system landscape begins with a thorough analysis of the company’s business needs, key processes and development plan. Typically, the entire process is adapted to the financial, operational and strategic goals, taking into account the current market situation. The systems are faced with increasingly greater challenges: analytical – often with large volumes of data, and reliability challenges – reducing the risk of their unavailability. It also requires a continuous optimization of the SAP systems configuration, the hardware and software platform, as well as a flexible scalability of the solutions, with particular emphasis on temporary, periodic or long-term increases in workload.
The purpose of the technical audit in Bombardier was to present an accurate and objective picture of the SAP systems. The audit was conducted based on the knowledge and experience of BCC experts.
In December 2010, Bombardier together with BCC (now All for One pPland) initiated the first project of the SAP ERP system analysis, taking into account high availability based on the solutions that use an IBM platform. A detailed analysis of the systems and preparation of recommendations took several days. During the analysis, the team used a proprietary script of BCC supporting the process of collecting data on the configuration of the hardware and software platform, EWA reports generated by the SAP Solution Manager and tools dedicated to the database.
At the same time, a direct analysis of the system was carried out, which was the basis for the preparation and presentation of the final report to the customer. The ultimate result of the work was a detailed document containing a series of recommendations to improve the system in the following categories: general configuration, security and performance.
The proposed activities included the redesigning of HA/DR solutions and transfer of SAP systems to the “private cloud" environment. The proposal was preceded by a detailed verification of the best solutions available on the market that met the customer’s expectations in terms of the Service Level Agreement and Operational Level Agreement (SLA/OLA) terms, as well as the technical efficiency of the architecture and tools.
That kind of approach we were looking
Having a reliable team with an appropriate set of skills is crucial for running 24/7 operations support for large critical IT systems at Bombardier. However, this is not only about keeping the lights on. We still require to implement changes (patches, upgrades, fixes) and support projects during their life cycle. The demand for our skill set fluctuates throughout the year, causing this to be one of our biggest challenges: how do we respond to increased demand without impacting our bottom line?
Having an external partner is one of our solutions. When required, we can outsource (out-task) some of the activities to our partner. This will of course generate cost but this cost is easy to justify by increased demand.
We ran an exercise with BCC where a consultant was actively working with our team handing over some activities or tasks from Montreal. This allowed us to be more efficient by leveraging the time zone difference and spending 16 hours instead of 8 on these activities. We managed not only to meet our deadlines but deliver ahead of schedule.
We paid particular attention to BCC’s consultants reaction to changes in the scope. We know what needs to be done at Bombardier, however due to the size of our organization, and a very demanding market, last minute changes are often common. This is a case of the “Business” driving IT and not the opposite way around. I must say that this is where BCC’s consultants either met or over-achieved our expectations. Their approach of “your problem is my problem” is what we were looking for.
Domenico Ferrara, Operations Manager, ERP/PLM/LPAR Management, Bombardier
Special attention was paid to the customer’s expectations related to the optimization of the system performance. The report included recommendations for data distribution at the level of the database environment and parameterization of the SAP environment, and detailed recommendations regarding specific tables and reports from the user and SAP namespaces. One of the recommendations regarding the correction of errors in the software helped reduce the amount of distributed readings per day by two TB. The report included recommendations allowing for savings in the region of 10-20% of the total workload at the I/O level.
Bombardier positively assessed both the effect and progress of the audit. As a result, the customer decided to continue the cooperation with BCC. In subsequent years, audits of SAP PI, SAP BW systems and even the rarely performed audit of the SAP Portal system were carried out. The document from the portal system analysis contained additional recommendations on the parameters of the SAP JVM and SAP NW Java runtime environment as well as key parameters for optimal use of resources by applications that run on the Java platform for SAP.
The information collected by a proprietary script of BCC (categorization):
– name, operating system version along with version of the components
– configuration of HA/DR solutions
– database parameters
– SAP system parameters
Upgrade of SAP systems
In June 2011, Bombardier invited BCC (now All for One Poland) to participate in a large-scale project of the SAP system landscape upgrade. BCC consultants were involved at each stage of work: from the preparation of the full upgrade procedure up to planning and direct support of the upgrade of production systems. The entire project was completed in September 2014.
The upgrade of SAP systems in a vast landscape is always a challenge for companies opting for such a step. The key is to carry out the procedure in the correct order so that upgraded versions of subsequent systems do not prevent further interoperation of the systems. The complexity of the project stems not only from its impact on business processes, but also from the need to involve people with extensive technical and modular knowledge in the project teams.
A proper coordination of the project and preparation of detailed procedures to be followed during subsequent phases provides a valuable expertise, which is the basis for a successful process of change and modernization.
Usually during such complex projects, there is a need to secure additional support to comply with the scheduled time window or acquire highly specialized, rare competences. The use of external consultants with experience in similar upgrade projects for other customers makes the entire project run smoother and helps to narrow down the project time frame.
The preparation for the upgrade of the SAP Solution Manager from version 7.01 to version 7.1 and reorganization of the System Landscape Directory (SLD) began in 2011.
SAP Solution Manager is a central system for support and management of systems in the customer’s landscape. In the upgrade procedure, the main task of the system is to create an appropriate vector of components required for the system transition from the source version to the target one. An appropriate version of the Solution Manager is required to carry out the procedure for creating the vector.
At Bombardier, it was decided that the system upgrade would be divided into two stages. BCC consultants were entrusted with the task of carrying out the conversion of SAP Solution Manager to Unicode and proposing a method of consolidation of SLD systems into one common system. SAP Basis administrators from the internal team were responsible for the execution of the system upgrade at a later date, with the support of BCC on demand. A current SAP Solution Manager system with up-to-date information about the connected systems is essential to carry out effective update operations related to the versions of systems and components.
The properly conducted version upgrade of the SAP Solution Manager enabled the upgrade of the other ERP systems to SAP ERP 6.0 EHP6 version. BCC’s main task was to prepare a thorough manual for the procedure, taking into account the time of individual phases, and to propose an upgrade scenario. The prepared procedure involved a detailed description of all the steps: preparing the system, performing the upgrade, and carrying out all technical steps ending the procedure. Such a detailed document enabled a smooth re-verification of the procedure using a copy of the production system, which was the basis for the accurate specification of time intervals required for the execution of subsequent phases, including the time of the system unavailability for users.
On the basis of the upgrade manual, Basis administrators of Bombardier upgraded the production system. BCC provided direct support and took over the operations in case of occurrence of previously unidentified problems.
At the same time, the upgrade of the SAP Portal production system was carried out, in which BCC consultants had earlier conducted a verification procedure and provided direct support.
The thorough planning helped complete the entire process within the time window specified by Bombardier and after the verification of the systems by designated key users, it was made available to the other users.
More efficient Basis support
At Bombardier we support very large infrastructure of different flavors of SAP systems. This is a very demanding task and requires an appropriate level of maturity from basis team members. On top of regular/standard support, we perform upgrades, maintenance, conversion projects, performance optimizations etc. However, at a certain point, we started to ask ourselves the question: do we do this right? Do we follow the best practices – both for SAP and the industry?
This is how we came to the conclusion that we need an external partner, someone with knowledge and experience to tell us whether we do things right. From all available options, we picked BCC (now All for One Poland) as a partner that can do that for us.
The system reviews performed gave us a set of recommendations on how to improve the system’s configuration/performance/scalability and at the same time was a proof for us and our management that we do things right here.
Letting BCC’s consultants do some of our maintenance (Support Packs install/upgrades and others) gave us more time to concentrate on other areas of support. The idea was to give to experts what they do best and concentrate on our area of expertise. This way of running Basis support is much more efficient.
What’s really important is the way BCC’s consultants integrated with our team. We call ourselves an extended team where consultants are treated as team members during their assignments. Having this high level of synergy between two groups helped us to deliver much more and much quicker than previously.
Marek Silski, SAP Basis Architect, Bombardier
During the several years of cooperation between the two companies, BCC consultants did not only work on specific tasks in projects, but also helped achieve the objectives related to the development of competences of the members of the Bombardier administrators’ team.
The possibility to use the potential of a proven partner and the guarantee that the work entrusted will be done in accordance with the specification, within the scheduled time and budget, without any concerns regarding quality is the essence of cooperation between organizations. For Bombardier, a no less important goal of the joint projects was to extend the competences of system administrators by leveraging the experience of BCC consultants gained working in a wide variety of IT/SAP projects.
The continuous technological progress and the development and changes in the organization and its environment quickly make knowledge obsolete. A continuous effort is needed to keep it up-to-date through regular verification, review and update. As a result of such cooperation between BCC and Bombardier, the benefits are always mutual. The corporation gained a methodology of operation in accordance with best practices and measurable benefits of having appropriate procedures in the implementation of complex projects in the field of SAP administration. BCC (now All for One Poland) expanded its experience, which will help maintain a high level of the consultants’ expertise and will be used in future projects.
In practice, the transfer of knowledge was accomplished through cooperation in creating procedures and during their mutual verification. A good opportunity was also the cooperation in solving technical and project-related problems.
Excellent professional qualifications and competences of the team of Bombardier administrators, flexibility and BCC’s individual approach to the customer’s needs enabled the successful completion of complex IT projects and maintaining satisfaction with the ongoing relationships.
Cooperation: Marek Biliński