Best Practices for Migrating SAP Systems to Oracle Infrastructure--Part 6: SAP Environment Migration--Post-Database-Migration Tasks

Version 2

    by Jan Brosowski, Victor Galis, Gia-Khanh Nguyen, and Pierre Reynes

     

    This article, along with related articles, can be used to prepare a highly available SAP implementation after migrating an Oracle Database instance, regardless of how the database was duplicated.

     

    Introduction

     

    This article is Part 6 of a six-part series that provides best practices and recommendations for migrating a complete SAP system to an Oracle platform (in this example, to Oracle SuperCluster M7). A team of Oracle engineers and SAP experts worked together to test and tune different migration methods, compiling the step-by-step procedures and guidance given in these articles.

     

    The articles in this "Best Practices for Migrating SAP Systems to Oracle Infrastructure" series are located here:

    OOS_Logo_small125.png
    in collaboration with
    OSC_Logo_small125.png
    Oracle Optimized Solutions provide tested and proven best practices for how to run software products on Oracle systems. Learn more.

     

     

    To develop the entire series of articles (documenting the three different migration methods), engineers used three different database SAP system ID (SID) values for testing:

     

    • PR2: The SID used to test the Oracle Recovery Manager (Oracle RMAN) DUPLICATE method (that migrates  an active source database)
    • PR3: The SID used to test the Transportable Tablespaces method
    • PR4: The SID used to test the Oracle RMAN cross-platform BACKUP and RESTORE method

     

    In all cases, the SID for the source database was PR1.

     

    Regardless of the method chosen to duplicate the Oracle Database instance, similar steps need to be taken afterwards. So this article is applicable to all three migration processes. In this and related articles, the example commands use SAP SID PR2 as the destination database SID.

     

    Post-Database-Migration Steps

     

    After the source database has been migrated to Oracle SuperCluster, it is recommended to first validate the resulting destination database. The Oracle RMAN command VALIDATE DATABASE should be executed to verify the database's physical structure and identify any corrupted data blocks. Use the dynamic performance views V$CONTROLFILE, V$DATAFILE, and V$LOGFILE to verify that the new file locations are in Oracle Automatic Storage Management (Oracle ASM).

     

    After the database is successfully migrated and validated, there are several major steps to get the SAP environment up and running in a highly available (HA) configuration. This article gives an overview of the process, and describes initial configuration and final validation steps. Other articles referenced here provide extended step-by-step instructions for installing software components and configuring the HA deployment.

     

    The following four main tasks must be performed to construct an HA SAP implementation on Oracle SuperCluster after the database migration:

     

    1. Task 1: Register the migrated database with Oracle Clusterware.

     

    See the section later in this article called "Task 1: Integrating the Migrated Oracle Database in Oracle Clusterware."

     

    2. Task 2: Convert from a single-instance Oracle Database to Oracle Real Application Clusters (Oracle RAC).

     

    A database migration usually results in the creation of a single-instance Oracle Database on the new platform. To achieve a highly available database implementation, it is necessary to convert the single-instance deployment to an Oracle RAC environment. A separate article, "Converting Single-Instance Oracle Databases for SAP to Oracle RAC," shows step-by-step instructions for the conversion process.

     

    3. Task 3: Deploy SAP in an HA configuration on Oracle SuperCluster.

     

    To achieve high availability, it is necessary to put the critical SAP components under control of Oracle Solaris Cluster in a cluster of two or more zones that are created on separate physical nodes. Oracle Solaris Cluster can then orchestrate failover or disaster recovery strategies while managing infrastructure resources such as the database, network connectivity, and shared storage. The following separate three-part article series ("Setting up Highly Available SAP Systems on Oracle SuperCluster") documents the installation and configuration procedures:

     

     

    • Task 3, Part 2: Deploying and Configuring Oracle Solaris Cluster for SAP on Oracle SuperCluster

      This article describes the steps for preparing and installing the Oracle Solaris Cluster software, which is then used to configure the two-node cluster. Migrating Oracle Solaris Zones from the source platform to the destination Oracle SuperCluster platform creates virtual servers on the nodes. Using the zone clustering capabilities of Oracle Solaris Cluster, the virtual servers are then clustered. The procedures in this article construct two zone clusters: one for the SAP ABAP Central Services instance (ASCS) and Enqueue Replication Server instance (ERS), and one for the SAP Primary Application Server (PAS) and other mission-critical application services (Figure 1).

     

     

    4. Task 4: Validate the SAP HA configuration

     

    See the section later in this article called "Task 4: Starting SAP and Conducting Testing," that documents the final step of validating the SAP HA deployment.

     

    f1.png

    Figure 1. Example HA configuration for SAP deployment.

     

    Task 1: Integrating the Migrated Oracle Database in Oracle Clusterware

     

    After the database has been migrated and validated on the destination platform, the following steps register the database server in Oracle Clusterware so that it can be started and stopped via the server control utility (srvctl).

     

    Step 1. Add single-instance database configuration information to Oracle Clusterware.

     

    Database migration usually results in the creation of a single-instance Oracle Database on the new system. Listing 1 shows how a single-instance database is added to Oracle Clusterware using srvctl.

     

    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ srvctl add database -d PR2 -oraclehome /oracle/PR2/121 \
    -dbtype single -node `hostname` -policy AUTOMATIC -diskgroup DATAC1,RECOC1 \
    -spfile +DATAC1/PR2/spfilePR2

    Listing 1: Adding database configuration information using srvctl.

     

    Step 2. Check the database status and registration.

    Listing 2 shows that the database instance is running on this node and that Oracle Clusterware is not yet aware of the status of the database. A restart is necessary.

     

    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ srvctl status database -d PR2
    The instance PR2 is not running on node sapm7zdbadm1c1
     
    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ ps -eaf |grep pmon
    ...
      oracle 49655 10695   0   May 20 ?           0:44 ora_pmon_PR2
    ...

    Listing 2: Checking the database status and registration using srvctl.

     

    Step 3. Shut down the database instance.

     

    Shut down the database instance, as shown in Listing 3.

     

    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ sqlplus / as sysdba
    SQL*Plus: Release 12.1.0.2.0 Production on Thu May 26 16:48:05 2016
    Copyright (c) 1982, 2014, Oracle.  All rights reserved.
     
    Connected to:
    Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
    With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
    and Real Application Testing options
     
    SQL> shutdown immediate
    Database dismounted.
    ORACLE instance shut down.
    SQL> exit
    Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
    With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
    and Real Application Testing options

    Listing 3: Shutting down the database instance.

     

    Step 4. Restart the instance and recheck status.

     

    Restart the instance using srvctl and recheck its status:

     

    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ srvctl start database -d PR2
    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ ps -eaf |grep pmon
       
      oracle 52741 10695   0 16:49:54 ?           0:00 ora_pmon_PR2
    ...
    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ srvctl status database -d PR2
    Instance PR2 is running on node sapm7zdbadm1c1

    Listing 4: Restarting the database instance and checking its status.

     

    Step 5. Verify the instance configuration.

     

    Verify the configuration of the database instance:

     

    oracle@sapm7zdbadm1c1:/oracle/PR2/121$ srvctl config database -d PR2
    Database unique name: PR2
    Database name:
    Oracle home: /oracle/PR2/121
    Oracle user: oracle
    Spfile: +DATAC1/PR2/spfilePR2
    Password file:
    Domain:
    Start options: open
    Stop options: immediate
    Database role: PRIMARY
    Management policy: AUTOMATIC
    Server pools:
    Disk Groups: DATAC1,RECOC1
    Mount point paths:
    Services:
    Type: SINGLE
    OSDBA group: dba
    OSOPER group: oper
    Database instance: PR2
    Configured nodes: sapm7zdbadm1c1
    Database is administrator managed
    oracle@sapm7zdbadm1c1:/oracle/PR2/121$

    Listing 5: Checking the database instance's configuration.

     

    Tasks 2 and 3: Next Steps

     

    To achieve highly available database services, it's first necessary to convert a single-instance deployment to an Oracle RAC environment (see the article "Converting Single-Instance Oracle Databases for SAP to Oracle RAC" for the conversion process). Before installing and configuring Oracle Solaris Cluster and the SAP software components, you must configure the clustered virtual environments on Oracle SuperCluster appropriately. See the article series "Setting up Highly Available SAP Systems on Oracle SuperCluster," and perform the steps in "Part 1: Configuring Virtualization on Oracle SuperCluster," "Part 2: Deploying and Configuring Oracle Solaris Cluster for SAP on Oracle SuperCluster," and "Part 3: Installing Highly Available SAP with Oracle Solaris Cluster on Oracle SuperCluster." After these procedures are completed, you can complete Task 4 to validate that the configuration is properly set up to support highly available SAP application services.

     

    Task 4: Starting SAP and Conducting Testing

     

     

    SAP requires the use of secure file systems with Oracle Database 12c. Check that the correct information was generated in DEFAULT.PFL:

     

    rsec/ssfs_datapath = $(DIR_GLOBAL)$(DIR_SEP)security$(DIR_SEP)rsecssfs$(DIR_SEP)data
    rsec/ssfs_keypath = $(DIR_GLOBAL)$(DIR_SEP)security$(DIR_SEP)rsecssfs$(DIR_SEP)key

    Listing 6: Confirming the security settings for Oracle Database file systems.

     

    Basic testing and troubleshooting of a deployment is recommended to validate HA capabilities and service failover. At a minimum, perform the following checks as a test of HA functionality:

     

    • Switch over every Oracle Solaris Cluster resource group and observe the proper startup of services on the second node:

      clrg switch -n em7pr1-haapps-02 pas-rg

     

    • Switch over the SAP ASCS instance and observe ERS instance relocation:

      clrg switch -n em7pr1-ascs-02 ascs-rg

     

    • Unmonitor and monitor resources, and shut them down and restart them manually:

      clrs unmonitor d01-rs

     

    • Shut down one zone in each zone cluster and observe resource relocation:

      clzc halt -n sapm7adm-haapp-0101 pr1-haapps-zc

     

    • Unplug a network cable (or otherwise simulate a network failure) and observe resource group relocation.

     

    • For databases that have been migrated to Oracle RAC, shut down an Oracle Database instance and observe database service relocation, as well as the impact on SAP application users:

      srvctl stop instance -d PR2 -i PR2001

     

    • Reboot physical domains (PDOMs) and confirm that all SAP services come back up and that all configurations are properly stored. Verify that file systems are mounted according to the entries in the /etc/vfstab file.

     

    During testing, be sure to check these log files in Oracle Solaris global zones for more information:

     

    • /var/cluster/logs, including eventlog and commandlog
    • /var/adm/messages

     

    Additional Environment-Specific Post-Migration Steps

     

    Once the migration tasks detailed in the other articles are successfully executed, you must validate that the SAP system is up and running. Obviously, an SAP migration also includes the application server layer and, depending on your specific SAP implementation, the SAP application itself. These tasks are highly dependent on the customization and usage of the SAP system and, therefore, cannot be entirely covered in an article that focuses mainly on database migration methods for SAP.

     

    Within your SAP migration project team, SAP Basis administrators will know about the dependencies of specific systems. The following is a list of typical post-migration activities that need to be taken into consideration:

     

    • Checking the RFI destinations used by and offered by the SAP system. Due to the new configuration of the system, RFI destinations should be verified. This includes both the RFI destination that the SAP system uses in other systems, as well as the RFI destinations that the SAP systems offer to others. This is necessary particularly when the number of application servers or their IP addresses or hostnames has been changed during the migration (check transactions SM59/SM58).
    • Checking the logon group definitions. Logon groups are useful particularly with larger installations using multiple application servers. Because a migration to new hardware can change both the number and the addresses of the application servers, the configuration should be verified (transaction SMLG).
    • A common activity after a system copy is the renewal of the license key, because the SAP license key includes some installation-related information. Because this information changes during the migration, the license key must be renewed. This is an SAP-specific activity that is described on the SAP Support Portal.
    • Other SAP customizations closely related to the underlying hardware must be reviewed. For example, a very common hardware-related customization concerns printers and spooler services.
    • SAP generates platform-dependent programs (called ABAP loads) from all its ABAP programs, which are generated during runtime and stored in database tables. These are generated in the target system when they are first used, which can result in reduced system performance. To avoid this, use the transaction SGEN to generate the missing loads.
    • Finally, the SAP system must be reintegrated with environment-specific system tools, such as backup and recovery procedures. This is important because the database is moved to Oracle ASM on Oracle Exadata Storage Servers during a migration, so backup and restore procedures must be adapted accordingly.

     

    While the list above captures many potential environment-specific post-migration activities, each customer's list of required activities will vary. The steps for each specific environment are typically outlined before the migration is performed and must then be validated during migration testing.

     

    Final Thoughts

     

    The complete process of migrating an SAP environment to an Oracle engineered system infrastructure can be a challenging undertaking. An Oracle engineered system (such as the Oracle SuperCluster M7 platform used in this proof-of-concept testing) offers significant advantages for performance, scalability, security, and high availability, and it can simplify operations. In addition, Oracle SuperCluster uses InfiniBand connections, so moving large amounts of data is more efficient than with Ethernet networks. This series of articles provides best practices, concrete procedures, and guidance that can help to simplify the complete SAP migration process.

     

    The first step, of course, is to determine an optimal database migration strategy based on migration project requirements, which is the subject of the first article in this six-part series. The next three articles in the series give in-depth procedures for different database migration methods: Oracle RMAN DUPLICATE, Transportable Tablespaces (TTS), and Oracle RMAN Cross-Platform BACKUP and RESTORE. After performing the database migration, it's necessary to convert the single-instance database to Oracle RAC, as described in the separate article ("Converting Single-Instance Oracle Databases for SAP to Oracle RAC").

     

    Lastly, to enable highly available SAP application services, it is necessary to put mission-critical SAP components under control of Oracle Solaris Cluster. Oracle Solaris Cluster can then monitor the infrastructure components in the engineered system, and coordinate failover. A three-part article series ("Setting Up Highly Available SAP Systems on Oracle SuperCluster") gives the step-by-step instructions for installing Oracle Solaris Cluster and configuring the environment to provide a highly available SAP production environment on Oracle SuperCluster.

     

    See Also

     

    Refer to these resources for more information:

     

    Online Resources

     

     

    Papers

     

     

    Documentation

     

     

    About the Authors

     

    Jan Brosowski is a principal sales consultant for Oracle Systems in Europe North. Located in Walldorf, Germany, he is responsible for developing customer-specific architectures and operating models for both SAP and Hyperion systems, accompanying the projects from the requirements specification process to going live. Brosowski holds a Master of Business and Engineering degree and has been working for over 15 years with SAP systems in different roles.

     

    Victor Galis is a master sales consultant, part of the global Oracle Solution Center organization. He supports customers and sales teams architecting SAP environments based on Oracle hardware and technology. He works with SAP Basis and DBA teams, systems and storage administrators, as well as business owners and executives. His role is to understand current environments, business requirements, and pain points as well as future growth and map them to SAP landscapes that meet both performance and high availability expectations. He has been involved with many SAP on Oracle SuperCluster customer environments as an architect and has provided deployment and go-live assistance. Galis is an SAP-certified consultant and Oracle Database administrator.

     

    Gia-Khanh Nguyen is an architect of the Oracle Solaris Cluster product. He contributes to product requirement and design specifications for the support of enterprise solutions on Oracle systems, and develops demonstrations of key features.

     

    Pierre Reynes is a solution manager for Oracle Optimized Solution for SAP and Oracle Optimized Solution for PeopleSoft. He is responsible for driving the strategy and efforts to help raise customer and market awareness for Oracle Optimized Solutions in these areas. Reynes has over 25 years of experience in the computer and network industries.