23 Patching Oracle RAC

Deployment Phases

The following describes the different phases involved in this Deployment Procedure:

Table 23-2 Deployment Phases for Patching Oracle RAC in Online Mode

Phase	Description
	Software Updates Allows you to select the software updates (patches and patch sets) you want to apply, apply SQL script, upgrade OPatch, black out associated targets, clean up backup files, and provide advanced options for OPatch. Note: You must not combine patches and patch sets in a single selection set.
	Target List Allows you to select the database targets you want to patch.
	Library Step Properties Allows you to specify values for steps that you want to edit at runtime.
	Oracle Configuration Manager Allows you to configure Oracle Configuration Manager.
	Credentials Allows you to specify Oracle home credentials required to patch RAC Oracle homes, and host credentials to access the hosts where the Oracle RAC instances are present.
	Schedule Allows you to schedule the Deployment Procedure to run immediately or later.
	Review Allows you to review the details you have provided for patching the selected Oracle RAC instances.

Prerequisites

Before running the Deployment Procedure, meet the following prerequisites:

Prerequisites for Designers

• Ensure that you meet the prerequisites described in the following sections:

  • Mandatory Infrastructure Requirements for Provisioning and Patching

  • Optional Infrastructure Requirements

  • Setting Up Infrastructure for Online Patching

• Ensure that operating system users such as oracle and crsuser are available on all the nodes of the cluster. These users must be a part of the relevant operating system groups such as dba or oinstall.

• Ensure that you run the patchability report to understand whether the Oracle Database you want to patch is suitable for this patching operation. In the report, if the Oracle Database you want to patch appears as a target that cannot be patched, then fix the target configuration issues to make it a target that can be patched. For more information about the patchability report, and to understand how you can diagnose the issues with an Oracle Database target and take corrective actions, see Appendix H, "About Patching Automation Reports".

• Ensure that you run the Deployment Procedure in the Analyze mode.

Prerequisites for Operators

• Ensure that you use an operating system user that has the privileges to run the Deployment Procedure, and that can switch to root user and run all commands on the target hosts. For example, commands such as mkdir, ls, and so on.

  If you do not have the privileges to do so, that is, if you are using a locked account, then request your administrator (a designer) to either customize the Deployment Procedure to run it as another user or ignore the steps that require special privileges.

  For example, user account A might have the root privileges, but you might use user account B to run the Deployment Procedure. In this case, you can switch from user account B to A by customizing the Deployment Procedure.

  For information about customization, see Chapter 30, "Customizing Steps to Run as Another User".

• Ensure that you are the owner of the Oracle home of the targets being patched. If you are not the owner or if you do not have direct access to those targets, then customize the Deployment Procedure to run as another user.

  For information about customization, see Chapter 30, "Customizing Steps to Run as Another User".

Patching Procedure

To patch Oracle Real Application Clusters, follow these steps:

1. In Grid Control, click the Deployments tab.

2. On the Deployments page, in the Patching section, click Patching through Deployment Procedures.

3. On the Deployment Procedure Manager page, in the Procedures subtab, from the table, select Patch Oracle RAC Database - Rolling. Then click Schedule Deployment.

   Enterprise Manager Grid Control displays the Software Updates page of the Deployment Procedure.

4. On the Software Updates page, do the following:

   1. In the Select Stage Location section, retain the default selection, that is, Enter Stage Location that has the value %emd_root%/EMStage.

      This default value refers to a location on the target host where the Oracle RAC to be patched is running. The software updates are downloaded from My Oracle Support to Oracle Software Library, and then staged in this location on the target hosts.

      
      

      You can modify the location if you want to, and click Save. If you save multiple custom locations, then the Select Stage Location option gets enabled and allows you to select any of the saved custom locations the next time you visit the same page.

      Instead of %emd_root%, you can also specify %oracle_home% to indicate the current home location being patched.

   2. In the RAC Database Updates section, click Add to search and select software updates from My Oracle Support. Alternatively, if you have a text file that contains details about the patches, then click Upload From File to select and upload that text file.

      On clicking Add, Enterprise Manager Grid Control displays the Search and Select Software Updates page. On this page, select Search My Oracle Support, specify the details of the patches you want to search, and click Go.

      Enterprise Manager Grid Control connects to My Oracle Support, searches the patches, and displays details about them on the Search and Select Software Updates page. Select the ones you want to apply using the Deployment Procedure, and click Select.

      Note:

      Selection of multiple patches is supported.

      On clicking Upload From File, Enterprise Manager Grid Control displays the Upload File page. On this page, select Upload from Local Machine or Upload from Software Library depending on where the text file is available, click Browse to search and select that file, and then click Upload.

   3. (Optional) In the Apply SQL Script section, retain the default selection so that the default SQL script that is packaged with the patch is used for modifying the schemas.

      
      

      If you want to run any custom script, then select Enter the script to apply SQL and specify the full path to the location where the custom script is available. For example, %oracle_home%/rdbms/admin/catupgrd.sql .

      Enterprise Manager Grid Control automatically runs certain scripts such as Catcpu.sql, utlrp.sql for CPUs; Catpsu.sql, utlrp.sql for PSUs; and catupgrade.slq for Patch Sets. However, some scripts mentioned in the ReadMe of a patch might have to be run manually. To run those scripts, you can select Enter the script to apply SQL and specify the full path to the location where the custom scripts are available.

      The variables you can use for the directory location are %emd_root% to indicate a location on the target host, %oracle_home% to indicate the current home location being patched, or %perlbin% to indicate the location of perl binary used by the Management Agent.

      You have an option of running both the scripts by selecting both these options or one of the scripts by selecting only one of these options. If you select both these options, then both the scripts are run, that is, the default script runs first and then the custom script runs.

   4. (Optional) In the Upgrade OPatch section, retain the default selection so that the OPatch software on the target host is upgraded before the patches are applied on the database targets.

      
      

      Enterprise Manager Grid Control uses OPatch software to apply the one-off patches. It is assumed that this software is already available on the target hosts managed by Oracle Management Agents (Management Agent), but Oracle recommends you to retain the selection so that the existing software is upgraded to the latest release.

   5. (Optional) In the Black Out Associated Targets section, retain the default selection so that all targets associated with the clusterware to be patched are blacked out while the patching is in progress.

      
      

   6. (Optional) In the Clean Up Backup Files section, select the option if you want to clean up all patching backup files after the patches are applied. The backup files are cleaned up by running OPatch utility cleanup command. Select this option depending on the space available on the target Oracle home.

   7. (Optional) In the Advanced OPatch Options section, specify any opatch-related options you want to pass while running this Deployment Procedure. Typically, you can use this section when you have patch conflicts or when inventory location is not default for the target Oracle home being patched. For example, after running the Deployment Procedure in Analyze Mode, if you identified any patch conflicts, then you can pass options to rectify the patch conflicts.

      
      

      The following are the options you can specify:

      -skip_duplicate, skips duplicate patches

      -skip_conflict, skips conflicting patches

      -skip_missing_component, skips patches whose components are missing

      -invPtrLoc, the location of oraInst.loc file

      Note:

      If you want to specify multiple options, then separate the list with a white space. For example, -skip_duplicate -skip_conflict

   8. Click Next.

5. On the Target List page, in the RAC Database Targets to be Patched section, click Add to add the Oracle RAC database targets you want to patch. If you have multiple Oracle RAC databases in a cluster and if you are not sure of the selection, then search for all RAC databases in that cluster by selecting Cluster from the Target Type list. Then click Next.

   
   

   Note:

   Selection of multiple targets to be patched in a single operation is supported. Typically, this procedure orchestrates patching in parallel across multiple hosts and does a sequential operation across the Oracle homes within a host. Within a given Oracle home, the procedure patches all of the Databases simultaneously.

6. On the Library Step Properties page (appears only if you had customized the Deployment Procedure and marked some steps to be prompted at runtime), specify values for the library steps.

7. (Optional) On the Oracle Configuration Manager page, specify an e-mail address to configure Oracle Configuration Manager. If you had already set up My Oracle Support credentials for the Enterprise Manager user account being used, then the fields are pre- filled with those details. If you have a proxy server, then provide details about the proxy server, and click Next.

   If you do not want to configure Oracle Configuration Manager, then delete the details appearing on this screen, and click Next.

8. On the Credentials page, in the Home Credentials section, specify the Oracle home credentials required to patch the Oracle homes, and in the Host Credentials section, specify the operating system credentials to log in to the hosts where the Oracle homes are present. Then click Next.

   
   

   Note:

   When there are Oracle home active steps, the Home Credentials section appears. When there are host-level active steps, the Host Credentials section appears. When Oracle home active steps and host-level active steps are present, both the sections appear.

   For both sections, choose to use the preferred credentials so that the credentials stored in the Management Repository can be used.

   Of course, you can always override the preferred credentials with a new set of credentials. If you choose to do so, you can specify either a common set of credentials to be used across Oracle homes and hosts, or a unique set of credentials for each Oracle home and host.

   After specifying a new set of credentials to override the preferred credentials, click Save OH Credential if you want to store the new credentials in the Management Repository.

9. On the Schedule page, schedule the Deployment Procedure to run either immediately or later.

   For Instance Name, specify a unique name for this Deployment Procedure instance so that it can be tracked later and reused with the same settings.

10. On the Review page, review the details and do one of the following:

    • If you want to check for prerequisites, then click Analyze.

    • If you have already checked for prerequisites or if you do not want to check for prerequisites, then click Deploy to patch the selected databases.

Deployment Phases

The following describes the different phases involved in this Deployment Procedure:

Table 23-3 Deployment Phases for Patching Oracle RAC in Offline Mode

Phase	Description
	Software Updates Allows you to select the software updates (patches and patch sets) you want to apply, apply SQL script, upgrade OPatch, black out associated targets, clean up backup files, and provide advanced options for OPatch. Note: You must not combine patches and patch sets in a single selection set.
	Target List Allows you to select the database targets you want to patch.
	Library Step Properties Allows you to specify values for steps that you want to edit at runtime.
	Oracle Configuration Manager Allows you to configure Oracle Configuration Manager.
	Credentials Allows you to specify Oracle home credentials required to patch RAC Oracle homes, and host credentials to access the hosts where the Oracle RAC instances are present.
	Schedule Allows you to schedule the Deployment Procedure to run immediately or later.
	Review Allows you to review the details you have provided for patching the selected Oracle RAC instances.

Prerequisites

Before running the Deployment Procedure, meet the following prerequisites:

Prerequisites for Designers

• Ensure that you meet the prerequisites described in the following sections:

  • Mandatory Infrastructure Requirements for Provisioning and Patching

  • Optional Infrastructure Requirements

  • Setting Up Infrastructure for Offline Patching

• Ensure that operating system users such as oracle and crsuser are available on all the nodes of the cluster. These users must be a part of the relevant operating system groups such as dba and oinstall.

• Ensure that you run the patchability report to understand whether the Oracle Database you want to patch is suitable for this patching operation. In the report, if the Oracle Database you want to patch appears as a target that cannot be patched, then fix the target configuration issues to make it a target that can be patched. For more information about the patchability report, and to understand how you can diagnose the issues with an Oracle Database target and take corrective actions, see Appendix H, "About Patching Automation Reports".

• Ensure that you run the Deployment Procedure in the Analyze mode.

Prerequisites for Operators

• Ensure that you use an operating system user that has the privileges to run the Deployment Procedure, and that can switch to root user and run all commands on the target hosts. For example, commands such as mkdir, ls, and so on.

  If you do not have the privileges to do so, that is, if you are using a locked account, then request your administrator (a designer) to either customize the Deployment Procedure to run it as another user or ignore the steps that require special privileges.

  For example, user account A might have the root privileges, but you might use user account B to run the Deployment Procedure. In this case, you can switch from user account B to A by customizing the Deployment Procedure.

  For information about customization, see Chapter 30, "Customizing Steps to Run as Another User".

• Ensure that you are the owner of the Oracle home of the targets being patched. If you are not the owner or if you do not have direct access to those targets, then customize the Deployment Procedure to run as another user.

  For information about customization, see Chapter 30, "Customizing Steps to Run as Another User".

Patching Procedure

To patch an Oracle RAC database, follow these steps:

1. In Grid Control, click the Deployments tab.

2. On the Deployments page, in the Patching section, click Patching through Deployment Procedures.

3. On the Deployment Procedure Manager page, in the Procedures subtab, from the table, select Patch Oracle RAC Database - Rolling. Then click Schedule Deployment.

   Enterprise Manager Grid Control displays the Software Updates page of the Deployment Procedure.

4. On the Software Updates page, do the following:

   1. In the Select Stage Location section, retain the default selection, that is, Enter Stage Location that has the value %emd_root%/EMStage.

      This default value refers to a location on the target host where the Oracle RAC to be patched is running. The software updates are downloaded from My Oracle Support to Oracle Software Library, and then staged in this location on the target hosts.

      
      

      You can modify the location if you want to, and click Save. If you save multiple custom locations, then the Select Stage Location option gets enabled and allows you to select any of the saved custom locations the next time you visit the same page.

      Instead of %emd_root%, you can also specify %oracle_home% to indicate the current home location being patched.

   2. In the RAC Database Updates section, click Add to search and select software updates from My Oracle Support. Alternatively, if you have a text file that contains details about the patches, then click Upload From File to select and upload that text file.

      On clicking Add, Enterprise Manager Grid Control displays the Search and Select Software Updates page. On this page, select Search Software Library, specify the details of the patches you want to search, and click Go.

      Enterprise Manager Grid Control searches the patches in Oracle Software Library, and displays details about them on the Search and Select Software Updates page. Select the ones you want to apply using the Deployment Procedure, and click Select.

      Note:

      Selection of multiple patches is supported.

      On clicking Upload From File, Enterprise Manager Grid Control displays the Upload File page. On this page, select Upload from Local Machine or Upload from Software Library depending on where the text file is available, click Browse to search and select that file, and then click Upload.

   3. (Optional) In the Apply SQL Script section, retain the default selection so that the default SQL script that is packaged with the patch is used for modifying the schemas.

      
      

      If you want to run any custom script, then select Enter the script to apply SQL and specify the full path to the location where the custom script is available. For example, %oracle_home%/rdbms/admin/catupgrd.sql .

      Enterprise Manager Grid Control automatically runs certain scripts such as Catcpu.sql, utlrp.sql for CPUs; Catpsu.sql, utlrp.sql for PSUs; and catupgrade.slq for Patch Sets. However, some scripts mentioned in the ReadMe of a patch might have to be run manually. To run those scripts, you can select Enter the script to apply SQL and specify the full path to the location where the custom scripts are available.

      The variables you can use for the directory location are %emd_root% to indicate a location on the target host, %oracle_home% to indicate the current home location being patched, or %perlbin% to indicate the location of perl binary used by the Management Agent.

      You have an option of running both the scripts by selecting both these options or one of the scripts by selecting only one of these options. If you select both these options, then both the scripts are run, that is, the default script runs first and then the custom script runs.

   4. (Optional) In the Black Out Associated Targets section, retain the default selection so that all targets associated with the Oracle RAC database to be patched are blacked out while the patching is in progress.

      
      

   5. (Optional) In the Clean Up Backup Files section, select the option if you want to clean up all patching backup files after the patches are applied. The backup files are cleaned up by running OPatch utility cleanup command. Select this option depending on the space available on the target Oracle home.

   6. (Optional) In the Advanced OPatch Options section, specify any opatch-related options you want to pass while running this Deployment Procedure. Typically, you can use this section when you have patch conflicts or when inventory location is not default for the target Oracle home being patched. For example, after running the Deployment Procedure in Analyze Mode, if you identified any patch conflicts, then you can pass options to rectify the patch conflicts.

      
      

      The following are the options you can specify:

      -skip_duplicate, skips duplicate patches

      -skip_conflict, skips conflicting patches

      -skip_missing_component, skips patches whose components are missing

      -invPtrLoc, the location of oraInst.loc file

      Note:

      If you want to specify multiple options, then separate the list with a white space. For example, -skip_duplicate -skip_conflict

   7. Click Next.

5. On the Target List page, in the RAC Database Targets to be Patched section, click Add to add the Oracle RAC database targets you want to patch. If you have multiple Oracle RAC databases in a cluster and if you are not sure of the selection, then search for all RAC databases in that cluster by selecting Cluster from the Target Type list. Then click Next.

   
   

   Note:

   Selection of multiple targets to be patched in a single operation is supported. Typically, this procedure orchestrates patching in parallel across multiple hosts and does a sequential operation across the Oracle homes within a host. Within a given Oracle home, the procedure patches all of the Databases simultaneously.

6. On the Library Step Properties page (appears only if you had customized the Deployment Procedure and marked some steps to be prompted at runtime), specify values for the library steps.

7. (Optional) On the Oracle Configuration Manager page, specify an e-mail address to configure Oracle Configuration Manager. If you had already set up My Oracle Support credentials for the Enterprise Manager user account being used, then the fields are pre- filled with those details. If you have a proxy server, then provide details about the proxy server, and click Next.

   If you do not want to configure Oracle Configuration Manager, then delete the details appearing on this screen, and click Next.

8. On the Credentials page, in the Home Credentials section, specify the Oracle home credentials required to patch the Oracle homes, and in the Host Credentials section, specify the operating system credentials to log in to the hosts where the Oracle homes are present. Then click Next.

   
   

   Note:

   When there are Oracle home active steps, the Home Credentials section appears. When there are host-level active steps, the Host Credentials section appears. When Oracle home active steps and host-level active steps are present, both the sections appear.

   For both sections, choose to use the preferred credentials so that the credentials stored in the Management Repository can be used.

   Of course, you can always override the preferred credentials with a new set of credentials. If you choose to do so, you can specify either a common set of credentials to be used across Oracle homes and hosts, or a unique set of credentials for each Oracle home and host.

   After specifying a new set of credentials to override the preferred credentials, click Save OH Credential if you want to store the new credentials in the Management Repository.

9. On the Schedule page, schedule the Deployment Procedure to run either immediately or later.

   For Instance Name, specify a unique name for this Deployment Procedure instance so that it can be tracked later and reused with the same settings.

10. On the Review page, review the details and do one of the following:

    • If you want to check for prerequisites, then click Analyze.

    • If you have already checked for prerequisites or if you do not want to check for prerequisites, then click Deploy to patch the selected databases.