Opatch Log Files
- Main log file: /u01/app/11204/grid/cfgtoollogs/opatchauto2014-07-21_10-23-50.log
- Configuration log file with details about CRS commands: /u01/app/11204/grid/cfgtoollogs/opatchauto2014-07-21_10-23-50.report.log
Running Opatch auto
[root@grac41 Desktop]# $GRID_HOME/OPatch/opatch auto /PATCHES/Patch112043/18706472 -ocmrf $GRID_HOME/OPatch/ocm.rsp
Executing /u01/app/11204/grid/perl/bin/perl /u01/app/11204/grid/OPatch/crs/patch11203.pl -patchdir /PATCHES/Patch112043
-patchn 18706472 -ocmrf /u01/app/11204/grid/OPatch/ocm.rsp
-paramfile /u01/app/11204/grid/crs/install/crsconfig_params
This is the main log file: /u01/app/11204/grid/cfgtoollogs/opatchauto2014-07-21_10-23-50.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11204/grid/cfgtoollogs/opatchauto2014-07-21_10-23-50.report.log
2014-07-21 10:23:50: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11204/grid/crs/install/crsconfig_params
Stopping RAC /u01/app/oracle/product/11204/racdb ...
Stopped RAC /u01/app/oracle/product/11204/racdb successfully
patch /PATCHES/Patch112043/18706472/18522509 apply successful for home /u01/app/oracle/product/11204/racdb
patch /PATCHES/Patch112043/18706472/18522515/custom/server/18522515 apply successful for home /u01/app/oracle/product/11204/racdb
Stopping CRS...
The Oracle Grid Infrastructure stack failed to stop.
Resetting the status, starting CRS...
ERROR: Failed to stop CRS.Refer log file for more details.
--> Patching CRS was unsucessfull
Check Main Install Log file
2014-07-21 10:27:20: postpatch execution for DB home ... success
2014-07-21 10:27:20: Processing oracle home /u01/app/11204/grid
2014-07-21 10:27:20: Opening file /etc/oracle/ocr.loc
2014-07-21 10:27:20: Value (false) is set for key=local_only
2014-07-21 10:27:20: Home type of /u01/app/11204/grid is CRS
2014-07-21 10:27:20: Unlock crshome...
...
2014-07-21 10:33:23: /u01/app/11204/grid/bin/crsctl stop crs -f
2014-07-21 10:33:23: Executing cmd: /u01/app/11204/grid/bin/crsctl check cluster -n grac41
...
CRS-2799: Failed to shut down resource 'procwatcher' on 'grac41'
CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'grac41' has failed
CRS-5022: Stop of resource "ora.crsd" failed: current state is "UNKNOWN"
CRS-2675: Stop of 'ora.crsd' on 'grac41' failed
CRS-2799: Failed to shut down resource 'ora.crsd' on 'grac41'
CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'grac41' has failed
CRS-4687: Shutdown command has completed with errors.
CRS-4000: Command Stop failed, or completed with errors.
################################################################
#You must kill processes or reboot the system to properly #
#cleanup the processes started by Oracle Grid Infrastructure #
################################################################
The Oracle Grid Infrastructure stack failed to stop.
You should stop the stack with 'crsctl stop crs' and rerun the command
..
2014-07-21 10:35:25: ERROR: Failed to stop CRS.Refer log file for more details.
Configuration Log:
*********** Configuration Data ***********
* It shows only those targets that will be patched in this session *
crs_home=/u01/app/11204/grid version=11.2.0.4.0 owner=grid opatch_ver=11.2.0.3.6
rac_home=/u01/app/oracle/product/11204/racdb version=11.2.0.4.0 owner=oracle opatch_ver=11.2.0.3.6
*********** Steps to be executed as owner unless specified as root ***********
...
19: /PATCHES/Patch112043/18706472/18522515/custom/server/18522515/custom/scripts/postpatch.sh
-dbhome /u01/app/oracle/product/11204/racdb
20: /u01/app/11204/grid/crs/install/rootcrs.pl -unlock : run as root
--> CRS patching failed running last step in CRS Configuratuon file : rootcrs.pl -unlock
rootcrs.pl -unlock failed running : /u01/app/11204/grid/bin/crsctl stop crs -f
/u01/app/11204/grid/bin/crsctl stop crs -f fails to stop the procwatcher resource
Try to stop clusterware manually :
[root@grac41 Desktop]# /u01/app/11204/grid/bin/crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'grac41'
CRS-2673: Attempting to stop 'ora.crsd' on 'grac41'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'grac41'
CRS-2679: Attempting to clean 'procwatcher' on 'grac41'
CRS-2679: Attempting to clean 'procwatcher' on 'grac41'
CRS-2680: Clean of 'procwatcher' on 'grac41' failed
CRS-5809: Failed to execute 'ACTION_SCRIPT' value of '/u01/app/11204/grid/log/procwatcher/prw.sh' for 'procwatcher'. Error information
'cmd /u01/app/11204/grid/log/procwatcher/prw.sh not found', Category : -2, OS error : 2
CRS-2680: Clean of 'procwatcher' on 'grac41' failed
CRS-2799: Failed to shut down resource 'procwatcher' on 'grac41'
CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'grac41' has failed
CRS-5022: Stop of resource "ora.crsd" failed: current state is "UNKNOWN"
CRS-2675: Stop of 'ora.crsd' on 'grac41' failed
CRS-2799: Failed to shut down resource 'ora.crsd' on 'grac41'
CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'grac41' has failed
CRS-4687: Shutdown command has completed with errors.
CRS-4000: Command Stop failed, or completed with errors.
--- > Following errors need to be fixed
CRS-2799: Failed to shut down resource 'procwatcher' on 'grac41'
CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'grac41' has failed
Fix the problem with missing procwatcher resource
As user grid run
$ scp grac42:/u01/app/11204/grid/log/procwatcher/prw.sh /u01/app/11204/grid/log/procwatcher
$ chmod 755 /u01/app/11204/grid/log/procwatcher/prw.sh
Now test CRS stutdown
[root@grac41 Desktop]# /u01/app/11204/grid/bin/crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'grac41'
..
CRS-4133: Oracle High Availability Services has been stopped.
As user root run the failing step again.
[root@grac41 Desktop]# /u01/app/11204/grid/crs/install/rootcrs.pl -unlock
Using configuration parameter file: /u01/app/11204/grid/crs/install/crsconfig_params
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.
Successfully unlock /u01/app/11204/grid
After fixing the CRS shutdown problem rerun opatch auto again
--> it should not install any remaining not ye installed patches
$GRID_HOME/OPatch/opatch auto /PATCHES/Patch112043/18706472 -ocmrf $GRID_HOME/OPatch/ocm.rsp
Verify patch installation with
[oracle@grac41 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory
grid@grac41 ~]$ $GRID_HOME/OPatch/opatch lsinventory