disk access issue
The oracle instances are terminating on various nodes with errors like these but then we are able to start the instance in some cases. It looks like disk access issues.
This is from node 5
ORA-00604: error occurred at recursive SQL level 3
ORA-00376: file 1 cannot be read at this time
ORA-01110: data file 1: '+DATA_PR02/orp722z/datafile/system.1256.861636119'
Errors in file /u01/app/oracle/diag/rdbms/orp722z/orp722z5/trace/orp722z5_rsm0_31902.trc:
I checked the storage and all the physical disks are showing normal .I also don’t see any alert from cell/storage server .But I see ASM/db instances terminated multiple times