This forum is currently locked. You can't register or post questions at this time. (read more)

root.sh fails on second node

Questions relating to Oracle Real Application Clusters (RAC) and Clusterware.

Moderator: Tim...

Tim...
Site Admin
Posts: 18437
Joined: Mon Nov 01, 2004 5:56 pm
Location: England, UK
Contact:

Re: root.sh fails on second node

Postby Tim... » Wed Jun 25, 2014 12:48 pm

Hi.

Yes. That should work fine.

Cheers

Tim...
Tim...
Oracle ACE Director
Oracle ACE of the Year 2006 - Oracle Magazine Editors Choice Awards
OakTable Member
OCP DBA 7.3, 8, 8i, 9i, 10g, 11g
OCP Advanced PL/SQL Developer
Oracle Database: SQL Certified Expert
My website: http://www.oracle-base.com
My blog: http://www.oracle-base.com/blog

1982
Senior Member
Posts: 72
Joined: Mon Apr 07, 2014 3:53 pm

Re: root.sh fails on second node

Postby 1982 » Thu Jul 24, 2014 10:48 am

Hi Tim,


Sorry to repeat the problem again But I am still stucking in the same scenario, Kindly help me over the same, this time i done little bit more analysis:-

I am running Linux 32bit on VmWare Workstation. Installing Oracle 11gR2 Grid for Oracle 2 Node RAC.
It passed all the per-requisite. Run root.sh on first node. It finished with no errorrs.
On second node I got the following:

Running Oracle 11g root.sh script...

The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid


[root@rac2 rpm]# sh /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: n
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: n
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: n

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2014-03-23 06:03:12: Parsing the host name
2014-03-23 06:03:12: Checking for super user privileges
2014-03-23 06:03:12: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'
CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
Timed out waiting for the CRS stack to start.


When i check the log on node 2:-


[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:26.377
[crsd(12144)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:27.289
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:28.475
[crsd(12163)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:29.363
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:30.541
[crsd(12178)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:31.419
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:32.632
[crsd(12193)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:33.459
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:34.586
[crsd(12213)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:35.488
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:36.659
[crsd(12228)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:37.514
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:38.690
[crsd(12247)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:39.546
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:40.732
[crsd(12262)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:41.631
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:42.944
[crsd(12278)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/ol5-112-rac2/crsd/crsd.log.
2014-07-24 15:37:43.782
[ohasd(10490)]CRS-2765:Resource 'ora.crsd' has failed on server 'ol5-112-rac2'.
2014-07-24 15:37:43.783
[ohasd(10490)]CRS-2771:Maximum restart attempts reached for resource 'ora.crsd'; will not restart.
2014-07-24 15:41:21.412
[ctssd(11924)]CRS-2408:The clock on host ol5-112-rac2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.
2014-07-24 15:41:37.416
[ctssd(11924)]CRS-2408:The clock on host ol5-112-rac2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.

After this i search further and getting the details in log as :-

2014-07-24 15:37:42.740: [ GPnP][3046856400]clsgpnpwu_walletfopen: [at clsgpnpwu.c:494] Opened SSO wallet: '/u01/app/11.2.0/grid/gpnp/ol5-112-rac2/wallets/peer/cwallet.sso'
2014-07-24 15:37:42.740: [ GPnP][3046856400]clsgpnp_getCK: [at clsgpnp0.c:1965] Result: (0) CLSGPNP_OK. Get gpnp wallet - provider 1 of 2 (LSKP-FSW(1))
2014-07-24 15:37:42.740: [ GPnP][3046856400]clsgpnp_getCK: [at clsgpnp0.c:1982] Got gpnp security keys (wallet).>
2014-07-24 15:37:42.740: [ GPnP][3046856400]clsgpnp_Init: [at clsgpnp0.c:837] GPnP client pid=12278, tl=3, f=0
2014-07-24 15:37:42.748: [GIPCXCPT][3046856400] gipcShutdownF: skipping shutdown, count 2, from [ clsinet.c : 1732], ret gipcretSuccess (0)
2014-07-24 15:37:42.775: [GIPCXCPT][3046856400] gipcShutdownF: skipping shutdown, count 1, from [ clsgpnp0.c : 1021], ret gipcretSuccess (0)
2014-07-24 15:37:42.907: [ OCRASM][3046856400]proprasmo: Error in open/create file in dg [DATA]
[ OCRASM][3046856400]SLOS : SLOS: cat=8, opn=kgfoOpenFile01, dep=15056, loc=kgfokge
ORA-17503: ksfdopn:DGOpenFile05 Failed to open file +DATA.255.4294967295
ORA-17503: ksfdopn:2 Failed to open file +DATA.255.4294967295
ORA-15001: diskgroup "DAT


2014-07-24 15:37:42.943: [ OCRASM][3046856400]proprasmo: kgfoCheckMount returned [6]
2014-07-24 15:37:42.943: [ OCRASM][3046856400]proprasmo: The ASM disk group DATA is not found or not mounted
2014-07-24 15:37:42.944: [ OCRRAW][3046856400]proprioo: Failed to open [+DATA]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE.
2014-07-24 15:37:42.944: [ OCRRAW][3046856400]proprioo: No OCR/OLR devices are usable
2014-07-24 15:37:42.944: [ OCRASM][3046856400]proprasmcl: asmhandle is NULL
2014-07-24 15:37:42.944: [ OCRRAW][3046856400]proprinit: Could not open raw device
2014-07-24 15:37:42.944: [ OCRASM][3046856400]proprasmcl: asmhandle is NULL
2014-07-24 15:37:42.944: [ OCRAPI][3046856400]a_init:16!: Backend init unsuccessful : [26]
2014-07-24 15:37:42.944: [ CRSOCR][3046856400] OCR context init failure. Error: PROC-26: Error while accessing the physical storage ASM error [SLOS: cat=8, opn=kgfoOpenFile01, dep=15056, loc=kgfokge
ORA-17503: ksfdopn:DGOpenFile05 Failed to open file +DATA.255.4294967295
ORA-17503: ksfdopn:2 Failed to open file +DATA.255.4294967295
ORA-15001: diskgroup "DAT
] [8]
2014-07-24 15:37:42.944: [ CRSD][3046856400][PANIC] CRSD exiting: Could not init OCR, code: 26
2014-07-24 15:37:42.945: [ CRSD][3046856400] Done.

As per the logs, it is clear that node 2 is not able to access the asm disks. But when i issue:-

On Node1:-

[oracle@ol5-112-rac1 bin]$ oracleasm listdisks
DISK1
DISK2
DISK3
DISK4
DISK5
On Node 2:-

[root@ol5-112-rac2 ol5-112-rac2]# oracleasm listdisks
DISK1
DISK2
DISK3
DISK4
DISK5

It means that both node able to see the ASM disk


The Status of both nodes are as:-

[oracle@ol5-112-rac1 bin]$ ./crsctl check cluster -all
**************************************************************
ol5-112-rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
ol5-112-rac2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

Tim...
Site Admin
Posts: 18437
Joined: Mon Nov 01, 2004 5:56 pm
Location: England, UK
Contact:

Re: root.sh fails on second node

Postby Tim... » Thu Jul 24, 2014 1:51 pm

Hi.

I would do a diagnostics check to see if everything is set up correctly.

http://www.oracle-base.com/articles/rac ... php#ORAchk

There is obviously a problem somewhere...

Cheers

Tim...
Tim...
Oracle ACE Director
Oracle ACE of the Year 2006 - Oracle Magazine Editors Choice Awards
OakTable Member
OCP DBA 7.3, 8, 8i, 9i, 10g, 11g
OCP Advanced PL/SQL Developer
Oracle Database: SQL Certified Expert
My website: http://www.oracle-base.com
My blog: http://www.oracle-base.com/blog

kiranta
Member
Posts: 1
Joined: Fri Aug 15, 2014 9:42 am

Re: root.sh fails on second node

Postby kiranta » Fri Aug 15, 2014 9:52 am

Hi.

Strange. ASMLib is supposed to cope with permissions and disk order. The fact you can list the disks suggests ASMLib should be working properly.

Can you check that SE Linux is set to permissive? What do you get when you run the following command on each node?


Return to “Oracle RAC and Clusterware”

Who is online

Users browsing this forum: No registered users and 1 guest

cron