SAN migration in RAC

[sivakumarocp]

Hi Tim,

Right now I am working in SAN Migration, we are moving our storage from SAN to NETAPP for one of our 3 node RAC cluster.

our env:

Gird version : 11.2.0.1
DB Version : 10.2.0.4

Code: Select all

NAME         Total_GB    Free_GB
---------- ---------- ----------
ARCHLOG           400     397.58
CRS_VOTE            5       4.61
DATA          20501.4   10195.64
LOG1               40      12.55
LOG2               40      12.59
LOG3               40      32.69
LOG4               40      39.83

Since this is 11g R2 cluster we placed OCR and Vote disk into single Diskgroup named CRS_VOTE.

Right now we are in the process of adding new disks (from NETAPP storage) into DATA diskgroup and parallely dropping old disks (from SAN storage) in one command.
once the rebalance got completed we need to move OCR and Votedisk to new storage for this how can I do.

means what are the steps should i follow inorder to move those two files from SAN to NETAPP storage.

Right now we are doing this migration on Test ENV once it completed successfully we will implement the same in Production also.

Code: Select all

oracle:+ASM1@lnxractst3a /oracle > ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3280
         Available space (kbytes) :     258840
         ID                       : 1403990432
         Device/File Name         :  +CRS_VOTE
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check bypassed due to non-privileged user



We are not using any mirroring for OCR disk.

in this situation how can I achive this task.

[Tim...]

Hi.

For the OCR you can do this:

http://docs.oracle.com/cd/E11882_01/rac ... m#BABBDAHA

Note. You need at least two OCR location to allow a move, you if you want to keep a single OCR, you will need to add a new one in the new location, then delete the old one, rather than replacing the existing one.

The same page contains similar information for the voting disk:

http://docs.oracle.com/cd/E11882_01/rac ... m#BGBBIGJH

Cheers

Tim...

[sivakumarocp]

Hi Tim,

I was trying to add new OCR location in another Diskgroup, As per the Oracle Doc which you given, but I am getting below error during the OCR addition.

Code: Select all

[root@rac1 bin]# ./ocrconfig -add +NEW_CRS
PROT-30: The Oracle Cluster Registry location to be added is not usable
PROC-8: Cannot perform cluster registry operation because one of the parameters is invalid.
ORA-15056: additional error message
ORA-17502: ksfdcre:4 Failed to create file +NEW_CRS.255.1
ORA-15221: ASM operation requires compatible.asm of 11.1.0.0.0 or higher
ORA-06512: at line 4


while creating +NEW_CRS diskgroup i didn't mention the compatible.asm attribute. is it mandatory to specify the compatible.asm attribute during the diskgroup creation?

why I am hitting this error while adding new OCR location.

I tried this in on my 11g R2 test env.

Code: Select all

NAME                           STATE
------------------------------ -----------
CRS_VOTE                       MOUNTED
DATAFILE                       MOUNTED
NEW_CRS                        MOUNTED


[sivakumarocp]

I created new diskgroup with compatible.asm attribute.

follower your doc http://www.oracle-base.com/articles/11g ... -11gr1.php

after that i was able to create OCR mirror on new location.

Here my question is, is that mandatory to specify compatible.asm parameter during Diskgroup creation? eventhough i am using 11 R2 cluster

[sivakumarocp]

Hi Tim,

I have successfully migrated OCR and Votedisk into New diskgroup.

Code: Select all

[root@rac2 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3052
         Available space (kbytes) :     259068
         ID                       : 2027747662
         Device/File Name         :   +NEW_CRS
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

[root@rac2 bin]#
[root@rac2 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   9678360b6b454f4bbf8bac496a5dafb2 (/dev/oracleasm/disks/DISK4) [NEW_CRS]
Located 1 voting disk(s).
[root@rac2 bin]#


After that I was dismounting my old CRS_VOTE diskgroup but I am hitting with below error. I am not sure why I am not able to dismount this Diskgroup.

Code: Select all

SQL> alter diskgroup CRS_VOTE dismount;
alter diskgroup CRS_VOTE dismount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15027: active use of diskgroup "CRS_VOTE" precludes its dismount



is there my old OCR and Votedisk files are used on my CRS_VOTE diskgroup???

[Tim...]

Hi.

Unless you have stopped your clusterware, then it's possible something is actually accessing these disk groups.

Cheers

Tim...

[sivakumarocp]

I have tried that also. Even after I bounced my entire cluster I tried to dimount that diskgroup but still hitting with same error

can I dismount that diskgroup with force option???

If I do so will it become any problem to my OCR and Vote disk on new location?


2) is that mandatory to specify compatible.asm parameter during Diskgroup creation? eventhough i am using 11 R2 cluster.
If I am not specifying this parameter during Diskgroup creation then what would be the value for this parameter.

[Tim...]

Hi.

1) I wouldn't expect it to be a problem provided you have set up the new OCR and voting disks correctly. Before doing that, I would want to be convinced what it locking them though.

2) It is obviously not mandatory to specify it. The default values are specified in the manuals. It depends how the disc group was created.

http://docs.oracle.com/cd/E16338_01/ser ... m#BABGEICD

Cheers

Tim...

[sivakumarocp]

Is there any way to identify which process is accessing this +CRS_VOTE diskgroup?

Mover over I have migrated successfully both OCR & Votedisk into new dg i.e +NEW_CRS and i checked the integrity of OCR is fine.

Code: Select all

[root@rac2 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[sivakumarocp]

Hi Tim,

Finally I dropped my old dg +CRS_VOTE using FORCE option

After that when I check the status of the cluster resource, that time +CRS_VOTE diskgroup is in OFFLINE! why its still in cluster resource even though I dropped this diskgroup

Code: Select all

[oracle@rac1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS_VOTE.dg
               OFFLINE OFFLINE      rac1
               OFFLINE OFFLINE      rac2


I am not sure this is because of ASM Metadata, if so how can I delete the matadata for this Diskgroup

[Tim...]

Hi.

If you have dropped the disk group, the only possible cause for this is it is still present in the spfile or pfile as part of the ASM_DISKGROUPS parameter. Make sure the reference to the disk group is removed.

Cheers

Tim...