Automatic Storage Manager (ASM) Enhancements in Oracle Database 11g Release 1
This article provides an overview of the main Automatic Storage Manager (ASM) enhancements in Oracle Database 11g Release 1, including:
- Oracle Database Storage Administrator's Guide
- New Disk Group Compatibility Attributes
- Fast Mirror Resync
- Rolling Upgrade
- SYSASM Privilege and OSASM OS Group
- Scalability and Performance Enhancements
- New ASMCMD Commands and Options
- Preferred Read Failure Groups
- Fast Rebalance
- Miscellaneous Disk Group Maintenance Enhancements
- Automatic Storage Management (ASM) in Oracle Database 10g
- ASM using ASMLib and Raw Devices
- Using NFS with ASM
- Oracle and RAID
Oracle Database Storage Administrator's Guide
The documentation for this release now includes a Oracle Database Storage Administrator's Guide for the first time.
New Disk Group Compatibility Attributes
Oracle 11g ASM includes two new compatibility attributes that determine the version of the ASM and database software that can use specific disk groups:
COMPATIBLE.ASM- The minimum version of the ASM software that can access the disk group. In 11g, the default setting is 10.1.
COMPATIBLE.RDBMS- The minimum
COMPATIBLEdatabase initialization parameter setting for any database instance that uses the disk group. In 11g, the default setting is 10.1.
The compatibility versions of a disk group can only be increased, not decreased. If you have increased the version by mistake, you will need to create a new disk group.
The disk group compatibility attributes can be set during disk group creation by adding the
ATTRIBUTE clause to the
CREATE DISKGROUP command.
CREATE DISKGROUP data DISK '/dev/raw/*' ATTRIBUTE 'compatible.asm' = '11.1'; CREATE DISKGROUP data DISK '/dev/raw/*' ATTRIBUTE 'compatible.rdbms' = '11.1', 'compatible.asm' = '11.1';
The disk group compatibility attributes for existing disk groups can be altered using the
SET ATTRIBUTE clause to the
ALTER DISKGROUP command.
ALTER DISKGROUP data SET ATTRIBUTE 'compatible.asm' = '11.1'; ALTER DISKGROUP data SET ATTRIBUTE 'compatible.rdbms' = '11.1';
The current compatibility settings are available from the
COLUMN name FORMAT A10 COLUMN compatibility FORMAT A20 COLUMN database_compatibility FORMAT A20 SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup; GROUP_NUMBER NAME COMPATIBILITY DATABASE_COMPATIBILI ------------ ---------- -------------------- -------------------- 1 DATA 18.104.22.168.0 22.214.171.124.0 1 row selected. SQL> COLUMN name FORMAT A20 COLUMN value FORMAT A20 SELECT group_number, name, value FROM v$asm_attribute ORDER BY group_number, name; GROUP_NUMBER NAME VALUE ------------ -------------------- -------------------- 1 au_size 1048576 1 compatible.asm 126.96.36.199.0 1 compatible.rdbms 11.1 1 disk_repair_time 3.6h 4 rows selected. SQL>
Fast Mirror Resync
During transient disk failures within a failure group, ASM keeps track of the changed extents that need to be applied to the offline disk. Once the disk is available, only the changed extents are written to resynchronize the disk, rather than overwriting the contents of the entire disk. This can speed up the resynchronization process considerably.
Fast mirror resync is only available when the disk groups compatibility attributes are set to 11.1 or higher.
ALTER DISKGROUP disk_group_1 SET ATTRIBUTE 'compatible.asm' = '11.1'; ALTER DISKGROUP disk_group_1 SET ATTRIBUTE 'compatible.rdbms' = '11.1;
ASM drops disks if they remain offline for more than 3.6 hours. The disk groups default time limit is altered by changing the
DISK_REPAIR_TIME parameter with a unit of minutes (M or m) or hours (H or h).
-- Set using the hours unit of time. ALTER DISKGROUP disk_group_1 SET ATTRIBUTE 'disk_repair_time' = '4.5h'; -- Set using the minutes unit of time. ALTER DISKGROUP disk_group_1 SET ATTRIBUTE 'disk_repair_time' = '300m';
DROP AFTER clause of the
ALTER DISKGROUP command is used to override the disk group default
-- Use the default DISK_REPAIR_TIME for the diskgroup. ALTER DISKGROUP disk_group_1 OFFLINE DISK D1_0001; -- Override the default DISK_REPAIR_TIME. ALTER DISKGROUP disk_group_1 OFFLINE DISK D1_0001 DROP AFTER 30m;
If a disk goes offline during a rolling upgrade, the timer is not started until after the rolling upgrade is complete.
Clustered ASM instances for 11g onwards can be upgraded using a rolling upgrade. The ASM cluster is placed in rolling upgrade mode by issuing the following command from one of the nodes.
ALTER SYSTEM START ROLLING MIGRATION TO 188.8.131.52.0;
Once the cluster is in rolling upgrade mode each node in turn can be shutdown, upgraded and started. The cluster runs in a mixed version environment until the upgrade is complete. In this state, the cluster is limited to the following operations:
- Mount and dismount of the disk groups.
- Open, close, resize, and delete of database files.
- Access to local fixed views and fixed packages.
The current status of the ASM cluster can be determined using the following query.
SELECT SYS_CONTEXT('sys_cluster_properties', 'cluster_state') FROM dual;
Once the last node is upgraded, the rolling upgrade is stopped by issuing the following command, which checks all ASM instances are at the appropriate version, turns off rolling upgrade mode and restarts any pending rebalance operations.
ALTER SYSTEM STOP ROLLING MIGRATION;
Restrictions and miscellaneous points about the rolling upgrade process include:
- The Oracle clusterware must be fully patched before an ASM rolling upgrade is started.
- Rolling upgrades are only available from 11g onwards, so this method is not suitable for 10g to 11g upgrades.
- This method can be used to rollback to the previous version if the rolling upgrade fails before completion.
- If the upgrade fails, any rebalancing operations must complete before a new upgrade can be attempted.
- New instances joining the cluster during a rolling upgrade are automatically placed in rolling upgrade mode.
- If all instances in a cluster are stopped during a rolling upgrade, once the instances restart they will no longer be in rolling upgrade mode. The upgrade must be initiated as if it were a new process.
SYSASM Privilege and OSASM OS Group
The introduction of ASM moved the management of storage away from system adminstrators and into the DBA territory. Unfortunately this isn't how every company operates. As a result, some system administrators were required to have access to privileged users to handle disk storage. Oracle addresses this issue with the introduction of the SYSASM privilege and the OSASM operating system group, which provide two mechanisms to enable the separation of storage and database administration duties.
Users can be created in the ASM instance in a similar manner to database users. Granting these users the SYSASM privilege allows them to connect to the ASM instance and perform administration tasks. First, connect to the ASM instance.
$ export ORACLE_SID=+ASM $ sqlplus / as sysasm
Next, create a new user in the ASM instance and grant it the SYSASM privilege. The user is now able to connect using SYSASM.
SQL> CREATE USER asm_user_1 IDENTIFIED by asm_password; User created. SQL> GRANT SYSASM TO asm_user_1; Grant succeeded. SQL> CONN asm_user_1/asm_password AS SYSASM Connected. SQL>
Alternatively, assigning an operating system user to the OSASM group (asmadmin) allows then to connect as SYSASM using OS authentication. The following example creates a new OS user assigned to the OSASM group (asmadmin), which is immediately able to connect using SYSASM.
# useradd tim_hall -G asmadmin # su - tim_hall $ export ORACLE_SID=+ASM $ export ORACLE_HOME=/u01/app/oracle/product/11.1.0/db_1 $ $ORACLE_HOME/bin/sqlplus / as sysasm SQL*Plus: Release 184.108.40.206.0 - Production on Fri Aug 8 16:48:37 2008 Copyright (c) 1982, 2007, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 220.127.116.11.0 - Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL>
Scalability and Performance Enhancements
ASM files are stored in a disk group as a collection of extents. In Oracle 10g each extent mapped directly to one allocation unit (AU), but in Oracle 11g an extent can be made up of one or more allocation units. As files get larger, the extent size can grow to reduce the size of the extent map describing the file, thus saving memory.
When the disk group compatibility attributes are set to 11.1 or higher, the extent size will automatically grow as the file grows. In 11.1, the first 20,000 extents match the allocation unit size (1*AU). The next 20,000 extents are made up of 8 allocation units (8*AU). Beyond that point, the extent size becomes 64 allocation units (64*AU). In 11.2 this behavior has changed from 1, 8, 64 to 1, 4, 16.
In addition to the automatic expansion of the extent sizes, Oracle 11g also allows control over the allocation unit size using the
ATTRIBUTE clause in the
CREATE DISKGROUP statement, with values ranging from 1M to 64M.
CREATE DISKGROUP disk_group_2 EXTERNAL REDUNDANCY DISK '/dev/sde1' ATRRIBUTE 'au_size' = '32M';
The combination of expanding extent sizes and larger allocation units should result in increased I/O performance for very large databases.
New ASMCMD Commands and Options
The ASM command line utility includes several new commands. The following example output assumes you have already started the utility using the "-p" option.
$ asmcmd -p ASMCMD [+] >
The usage notes for each command is available by issuing the "
help <command>" command from within the
asmcmd utility, so I will avoid displaying all this information here.
lsdsk command lists information about ASM disks from the
V$ASM_DISK views. The summary usage is shown below.
lsdsk [-ksptcgHI] [-d <diskgroup_name>] [pattern]
An example of the output from the basic command and the "-k" option are shown below.
ASMCMD [+] > lsdsk Path /dev/sdc1 /dev/sdd1 /dev/sde1 ASMCMD [+] > lsdsk -d data -k Total_MB Free_MB OS_MB Name Failgroup Library Label UDID Product Redund Path 8189 6961 8189 DATA_0000 DATA_0000 System UNKNOWN /dev/sdc1 8189 6961 8189 DATA_0001 DATA_0001 System UNKNOWN /dev/sdd1 8189 6950 8189 DATA_0002 DATA_0002 System UNKNOWN /dev/sde1 ASMCMD [+] >
cp command allows files to be copied between ASM and local or remote destinations. The summary usage is shown below.
cp [-ifr] <[@connect_identifier:]src> <[@connect_identifier:]tgt>
The following example copies the current USERS datafile from ASM to the local file system.
ASMCMD [+] > cp +DATA/db11g/datafile/users.273.661514191 /tmp/users.dbf source +DATA/db11g/datafile/users.273.661514191 target /tmp/users.dbf copying file(s)... file, /tmp/users.dbf, copy committed. ASMCMD [+] >
There seems to be some concern that the
cp command doesn't work for control files (here).
md_backup command makes a copy of the metadata for one or more disk groups. The summary usage is shown below.
md_backup [-b location_of_backup] [-g dgname [-g dgname …]]
An example of the command is shown below. The resulting file contains all the metadata needed to recreate the ASM setup.
ASMCMD [+] > md_backup -b /tmp/backup.txt -g data Disk group to be backed up: DATA ASMCMD [+] >
md_restore command allows you to restore a disk group from the metadata created by the
md_backup command. It also allows a certain amount of manipulation of the final disk groups during the restore. The summary usage is shown below.
md_restore -b <backup_file> [-li] [-t (full)|nodg|newdg] [-f <sql_script_file>] [-g '<diskgroup_name>,<diskgroup_name>,...'] [-o '<old_diskgroup_name>:<new_diskgroup_name>,...']
A straight restore of the backup shown previously is shown below.
ASMCMD [+] > md_restore -b /tmp/backup.txt -t full -g data
remap command repairs a range of physical blocks on disk. The contents of each block is not validated, so only blocks exhibiting read errors are repaired. The summary usage is shown below.
remap <disk group name> <disk name> <block range>
An example of the command is show below.
ASMCMD [+] > remap data data_0001 5000-5999
For detailed usage information see the ASMCDM Command Reference.
Preferred Read Failure Groups
In Oracle 10g, ASM always reads the primary copy of the mirrored extent set. This isn't a problem when both nodes and both failure groups are all located in the same site, but it can be inefficient for extended clusters, causing needless network traffic. Oracle 11g allows each node to define a preferred failure group, allowing nodes in extended clusters to access local failure groups in preference to remote ones.
To configure preferred read failure groups the disk group compatibility attributes must be set to 11.1 or higher. Once the compatibility options are correct, the
ASM_PREFERRED_READ_FAILURE_GROUPS parameter is set to the preferred failure groups for each node.
SELECT name, failgroup FROM v$asm_disk; NAME FAILGROUP ------------------------------ ------------------------------ DATA_0000 DATA_0000 DATA_0001 DATA_0001 DATA_0002 DATA_0002 3 rows selected. SQL> ALTER SYSTEM SET ASM_PREFERRED_READ_FAILURE_GROUPS = 'data.data_0000', 'data.data_0001', 'data.data_0002';
ALTER DISKGROUP ... MOUNT" statement allows disk groups to be mounted in restricted mode.
SQL> SELECT name FROM v$asm_diskgroup; NAME ------------------------------ DATA SQL> ALTER DISKGROUP data DISMOUNT; Diskgroup altered. SQL> ALTER DISKGROUP data MOUNT RESTRICTED; Diskgroup altered. SQL> ALTER DISKGROUP data DISMOUNT; Diskgroup altered. SQL> ALTER DISKGROUP data MOUNT; Diskgroup altered. SQL>
In a RAC environment, a disk group mounted in
RESTRICTED mode can only be accessed by a single instance. The restricted disk group is not available to any ASM clients, even on the node where it is mounted.
RESTRICTED mode improves the performance of rebalance operations in a RAC environment as it elimitates the need for lock and unlock extent map messaging that occurs between ASM instances. Once the rebalance operation is complete, the disk group should be dismounted then mounted in
NORMAL mode (the default).
Miscellaneous Disk Group Maintenance Enhancements
ALTER DISKGROUPcommands include a new
ATTRIBUTEclause (compatible, disk_repair_time, au_size).
CHECKclause of the
ALTER DISKGROUPcommand has been simplified so there are only two options,
REPAIR, available, with
NOREPAIRas the default. Summarized errors are displayed, with full error messages writen to the alert log.
ALTER DISKGROUP data CHECK; -- Like NOREPAIR ALTER DISKGROUP data CHECK NOREPAIR; ALTER DISKGROUP data CHECK REPAIR;
- Disk groups can now be mounted in restricted mode, which can improve performance of some maintenance tasks.
ALTER DISKGOUPcommand now includes
OFFLINEclauses so disks can be taken offline for repair before being brought back online.
-- Individual disks. ALTER DISKGROUP data OFFLINE DISK 'disk_0000', 'disk_0001'; ALTER DISKGROUP data ONLINE DISK 'disk_0000', 'disk_0001'; -- Failure groups. ALTER DISKGROUP data OFFLINE DISKS IN FAILGROUP 'fg_0000'; ALTER DISKGROUP data ONLINE DISKS IN FAILGROUP 'fg_0000'; -- Bring online all disks in disk group. ALTER DISKGROUP data ONLINE ALL;
Disk groups that can't be mounted by ASM can now be dropped using the
FORCEkeyword of the
DROP DISKGROUP data FORCE;
For more information see:
- What's New in Oracle Database Storage Administration?
- Authentication for Accessing ASM Instances
- ASM Command-Line Utility
- ALTER DISKGROUP
- Automatic Storage Management (ASM) in Oracle Database 10g
Hope this helps. Regards Tim...