一.配置存储CRS DATA ARCH旧存储 /dev/sddlmaa /dev/sddlmai /dev/sddlmagOCR DATA ARCH新存储 /dev/sddlmab /dev/sddlmaj /dev/sddlmah/dev/sddlmad/dev/sddlmaf先配置好存储设备使操作系统能够看到新增加的硬盘。[rootracdb1 rules.d]# fdisk -l |grep /dev/sddlmDisk /dev/sddlmaa: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmab: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmac: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmad: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmae: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmaf: 2147 MB, 2147483648 bytes, 4194304 sectorsDisk /dev/sddlmag: 214.7 GB, 214748364800 bytes, 419430400 sectorsDisk /dev/sddlmah: 214.7 GB, 214748364800 bytes, 419430400 sectorsDisk /dev/sddlmai: 536.9 GB, 536870912000 bytes, 1048576000 sectorsDisk /dev/sddlmaj: 536.9 GB, 536870912000 bytes, 1048576000 sectors查看grid用户已经识别的硬盘[rootracdb1 rules.d]# ll /dev/sddlm* | grep gridbrw-rw---- 1 grid asmadmin 251, 0 Sep 16 15:46 /dev/sddlmaabrw-rw---- 1 grid asmadmin 251, 32 Sep 10 08:22 /dev/sddlmacbrw-rw---- 1 grid asmadmin 251, 64 Sep 10 08:22 /dev/sddlmaebrw-rw---- 1 grid asmadmin 251, 96 Sep 16 15:46 /dev/sddlmagbrw-rw---- 1 grid asmadmin 251, 128 Sep 16 15:46 /dev/sddlmai二、配置udev两个节点都需要操作1.修改规则文件[rootracdb1 ~]# cp -r /etc/udev/rules.d/99-oracle-asmdevices.rules /etc/udev/rules.d/99-oracle-asmdevices.rules.bak[rootracdb2 ~]# vim /etc/udev/rules.d/99-oracle-asmdevices.rulesSUBSYSTEMblock, KERNELsddlmaa, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmac, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmae, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmag, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmai, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmab, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmad, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmaf, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmah, GROUPasmadmin, OWNERgrid, MODE0660SUBSYSTEMblock, KERNELsddlmaj, GROUPasmadmin, OWNERgrid, MODE06602.加载规则文件加载规则文件如果规则文件修改后只需加载规则文件无需重新启动udev/sbin/udevadm control --reload-rules/sbin/udevadm trigger --typedevices --actionchange3.检查磁盘可用性[rootracdb1 ~]# ll /dev/sddlm* | grep gridbrw-rw---- 1 grid asmadmin 251, 0 Sep 20 10:06 /dev/sddlmaabrw-rw---- 1 grid asmadmin 251, 16 Sep 20 10:06 /dev/sddlmabbrw-rw---- 1 grid asmadmin 251, 32 Sep 20 10:06 /dev/sddlmacbrw-rw---- 1 grid asmadmin 251, 48 Sep 20 10:06 /dev/sddlmadbrw-rw---- 1 grid asmadmin 251, 64 Sep 20 10:06 /dev/sddlmaebrw-rw---- 1 grid asmadmin 251, 80 Sep 20 10:06 /dev/sddlmafbrw-rw---- 1 grid asmadmin 251, 96 Sep 20 10:06 /dev/sddlmagbrw-rw---- 1 grid asmadmin 251, 112 Sep 20 10:06 /dev/sddlmahbrw-rw---- 1 grid asmadmin 251, 128 Sep 20 10:06 /dev/sddlmaibrw-rw---- 1 grid asmadmin 251, 144 Sep 20 10:06 /dev/sddlmaj[gridracdb1 ~]$ asmcmd lsdsk/dev/sddlmaa/dev/sddlmag/dev/sddlmai[gridracdb1 ~]$ asmcmd lsdsk --candidate/dev/sddlmab/dev/sddlmac/dev/sddlmad/dev/sddlmae/dev/sddlmaf/dev/sddlmah/dev/sddlmajSQL sqlplus / as sysdbaset linesize 200 pages 1000;col name format a16;col path format a30;col HEADER_STATUS format a16;select name,path,HEADER_STATUS,TOTAL_MB from v$asm_disk;NAME PATH HEADER_STATUS TOTAL_MB---------------- ------------------------------ ---------------- ----------/dev/sddlmae CANDIDATE 0/dev/sddlmac CANDIDATE 0/dev/sddlmaf CANDIDATE 0/dev/sddlmad CANDIDATE 0/dev/sddlmab CANDIDATE 0/dev/sddlmah CANDIDATE 0/dev/sddlmaj CANDIDATE 0CRS_0000 /dev/sddlmaa MEMBER 2048DATA_0000 /dev/sddlmai MEMBER 512000ARCH_0000 /dev/sddlmag MEMBER 20480010 rows selected.三、创建新的ocr磁盘组orcsqlplus / as sysasmcol name form a20col COMPATIBILITY form a20col DATABASE_COMPATIBILITY form a20select name,state,total_mb,free_mb,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;NAME COMPATIBILITY DATABASE_COMPATIBILI-------------------- -------------------- --------------------ARCH 11.2.0.0.0 10.1.0.0.0CRS 11.2.0.0.0 10.1.0.0.0DATA 11.2.0.0.0 10.1.0.0.0###创建磁盘组compatible与之前查询出来的结果保持一致。SQL create diskgroup ocr external redundancy disk /dev/sddlmab, /dev/sddlmad,/dev/sddlmaf attribute compatible.rdbms10.1.0.0.0,compatible.asm11.2.0.0.0;Diskgroup created.四、备份原ocr和votedisk[rootracdb1 ~]# mkdir -p /backup/20220909[rootracdb1 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -export /backup/20220909/ocr_20220909.expdd if/dev/sddlmaa of/backup/20220909/votedisk_20220909_ocr1.bak五、迁移OCR磁盘组1.查看原OCR为CRS[gridracdb1 ~]$ ocrcheckStatus of Oracle Cluster Registry is as follows :Version : 3Total space (kbytes) : 262120Used space (kbytes) : 2892Available space (kbytes) : 259228ID : 696587786Device/File Name : CRS2.另一个需要先挂载OCR磁盘组SQL alter diskgroup OCR mount;Diskgroup altered.3.添加为OCR添加磁盘组OCR[rootracdb1 ~]$ /u01/app/11.2.0/grid/bin/ocrconfig -add OCR4.再次查看OCR已经多了一个磁盘组OCR了。[gridracdb1 ~]$ ocrcheckStatus of Oracle Cluster Registry is as follows :Version : 3Total space (kbytes) : 262120Used space (kbytes) : 2888Available space (kbytes) : 259232ID : 696587786Device/File Name : CRSDevice/File integrity check succeededDevice/File Name : OCRDevice/File integrity check succeededDevice/File not configuredDevice/File not configuredDevice/File not configuredCluster registry integrity check succeededLogical corruption check bypassed due to non-privileged user[gridracdb1 ~]$ more /etc/oracle/ocr.loc#Device/file getting replaced by device OCRocrconfig_locCRSocrmirrorconfig_locOCRlocal_onlyfalse5.替换OCR[rootracdb1 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -replace CRS -replacement OCRPROT-29: The Oracle Cluster Registry location is already configured6.删除原来的OCR:CRS磁盘组。/u01/app/11.2.0/grid/bin/ocrconfig -delete CRS7.再次查看ocr只看到磁盘组OCR了。[gridracdb1 ~]$ ocrcheckStatus of Oracle Cluster Registry is as follows :Version : 3Total space (kbytes) : 262120Used space (kbytes) : 2888Available space (kbytes) : 259232ID : 696587786Device/File Name : OCR创建pfilecreate pfile‘/u01/app/11.2.0/grid/dbs/asmpfile.ora’ from spfile;create spfile‘OCR’ from pfile‘/u01/app/11.2.0/grid/dbs/asmpfile.ora’;六、迁移votedisk[gridracdb1 ~]$ crsctl query css votedisk## STATE File Universal Id File Name Disk group-- ----- ----------------- --------- ---------1. ONLINE 217fca4d97bc4fe7bffafa8f995201e3 (/dev/sddlmaa) [CRS]Located 1 voting disk(s).[gridracdb1 ~]$ crsctl replace votedisk OCRSuccessful addition of voting disk 2ebcd42944624f57bfe06f96692b66fa.Successful deletion of voting disk 217fca4d97bc4fe7bffafa8f995201e3.Successfully replaced voting disk group with OCR.CRS-4266: Voting file(s) successfully replaced[gridracdb1 ~]$ crsctl query css votedisk## STATE File Universal Id File Name Disk group-- ----- ----------------- --------- ---------1. ONLINE 2ebcd42944624f57bfe06f96692b66fa (/dev/sddlmab) [OCR]Located 1 voting disk(s).六、删除原来的OCR和votediskCRS磁盘组。col c form a30set linesize 200col name form a20col COMPATIBILITY form a20col DATABASE_COMPATIBILITY form a20select inst_id,name|| ||path c,HEADER_STATUS,mount_status from gv$asm_disk order by 1,2;select name,state,total_mb,free_mb,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;停止crs/u01/app/11.2.0/grid/bin/crsctl stop crs启动crs/u01/app/11.2.0/grid/bin/crsctl start crs/u01/app/11.2.0/grid/bin/crsctl stat res -t卸载SQL alter diskgroup CRS dismount;另一节点停止crs/u01/app/11.2.0/grid/bin/crsctl stop crs启动crs/u01/app/11.2.0/grid/bin/crsctl start crs/u01/app/11.2.0/grid/bin/crsctl stat res -t删除磁盘组SQL drop diskgroup ocr including contents;Diskgroup dropped.七、迁移数据库存储[gridracdb1 ~]$ sqlplus / as sysasmalter diskgroup data add disk /dev/sddlmaj drop disk DATA_0000 rebalance power 10;##ASM_POWER_LIMIT范围是0~110表示不进行rebalance值越大 rebalance速度越快。##该命令相当于下面2条命令##alter diskgroup DATA add disk /dev/sddlmaj rebalance power 10;##alter diskgroup DATA drop disk DATA_0000 rebalance power 10;select sofar,EST_MINUTES from gv$asm_operation;##如果没有数据代表sam磁盘平衡操作完成。alter diskgroup ARCH add disk /dev/sddlmah drop disk ARCH_0000 rebalance power 10;select sofar,EST_MINUTES from gv$asm_operation;##如果没有数据代表sam磁盘平衡操作完成。八、清除udev规则文件多余部分。查询目前数据库用到的asm磁盘信息col c form a30set linesize 200col name form a20col COMPATIBILITY form a20col DATABASE_COMPATIBILITY form a20select inst_id,name|| ||path c,HEADER_STATUS,mount_status from gv$asm_disk order by 1,2;select name,state,total_mb,free_mb,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;在节点1删除多余的asm磁盘信息。cp /etc/udev/rules.d/99-oracle-asmdevices.rules /etc/udev/rules.d/99-oracle-asmdevices.rules.bak2vim /etc/udev/rules.d/99-oracle-asmdevices.rules加载规则文件如果规则文件修改后只需加载规则文件无需重新启动udev/sbin/udevadm control --reload-rules/sbin/udevadm trigger --typedevices --actionchange