12.1 ACFS installation and usage

Check OS and CRS version

$GRID_HOME/OPatch/opatch lsinventory
Oracle Home       : /u01/app/121/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/121/grid/oraInst.loc
OPatch version    : 12.1.0.1.0
OUI version       : 12.1.0.1.0
Log file location : /u01/app/121/grid/cfgtoollogs/opatch/opatch2014-06-25_14-50-13PM_1.log

Lsinventory Output file location : /u01/app/121/grid/cfgtoollogs/opatch/lsinv/lsinventory2014-06-25_14-50-13PM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (1): 
Oracle Grid Infrastructure 12c                                       12.1.0.1.0
Patch level status of Cluster nodes :
 Patching Level              Nodes
 --------------              -----
 0                           gract3,gract2,gract1
# uname -a
Linux gract1.example.com 2.6.39-400.109.6.el6uek.x86_64 #1 SMP Wed Aug 28 09:56:40 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/oracle-release
Oracle Linux Server release 6.4

Check whether related ACFS modules are already loaded
# lsmod | grep ora
oracleacfs           3053229  0 
oracleadvm            320180  2 
oracleoks             417171  2 oracleacfs,oracleadvm
oracleasm              53865  1 

You may check whether your kernel version is supported by reading
ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

Create a new ACFS diskgroup

Use asmca and check driver status 
--> ADVM Driver status sould be : Loaded 

Create ASM diskgroup with HIGH redundany
Make 3 ASM disks available at OS using UDEV rules - Reboot 
All custer nodes should show the same results
# ls -l  /dev/acfs*
brw-rw----. 1 grid asmadmin 8,  81 Jun 26 08:12 /dev/acfsdisk1_1G
brw-rw----. 1 grid asmadmin 8,  97 Jun 26 08:12 /dev/acfsdisk2_1G
brw-rw----. 1 grid asmadmin 8, 113 Jun 26 08:12 /dev/acfsdisk3_1G

Invoke acma and create a diskgroup with following paramters 
Change your disk discorvery path to : /dev/asm*,/dev/acfs*
Redundancy : HIGH
Disk PATH:
/dev/acfsdisk1_1G  CANDIDATE
/dev/acfsdisk2_1G  CANDIDATE
/dev/acfsdisk3_1G  CANDIDATE

ASM       Compatibility : 12.1.0.0.0
Show Advanced Options and set 
   ASM       Compatibility : 12.1.0.0.0
   Database  Compatibility : 12.1.0.0.0
   ADVM      Compatibility : 12.1.0.0.0

Verify diskgroup status ( should be mounted on all nodes )

[grid@gract1 ~]$  asmcmd lsdg -g ACFS_DG1
Inst_ID  State    Type  Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
      1  MOUNTED  HIGH  N         512   4096  1048576      3057     2643                0             881              0             N  ACFS_DG1/
      3  MOUNTED  HIGH  N         512   4096  1048576      3057     2643                0             881              0             N  ACFS_DG1/
      2  MOUNTED  HIGH  N         512   4096  1048576      3057     2643                0             881              0             N  ACFS_DG1/

[grid@gract1 ~]$  asmcmd lsdsk -g
Inst_ID  Path
      1  /dev/acfsdisk1_1G
      3  /dev/acfsdisk1_1G
      2  /dev/acfsdisk1_1G
      1  /dev/acfsdisk2_1G
      3  /dev/acfsdisk2_1G
      2  /dev/acfsdisk2_1G
      1  /dev/acfsdisk3_1G
      3  /dev/acfsdisk3_1G
      2  /dev/acfsdisk3_1G

Create and display the volume status 
grid@gract1 ~]$  asmcmd  volcreate -G ACFS_DG1 -s 800m acfs_vol1

[grid@gract1 ~]$ asmcmd volinfo -G ACFS_DG1 acfs_vol1
Diskgroup Name: ACFS_DG1
     Volume Name: ACFS_VOL1
     Volume Device: /dev/asm/acfs_vol1-443
     State: ENABLED
     Size (MB): 800
     Resize Unit (MB): 32
     Redundancy: HIGH
     Stripe Columns: 4
     Stripe Width (K): 128
     Usage: 
     Mountpath: 

Create an ACFS filesystem mkfs -t acfs /dev/asm/acfs_vol1-443
mkfs.acfs: version                   = 12.1.0.1.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfs_vol1-443
mkfs.acfs: volume size               = 838860800
mkfs.acfs: Format complete.

Register a mount point

[root@gract1 rules.d]#  mkdir -p /u01/acfs/acfs-vol1 
[root@gract1 rules.d]# /sbin/acfsutil registry -a  /dev/asm/acfs_vol1-443 /u01/acfs/acfs-vol1
acfsutil registry: mount point /u01/acfs/acfs-vol1 successfully added to Oracle Registry

Mount and dismount an ACFS filesystem

Locate any ACFS mounted filesystem 
[root@gract1 Desktop]#  mount -t acfs
/dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw)

Dismount an ACFS filesystem 
[root@gract1 Desktop]# umount /u01/acfs/acfs-vol1 
[root@gract1 Desktop]# mount -t acfs

Find ACFS mountpoints and mount ACFS fs
[root@gract1 Desktop]#   /sbin/acfsutil registry -l
Device : /dev/asm/acfs_vol1-443 : Mount Point : /u01/acfs/acfs-vol1 : Options : none : Nodes : all : Disk Group : ACFS_DG1 : Volume : ACFS_VOL1

[root@gract1 Desktop]# mount -t acfs /dev/asm/acfs_vol1-443  /u01/acfs/acfs-vol1
[root@gract1 Desktop]# mount -t acfs
/dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw)

Use lsof to investigate umount problem : device busy

[root@gract1 Desktop]#  umount /u01/acfs/acfs-vol1 
umount: /u01/acfs/acfs-vol1: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))
umount: /u01/acfs/acfs-vol1: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))
[root@gract1 Desktop]# lsof | grep  /u01/acfs/acfs-vol1
vi        11553      root  cwd       DIR         251,226817      4096     2  /u01/acfs/acfs-vol1
vi        11553      root    3u      REG         251,226817     12222     44 /u01/acfs/acfs-vol1/.test.swp
bash      29897      root  cwd       DIR         251,226817      4096      2 /u01/acfs/acfs-vol1
--> root run bash shell and this shell has open directory /u01/acfs/acfs-vol1 
    root also runs vi from the same direactory and vi has opene a .swp file 

After stopping or killing these processes and the dismount should go through
[root@gract1 Desktop]# lsof | grep  /u01/acfs/acfs-vol1
[root@gract1 Desktop]# umount /u01/acfs/acfs-vol1 

Verify  ACFS setup

# mount
/dev/asm/acfs_vol1-443 on /u01/acfs/acfs-vol1 type acfs (rw)
Commands to verify your setup

[root@gract1 rules.d]# lsmod | grep ora
oracleacfs           3053229  2
oracleadvm            320180  8
oracleoks             417171  2 oracleacfs,oracleadvm
oracleasm              53865  1

$ asmcmd volinfo -G ACFS_DG1 acfs_vol1
Diskgroup Name: ACFS_DG1
Volume Name: ACFS_VOL1
Volume Device: /dev/asm/acfs_vol1-443
State: ENABLED
Size (MB): 800
Resize Unit (MB): 32
Redundancy: HIGH
Stripe Columns: 4
Stripe Width (K): 128
Usage: ACFS
Mountpath: /u01/acfs/acfs-vol1

[root@gract1 rules.d]#  /sbin/acfsutil registry -l
Device : /dev/asm/acfs_vol1-443 : Mount Point : /u01/acfs/acfs-vol1 : Options : none : Nodes : all : Disk Group : ACFS_DG1 : Volume : ACFS_VOL1
help:  help

[root@gract1 rules.d]# asmcmd lsdg -g
Inst_ID  State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
1  MOUNTED  HIGH    N         512   4096  1048576      3057      159                0              53              0             N  ACFS_DG1/
3  MOUNTED  HIGH    N         512   4096  1048576      3057      159                0              53              0             N  ACFS_DG1/
2  MOUNTED  HIGH    N         512   4096  1048576      3057      159                0              53              0             N  ACFS_DG1/

[root@gract1 rules.d]# df -k
Filesystem                1K-blocks      Used Available Use% Mounted on
/dev/asm/acfs_vol1-443       819200    118852    700348  15% /u01/acfs/acfs-vol1

Reference

Leave a Reply

Your email address will not be published. Required fields are marked *