How to Add a New VG to an Active HACMP Resource Group

Contents

1. Introduction
2. Identify New Disks
3. Create a Concurrent-Capable VG
4. Create the Filesystem(s)
5. Import on the Other Node
6. Add the VG to the HACMP Resource Group
7. Final Synchronization of the Cluster

1. Related Information

1. Introduction

This article describes how to add a new volume group to an existing Resource Group on an active HACMP cluster.

2. Identify New Disks

First we need to scan for new disks:

nodeA# cfgmgr

and set a PVID to all new disks:

nodeA# lspv | awk '$3 ~ /^None$/ {print "chdev -l "$1" -a pv=yes"}' | sh

Finally we change the reservation policy for use with HACMP:

nodeA# lspv | awk '$3 ~ /^None$/ {print "chdev -l "$1" -a reserve_policy=no_reserve"}' | sh

The two commands above change all disks not belonging to a volume group. If your cluster owns disks for heartbeat or raw volumes not belonging to a volume group you cannot use these one-liners.

Since the reserve policy is stored in the ODM and not on the disk we have to set it on the other node as well:

nodeB# cfgmgr

nodeB# lspv | awk '$3 ~ /^None$/ {print "chdev -l "$1" -a reserve_policy=no_reserve"}' | sh

For correct mirroring we also need to identify the origin of the disks we found. In our example we find these disks:

·  from DatacenterA: hdisk56, hdisk57, hdisk58, hdisk59

·  from DatacenterB: hdisk60, hdisk61, hdisk62, hdisk63

3. Create a Concurrent-Capable VG

Before we actually create the new VG we need to find a major number that is available on both nodes¹.

nodeB#  lvlstmajor

44..97,108,109,111..118,128..138,148..158,168..178,188,189,192..198,203...

The command lvlstmajor lists all free major numbers on a system. We run the command on both nodes and pick a number that is free all nodes. In our example we pick 203. We use this number when we create the new VG called datavg05:

nodeA# mkvg -C -V 203 -S -s 32 -y datavg05 hdisk56 hdisk57 hdisk58 hdisk59 hdisk60 hdisk61 hdisk62 hdisk63

nodeA# varyonvg datavg05

Please note: The volume groups we create are enhanced-concurrent-capable. By default these volume groups have the following properties: Quorum is on, auto-varyon is off. However, quorum should be set to "off" in a two datacenter setup:

nodeA# chvg -Qn datavg05

0516-1804 chvg: The quorum change takes effect immediately.

---

¹ The same major number for the volume groups' device files is only needed if your cluster acts as a nfs server. However, it's good practice to keep the volume groups' major number in sync across all nodes.

4. Create the Filesystem(s)

With the volume groups created and vary'd on we can go on creating logical volumes. Since we want to mirror our filesystems between the two datacenters we need to ensure that all first LV copies (PV1 in »lslv -m« output) reside in DatacenterA and all second LV copies (PV2 in »lslv -m« output) reside in DatacenterB. Therefore we first create a non-mirrored minimal LV lvdata05 over the disks from DatacenterA:

nodeA# mklv -e x -u 4 -s s -t jfs2 -y lvdata05 datavg05 4 hdisk56 hdisk57 hdisk58 hdisk59

We check:

nodeA# lslv -m lvdata05

lvdata05:N/A

LP    PP1  PV1               PP2  PV2               PP3  PV3

0001  0206 hdisk56          

0002  0206 hdisk57          

0003  0206 hdisk58         

0004  0206 hdisk59         

Now we can add the mirror to the LV and create a filesystem on top of it:

nodeA# mklvcopy -e x lvdata05 2 hdisk60 hdisk61 hdisk62 hdisk63

nodeA# varyonvg datavg05          # this synchronizes the mirror

nodeA# crfs -v jfs2 -m /path/to/data05 -d /dev/lvdata05 -A no -p rw -a logname=INLINE

nodeA# mount /path/to/data05

nodeA# chown user:group /path/to/data05

We check our mirror:

nodeA# lslv -m lvdata05

lvdata05:N/A

LP    PP1  PV1               PP2  PV2               PP3  PV3

0001  0206 hdisk56           0206 hdisk60

0002  0206 hdisk57           0206 hdisk61

0003  0206 hdisk58           0206 hdisk62

0004  0206 hdisk59           0206 hdisk63

Since we set Superstrictness and an Upperbound of 4 we can now increase the filesystem to the final size:

nodeA# chlv -x 16384²

nodeA# chfs -a size=512G /path/to/data05

Once all filesystems are created we unmount everything we just created and close the volume group:

nodeA# umount /path/to/data05

nodeA# varyoffvg datavg05

---

² The default value for the maximum LPs per LV is 512 when you create a small LV. Thus changing the maximal number of LPs for a LV is needed before we can extend the filesystem.

5. Import on the Other Node

We need one PVID of VG (from the first node):

nodeA# lspv | grep -w datavg05 | head -1

hdisk56         00f6b4e4935bcb40                    datavg05

On the other node we need to run the configuration manager to scan for disks and to set the reserve_policy to no_reserve:

nodeB# cfgmgr

nodeB# lspv | awk '$3 ~ /^None$/ {print "chdev -l "$1" -a reserve_policy=no_reserve"}' | sh

Now we are ready to import the new Volume Groups into the other node.

nodeB# importvg -n -V 203 -y datavg05 00f6b4e4935bcb40

6. Add the VG to the HACMP Resource Group

Before we start to add the new VG to our resource group it might be a good idea to check the cluster configuration first:

nodeA# smitty hacmp

-> Problem Determination Tools

   -> HACMP Verification

      -> Verify HACMP Configuration

Correct any errors *before* changing resource groups!

Now it's time to let HACMP discover information:

nodeA# smitty hacmp

-> Extended Configuration

   -> Discover HACMP-related Information from Configured Nodes

Finally we add the new VG to the HACMP resource group (RG01 in our example):

nodeA# smitty hacmp

-> Extended Configuration

   -> Extended Resource Configuration

      -> HACMP Extended Resource Group Configuration

         -> Change/Show Resources and Attributes for a Resource Group

select RG01 and add datavg05 to the list of volume groups:

  ________________________________________________________________________________________________________________

  Service IP Labels/Addresses                        [haservice1]                                                +

  Application Servers                                [APP_SRV1]                                                  +

  Volume Groups                                      [datavg01 datavg02 datavg03 datavg04 datavg05 ]             +

  Use forced varyon of volume groups, if necessary    true                                                       +

  Automatically Import Volume Groups                  false                                                      +

  Filesystems (empty is ALL for VGs specified)       [ ]                                                         +

  Filesystems Consistency Check                       logredo                                                    +

  Filesystems Recovery Method                         sequential                                                 +

  ________________________________________________________________________________________________________________

7. Final Synchronization of the Cluster

Now synchronize the cluster...

nodeA# smitty hacmp

   -> Extended Configuration

     -> Extended Verification and Synchronization

This will varyon the VG and mount the filesystem.