Tanti Technology

My photo
Bangalore, karnataka, India
Multi-platform UNIX systems consultant and administrator in mutualized and virtualized environments I have 4.5+ years experience in AIX system Administration field. This site will be helpful for system administrator in their day to day activities.Your comments on posts are welcome.This blog is all about IBM AIX Unix flavour. This blog will be used by System admins who will be using AIX in their work life. It can also be used for those newbies who want to get certifications in AIX Administration. This blog will be updated frequently to help the system admins and other new learners. DISCLAIMER: Please note that blog owner takes no responsibility of any kind for any type of data loss or damage by trying any of the command/method mentioned in this blog. You may use the commands/method/scripts on your own responsibility. If you find something useful, a comment would be appreciated to let other viewers also know that the solution/method work(ed) for you.

Friday 13 December 2013

AIX TECHNOLOGY LEVEL(TL) UPDATE METHODS

As an AIX system administrator, you know the importance of keeping the systems up to date. Unfortunately, this is always a little tricky and, sometimes, due to the size of the environments, very hard to accomplish. This article shows the many ways available to update an AIX server.

Introduction

Staying current with the latest AIX Technology Level (TL) is always the best option to better availability, reliability and security. TL is a set of fixes, and new features added to an AIX version or new hardware support.

You should considered moving to a new TL version for the following reasons:

  • A new function provided in a new TL is needed.
  • If the existing TL is out or is about to go out of new fixes and service packs.
  • The system currently has a need for a fix, which is present on the new TL.
This article goes through the multiple ways of doing a TL update and the fallback options, as well. It shows the approved and supported TL upgrades methods provided by IBM.

Before you begin

TLs must be applied as a group. Installing a TL is an "all or nothing" operation. Installing a partial TL is not recognized from a support standpoint.

Before applying a TL, always create a backup of you current installation, and plan on a worst case scenario—on restoring that backup if needed to rollback the system to the previous level.

Also, TL updates should always be committed because they cannot be rejected. If a TL has been applied and needs to go back to the previous version, then a fallback plan is needed.

The general rule of thumb is to always create a backup before beginning. It can be any kind of image backup (mksysb image, a sysback image, etc).

After a valid backup image is available, the upgrade process can be started.

Also, it is a good practice to create a health checklist, that is, save as much as information from the system (netstat, ifconfig, lsvg, lsdev, lscfg, prtconf, etc.) and keep it somewhere other than the server that is being upgraded. Keep in mind that this information is only going to be used as support material in case of problems.

If the TL version to be installed is previous to AIX 5.3 TL10 and AIX 6.1 TL3, make sure all interim fixes (ifix) have been removed from the system. Listing 1 shows how to check for installed ifixes, and listing 2 shows how to uninstall an ifix:
Listing 1. Checking for installed ifixes
# emgr –l
Listing 2. Removing an ifix
# emgr –r –L 
For further information on how to use the emgr command, check IBM documentation or the man pages.

If you are updating to AIX 5.3 TL10 or AIX 6.1 TL3, these steps do not need to be executed since installp and emgr have been enhanced to automatically remove the ifix when present in the TL. Otherwise, the ifixes will have to be manually removed.

Check if all filesets are applied and are valid, as shown in Listing 3.
Listing 3. Checking installed filesets consistency
# instfix –i | grep ML
# lppchk –v
If problems are reported from running these commands, stop now and fix the problems. Applying a TL to an inconsistent AIX will not only damage the operating system, but it can also make it unbootable.

Next, all filesets in the APPLIED state needs to be COMMITTED. To commit them, follow the example in Listing 4.
Listing 4. Committing all APPLIED filesets
# installp -c -f -g –X
Or, use smitty commit and follow the instructions on the screen.

Here are most common methods that are used in AIX TL up gradations.

  1. Alternate Disk
  2. Multibos
  3. Same Disk
  4. Using NIM
Going forward you will learn about each method in detail.

Alternate disk

This is one of the most used practices to apply a new TL. It is referenced in IBM technical documents and books. Using an alternate disk gives the following options:

  1. Using a secondary disk and apply the TL without disruption, only rebooting after the TL upgrade is done—IBM recommends that all processes be terminated and users to be logged off (this procedure is covered in this article).
  2. Rebooting to a secondary disk and running the TL upgrade to what was the primary disk—some system administrator often use this to make sure the system is running on a consistent state and has no problems or malfunctions after a reboot is performed.
Either way, a free disk is needed to install using alternate disk.

For this alternate disk update method, the server to be updated has a rootvg with two mirrored disks. Therefore,unmirror it before beginning and use the second disk for the alternate disk installation, as shown in Listing 5. 
Listing 5. Unmirroring rootvg
# unmirrorvg rootvg
If a secondary dump device is configured to use the secondary disk, move its LPs to the remaining disk usingmigratepv or unconfigure the secondary dump device.

Remove the disk from the rootvg (assuming that X is the disk device number), as seen on Listing 6.
Listing 6. Removing disk from the rootvg
# reducevg rootvg hdiskX
Now create the alternate disk and apply the TL update to it. The TL files can be placed locally, as covered in this article, in a remote NFS share or in a CD-ROM.

Smitty fastpath smitty alt_clonecan be used, or the alt_disk_copy command line. Figure 1 shows the initial smitty screen.
Figure 1. Initial smitty alt_clone screen
image001

Remember, if you are unsure about a field, pressing F1 provides help.

Moving forward, use hdisk1 and the TL files are locally under the /stage_TL filesystem, as shown in Figure 2.
Figure 2. Added hdisk1, bundle to install, directory with images and the acceptance of license agreements
image002

All operations will be logged to /var/adm/ras/alt_disk_inst.log. To watch its progress, enter tail –f to it.

The server will need to be rebooted after the update process, so make sure the bootlist is showing the target alternate disk as the first boot device (as seen in Listing 7).
Example 7. Checking boot device order
# bootlist –m normal –o
hdisk1 blv=hd5
Once the server has been rebooted, issue oslevel –s or oslevel –r and check if the OS level is now at the updated TL, as demonstrated in Listing 8.
Listing 8. Check running AIX version
# oslevel –s
5300-10-01-0921
If the update is considered successful, the rootvg can be mirrored again. Listing 9 shows how to mirror the rootvg again.
Listing 9. Mirroring back rootvg
# exportvg old_rootvg
# extendvg –f rootvg hdisk0
# mirrorvg rootvg
Create a new boot image on hdisk0 and add it to the boot list, as seen in Listing 10.
Listing 10. Creating a boot image
# bosboot –ad /dev/hdisk0
# bootlist –m normal –o hdisk0 hdisk1

Multibos

This is by far the coolest way to have AIX upgraded. It was introduced with AIX 5.3 TL3. This is great in cases where only one disk is available on rootvg and no free disks for alternate disks are available.

Multibos creates and maintain two different and bootable AIX instances within the same rootvg. It is similar to alternate disk. In this case, the biggest difference is that multibos will create and copy only the following Logical Volumes (LVs):

  • /;
  • /usr;
  • /var;
  • /opt, and;
  • hd5 (Boot logical volume).
All others LVs will be shared with the original root volume group, although more logical volumes can be specified and copied. Multibos supports a new TL update, but AIX version upgrades are not supported by multibos.

In addition to the tasks mentioned in the Before you begin section, make sure that enough free disk space to copy each BOS logical volume to the same root volume group disk is available, otherwise multibos will not work.

Create a new standby BOS instance by running the multibos command. Check its options and documentation before you begin. Listing 11 shows how to create a new standby BOS instance.
Listing 11. Preview of multibos standby BOS creation
# multibos –sXp
This shows a preview of what multibos is about to execute. For further information always check its log file (/etc/multibos/logs/op.alog). If everything seems to be OK with the preview, execute it again without the preview flag (-p) as shown in Listing 12. 
Example 12. Multibos standby BOS creation
# multibos –sX
It will take a few minutes to copy all the contents, and after it’s completed all new LVs will be prefixed by "bos_". Listing 13 shows how the rootvg will look like after the new standby BOS has been created.
Listing 13. Multibos standby BOS created
# lsvg –l rootvg
rootvg:
LV NAME             TYPE       LPs   PPs   PVs  LV STATE      MOUNT POINT
hd5                 boot       1     1     1    closed/syncd  N/A
hd6                 paging     192   192   1    open/syncd    N/A
hd8                 jfs2log    1     1     1    open/syncd    N/A
hd4                 jfs2       1     1     1    open/syncd    /
hd2                 jfs2       17    17    1    open/syncd    /usr
hd9var              jfs2       8     8     1    open/syncd    /var
hd3                 jfs2       4     4     1    open/syncd    /tmp
hd1                 jfs2       1     1     1    open/syncd    /home
hd10opt             jfs2       1     1     1    open/syncd    /opt
lvdump1             sysdump    22    22    1    open/syncd    N/A
lvdump2             sysdump    22    22    1    open/syncd    N/A
bos_hd5             boot       1     1     1    closed/syncd  N/A
bos_hd4             jfs2       1     1     1    open/syncd    /bos_inst
bos_hd2             jfs2       17    17    1    open/syncd    /bos_inst/usr
bos_hd9var          jfs2       8     8     1    open/syncd    /bos_inst/var
bos_hd10opt         jfs2       1     1     1    open/syncd    /bos_inst/opt
It is a good idea to enter the newly created BOS instance shell and verify its current TL, as shown in Listing 14. To exit from the multibos environment just type 'exit':
Listing 14. Entering the multibos shell and checking AIX version
# multibos –S
Initializing multibos methods ...
Initializing log /etc/multibos/logs/op.alog ...
Gathering system information ...

+-----------------------------------------------------------------------------+
Multibos Shell Operation
+-----------------------------------------------------------------------------+
Verifying operation parameters ...

+-----------------------------------------------------------------------------+
Mount Processing
+-----------------------------------------------------------------------------+
Mounting all standby BOS file systems ...
Mounting /bos_inst
Mounting /bos_inst/usr
Mounting /bos_inst/var
Mounting /bos_inst/opt

+-----------------------------------------------------------------------------+
Multibos Root Shell
+-----------------------------------------------------------------------------+
Starting multibos root shell ...
Active boot logical volume is hd5.
Script command is started. The file is /etc/multibos/logs/scriptlog.090904032855.txt.
MULTIBOS> oslevel –s
5300-06-08-0831
If all prerequisite tasks have been completed (see Before you begin section), the TL update can be started. The command used in the Listing 15 will update your newly created standby BOS instance.
Listing 15. Updating the standby BOS instance
# multibos –Xac –l /stage_TL
After the command is completed, enter it again on the multibos shell and check the current TL, as seen in Listing 16.
Listing 16. Entering multibos shell
# multibos –S
Initializing multibos methods ...
Initializing log /etc/multibos/logs/op.alog ...
Gathering system information ...

+-----------------------------------------------------------------------------+
Multibos Shell Operation
+-----------------------------------------------------------------------------+
Verifying operation parameters ...

+-----------------------------------------------------------------------------+
Mount Processing
+-----------------------------------------------------------------------------+
Mounting all standby BOS file systems ...
Mounting /bos_inst
Mounting /bos_inst/usr
Mounting /bos_inst/var
Mounting /bos_inst/opt

+-----------------------------------------------------------------------------+
Multibos Root Shell
+-----------------------------------------------------------------------------+
Starting multibos root shell ...
Active boot logical volume is hd5.
Script command is started. The file is /etc/multibos/logs/scriptlog.090904035718.txt.
MULTIBOS> oslevel –s
5300-10-01-0921
Configure and ensure the boot list is pointing to the standby BOS as the first boot device, as shown in Listing 17.
Listing 17. Setting up boot to your new standby BOS instance
# multibos –B

# bootlist –m normal –o 
hdisk0 blv=bos_hd5
hdisk0 blv=hd5
If the update procedure failed and a fallback is needed, set and verify the boot list back to the previous boot LV and reboot—this will bring back the older AIX version. This procedure is shown in Listing 18.
Listing 18. Changing the boot device back to your original rootvg
# bootlist –m normal –o hdisk0 blv=hd5 hdisk0 blv=bos_hd5
# bootlist –m normal –o
hdisk0 blv=hd5
hdisk0 blv=bos_hd5
But, if no problems were found and the standby BOS is not necessary any longer, it can be removed by issuing the command shown in Listing 19.
Listing 19. Removing the old rootvg
# multibos -R

Same disk

This is the simplest method available. The downside of this method is that two reboots are needed in case of a fallback.

In this example, a backup to an alternate disk will be done before the update process. So, jump to smitty alt_clone again, as shown in the Figure 3 and select the desired backup disk and hit enter with the default values.
Figure 3. smitty alt_clone menu
image003

The command line can also be used, as shown in Listing 20.
Listing 20. Command-line for cloning rootvg
 # alt_disk_copy -P "all" -d "hdisk1" -B
To follow the progress of the alternate task look at the alternate disk log file, /var/adm/ras/ alt_disk_inst.log.

After the alternate disk is done, the update process can be performed. Use the smitty fastpath smitty update_all or from the command line use install_all_updates. Listing 21 shows the update process.

Enter the directory containing the TL filesets:
Listing 21. Same disk update process
>
 # cd /stage_TL

And create a Table of Contents file (ToC):

 # inutoc .
While still inside the directory containing the filesets, run our smitty update_all command as shown in Figure 4.
Figure 4: smitty update_all initial screen
image004

The first screen will ask where the filesets are, add a “.” (dot) and press enter.
Figure 5 shows how the smitty menu will look like.
Figure 5. smitty update_all menu with options
image005

After the update process is done, reboot the server.

Once the server has been rebooted, issue oslevel –s or oslevel –r and check if the OS level is now at the TL level that was applied as observed in Listing 22.
Listing 22: Checking AIX version after update
 # oslevel –s
 5300-10-01-0921
If the update was considered successfully the rootvg can be mirrored again. Follow the example in Listing 23 to re-mirror the rootvg and create a boot image on hdisk1 and add it to the boot list.
Listing 23. Re-mirroring the rootvg
 # exportvg alt_inst_rootvg
 # extendvg –f rootvg hdisk1
 # mirrorvg rootvg
 # bosboot –ad /dev/hdisk1
 # bootlist –m normal –o hdisk0 hdisk1

Using a NIM Server

This is also one of most popular ways of updating an AIX server. However, in this case an up and running NIM Server is needed.

This article does not intend to show how to configure a NIM server or its pieces: spots, lpp_sources, machines, etc.

As was done for the previous methods, refer to the Before you begin section to review if all requirements were met. If yes, follow the example in Figure 6 to get started:
Figure 6. Entering smitty nim menu
image006

This will enter the main NIM server smitty menu as shown in Figure 7. On the first screen select “Perform NIM Software Installation and Maintenance Tasks”
Figure 7. Main NIM server smitty menu
image007

Next, select “Alternate Disk Installation”, as shown in Figure 8.
Figure 8. Select "Alternate Disk Installation"
image008

As "Alternate Disk Installation" option is selected chose "Clone rootvg to an Alternate Disk".
Figure 9. Select "Clone the rootvg to an Alternate Disk"
image009

The next screen is where all the settings are done.

Select the client machine (server to be updated), type the disk which the TL will be applied—since this is going to be a alternate disk, make sure the disk is not used by any other volume group (VG). See Figure 10 for all options available on this menu.
Figure 10. NIM settings for Alternate Disk Install
image010

After you confirm and enter the values, NIM will automatically start updating the client.

Its progress can be seen from the client by looking at two log files (/var/adm/ras/nimlog and /var/adm/ras/alt_disk_inst.log).

Note:During the update process the screen won't be showing anything, but the process will be running on background. Make sure the process is completed before a reboot is done on the server—look at the log files.

After the update process is done, reboot the server.

Once the server has been rebooted, issue oslevel –s or oslevel –r and check if the OS level is now at the TL level that was applied.

 # oslevel –s
 5300-10-01-0921
If the update is considered successful, the rootvg can be mirrored again:

 # exportvg alt_inst_rootvg
 # extendvg –f rootvg hdisk1
 # mirrorvg rootvg

Create a new boot image on hdisk1 and add it to the boot list:

 # bosboot –ad /dev/hdisk1
 # bootlist –m normal –o hdisk0 hdisk1

No comments:

Post a Comment