VIO scsi

Contents

·         1 Get a list of scsi adapters

·         2 Get SCSI Mapping

·         3 get list of disks

·         4 Get the size of the disk

·         5 Get the LUN id of the disk

·         6 Get the VPD of the disk

·         7 Change the reserve policy

·         8 Create a Virtual target device

·         9 get the device mapping details

·         10 (Optional) create an LPAR

o    10.1 Comments

Get a list of scsi adapters

$ lsdev -type adapter

name             status      description

ent0             Available   2-Port 10/100/1000 Base-TX PCI-Express Adapter (14104003)

ent1             Available   2-Port 10/100/1000 Base-TX PCI-Express Adapter (14104003)

ent2             Available   Virtual I/O Ethernet Adapter (l-lan)

ent3             Available   Shared Ethernet Adapter

fcs0             Available   8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

fcs1             Available   8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)

pager0           Available   Pager Kernel Extension

vasi0            Available   Virtual Asynchronous Services Interface (VASI)

vbsd0            Available   Virtual Block Storage Device (VBSD)

vhost0           Available   Virtual SCSI Server Adapter

vsa0             Available   LPAR Virtual Serial Adapter

$ lsdev -virtual | grep -i scsi

vhost0           Available   Virtual SCSI Server Adapter

Get SCSI Mapping

$ lsmap -all

SVSA            Physloc                                      Client Partition ID

--------------- -------------------------------------------- ------------------

vhost0          U8233.E8B.1000A1P-V7-C12                     0x00000000

VTD                   NO VIRTUAL TARGET DEVICE FOUND

$

get list of disks

$ lsdev -type disk

name             status      description

hdisk0           Available   MPIO Other FC SCSI Disk Drive

hdisk1           Available   MPIO Other FC SCSI Disk Drive

hdisk2           Available   MPIO Other FC SCSI Disk Drive

hdisk3           Available   MPIO Other FC SCSI Disk Drive

hdisk4           Available   MPIO Other FC SCSI Disk Drive

hdisk5           Available   MPIO Other FC SCSI Disk Drive

hdisk6           Available   MPIO Other FC SCSI Disk Drive

hdisk7           Available   MPIO Other FC SCSI Disk Drive

$

we will use hdisk1 for this example

Get the size of the disk

echo "bootinfo -s hdisk1 " | oem_setup_env

Get the LUN id of the disk

$ lsdev -dev hdisk1 -attr

attribute       value                                               description                      user_settable

PCM             PCM/friend/fcpother                                 Path Control Module              False

algorithm       fail_over                                           Algorithm                        True

clr_q           no                                                  Device CLEARS its Queue on error True

dist_err_pcnt   0                                                   Distributed Error Percentage     True

dist_tw_width   50                                                  Distributed Error Sample Time    True

hcheck_cmd      test_unit_rdy                                       Health Check Command             True

hcheck_interval 60                                                  Health Check Interval            True

hcheck_mode     nonactive                                           Health Check Mode                True

location                                                            Location Label                   True

lun_id          0x1000000000000                                     Logical Unit Number ID           False

lun_reset_spt   yes                                                 LUN Reset Supported              True

max_retry_delay 60                                                  Maximum Quiesce Time             True

max_transfer    0x40000                                             Maximum TRANSFER Size            True

node_name       0x500507680100b855                                  FC Node Name                     False

pvid            none                                                Physical volume identifier       False

q_err           yes                                                 Use QERR bit                     True

q_type          simple                                              Queuing TYPE                     True

queue_depth     1                                                   Queue DEPTH                      True

reassign_to     120                                                 REASSIGN time out value          True

reserve_policy  single_path                                         Reserve Policy                   True

rw_timeout      30                                                  READ/WRITE time out value        True

scsi_id         0x4d0d25                                            SCSI ID                          False

start_timeout   60                                                  START unit time out value        True

unique_id       3321360050768019102C0F000000000032F7904214503IBMfcp Unique device identifier         False

ww_name         0x500507680140b855                                  FC World Wide Name               False

Note the 'lun_id' value.

Get the VPD of the disk

$ lsdev -dev hdisk1 -vpd

  hdisk1           U5877.001.M09J131-P1-C6-T1-W500507680140B855-L1000000000000  MPIO Other FC SCSI Disk Drive

        Manufacturer................IBM

        Machine Type and Model......2145

        ROS Level and ID............30303030

        Serial Number...............2145

        Device Specific.(Z0)........0000063268181002

        Device Specific.(Z1)........

        Device Specific.(Z2)........

        Device Specific.(Z3)........

  PLATFORM SPECIFIC

  Name:  disk

    Node:  disk

    Device Type:  block

Change the reserve policy

$ chdev -dev hdisk1 -attr reserve_policy=no_reserve

hdisk1 changed

verify

$ lsdev -dev hdisk1 -attr | grep -i reserve_policy

reserve_policy  no_reserve                                          Reserve Policy                   True

Create a Virtual target device

$ mkvdev -vdev hdisk1 -vadapter vhost0 -dev sys414_lpar7

sys414_lpar7 Available

Details

§  -vdev -- the backing devic for the vdev

§  -vadapter -- the virtual adapter on the VIO to bind the vscsi disk

§  -dev -- the virtual device name that you want to assign

get the device mapping details

$ lsmap -vadapter vhost0

SVSA            Physloc                                      Client Partition ID

--------------- -------------------------------------------- ------------------

vhost0          U8233.E8B.1000A1P-V7-C12                     0x00000000

VTD                   sys414_lpar7

Status                Available

LUN                   0x8100000000000000

Backing device        hdisk1

Physloc               U5877.001.M09J131-P1-C6-T1-W500507680140B855-L1000000000000

(Optional) create an LPAR

Create an LPAR.. Follow the Wizzard...

When it asks for the virtual devices.. create a new Vitual SCSI device. Assign the server's virtual SCSI ID to this Virtual SCSI's 'Server Adapter id'

Example:

AIX Migration From 5.3 to 6.1 TL6 SP05

                         

Pre Migration Checks

-------------------------- --                                                                   30 to 40 min

1. Check the consistency of installed Licensed Program Products (LPPs)

    #lppchk -v -m3

    This will ideally display the inconsistencies (if any) of the LPPs.

     # instfix -i|grep -i ml

        

2. Check previous day night incremental backup is completed successfully.

   if the scheduled backup still going on, wait for backup to complete.

   If the scheduled backup is not successful, run the backup manually                                                   

    more /tsmlog/dsmsched.log | grep -i successfully

    

3. Take the mksysb backup and copy the mksysb  to tusnim00 server

4. Capture all the below outputs into user home (/home/rchandra)directory and scp to tusnim00 server also.

df -g > df.orig

lspv > lspv.orig

cat /etc/filesystems > filesystems.orig

lscfg > lscfg.orig

lsconf> lsconf.orig

ps -ef > ps.orig

netstat -nr > netstat.orig

ifconfig -a > ifconfig.orig

lsfs > lsfs.orig

lsdev -Cc disk > diskconfig.orig

cp /etc/motd /home/rchandra/

cp /usr/sbin/sendmail  /home/rchandra/

cp /etc/inetd.conf /home/rchandra/

cp /etc/environment /home/rchandra/

5. Check the compatibility of firmware with the upgrade version ( AIX 6.1)

6. /tmp filesystem will be clear during migration. Check /tmp before initiate migration. Take the backup of  files and directories to some other filesystem or server if require.

7 . Check the status  of  all harddisks (local,virtual and SAN) status configured in server.

8. Make sure that all application/data VG  filesystems backing up to TSM server.

Clone the rootvg using alt_disk method 

--------------------------------------------------

Note: We are using only phase1 and phase3 of IBM alt_disk_install method

1. Disable the quorum before unmirroring                                       20 Min

     chvg -Qn rootvg

     Unmirrorvg rootvg hdiskx

    lsvg -l rootvg

    Check filesystems, which are not mirrored.

Also compare the output of "lspv -l hdisk0" and "lspv -l hdisk1" and try to find out the unmirror filesystems.

 Then migrate the filesystem as per your requirement. E.g. if you want to unmirror hdisk1, migrate all unmirrored filesystems to hdisk0 from hdisk1.

   e.g. #migratepv -l pdumplv hdisk1 hdisk0

   #unmirrorvg rootvg hdisk1

   #chpv -c hdisk1

   #reducevg rootvg hdisk1

   #bootlist -m normal hdisk0

   #chvg -Qn rootvg

   #synclvodm -Pv rootvg

2. Clone rootvg                                                                     40 to 50 Min

   #mkdir /tmp/Migration

   #chmod 777 /tmp/Migration

   #alt_disk_install -C -P 1 hdisk5 > /tmp/Migration/rootvg_clone.log_1 2>&1

   #tail -f /tmp/Migration/rootvg_clone.log_1

Note ; hdisk1 should be the empty disk and not assigned to any volumegroup(generally we will use the disk which we free up by unmirroring rootvg)

   It will create an additional VG called altinst_rootvg and filesyems named liked /alt_usr, /alt_home etc.

3. Complete update process:                                                 10 Min

   #alt_disk_install -C -P 3 hdisk5 > /tmp/Migration/altinst_P3.log 2>&1

   It will unmount all the /alt_usr, /alt_home etc. filesystems and then varyoff the altinst_rootvg

NIM server side preparation

-------------------------------------                                            15 Min

1 . Prepare the NIM resources as below

Define the client ( ex : tusfidb07)

Prepare the respective lppsource using BOS filesets (AIX 6.1)

Make a spot using LPPSOURCE

Bind the resources lppsource  and spot with the client machine( nim_bosinst)

Below are NIM resources which we are using currently in tusnim00

LPPSSOURCE - AIX61TL06SP05_lppsource

SPOT - AIX61TL06SP05_spot

Note: Here we should use rte option while binding client with the NIM resources (lppsource and spot).

NIM Client side Preparation

-------------------------------------                                         

Note : Perform a precautionary reboot before starting any migration steps.

1 . Shut down the server once we get the confirmation call from Application team.  10 Min

2 . Boot the client node in SMS menu and start the migration activity.     10 Min

Set the IP parameters properly

NIM server IP :

Client IP :

Gateway:

Subnet mask:

Perform ping test to NIM server ( If fails stop the activity and check the network connectivity between NIM server and client)

Select network as boot device.

3 . Change the installation option as Migration after language selection.   120 Mins    

Check the destination disk where migration going to happen (That should be your currnent running rootvg disk generally hdisk0).

Observe and give few inputs till  the fileset installation started .Once Migration completed server will automatically reboot with new version of OS

Post Migration Tasks

----------------------------

Check the oslevel and consistency of the filesets using below command

#lppchk –v –m3

1)      Upgrade the TL to latest level AIX 6.1 TL7 SP2 using  smit update_all      60 Mins

   Filesets available  in  tadmin server under  /aix_sw/AIX61TL7SP2.

Run bosboot and set your bootlist order to correct harddisk before you initiate reboot.

#bosboot –ad /dev/diskname ( disk should be part of rootvg)

#bootlist –m normal hdiskx hdisky

2)      Check sshd daemon is running after migration, If not do the uninstall and  reinstall of openssh and openssl filesets.

Recreate the ssh keys after installing ssh.                                                    15 Min

3)      Modify the /etc/motd file after migration with server and data centre

4)      Check /etc/environment and modify if required (ex: LANG parameter LANG=en_US)

VIOS Advisor

Have You Seen the VIOS Advisor?

Download the tool here. I'll let IBM developerWorks provide the introduction.

"The VIOS advisor is an application that runs within the customer's VIOS for a user specified amount of time (hours), which polls and collects key performance metrics before analyzing results and providing a health check report and proposes changes to the environment or areas to investigate further.

"The goal of the VIOS advisor is not to provide another monitoring tool, but instead have an expert system view performance metrics already available to the customer and make assessments and recommendations based on the expertise and experience available within the IBM systems performance group.

     Download vios_advisor.zip from the link provided in download section.
     Unzip vios_advisor.zip on a workstation that has a web browser.
     ftp vios_advisor onto the VIOS you wish to monitor. (Place in any directory.
     chmod +x vios_advisor to give the application execution priviliges.

 "The application "vios_advisor" takes only one paramter, which is the duration of the the monitoring period, in minutes.

 "For example, to monitor the VIOS for 30 minutes, run:

      vios_advisor 30

   Usage Statement:

      Usage: vios_advisor
      duration_in_minutes:
      Recommended monitoring time = >= 30 min
      Minimum monitoring time = 5 min (only recommended for settings verification)
      Maximum monitoring time = 1440 min (24 hours)

      -v : Version

"The vios_advisor application is silent (does not produce any output to screen) and upon termination, will generate an xml file in the current running directory labeled:

     vios_advisor.xml

"Copy over the vios_advisor.xml file to the workstation where the zip file: vios_advisor.zip was extracted, and place the file in the vios_advisor folder. Open the vios_advisor.xml file with the web-browser of your choice to see the report.

"The measured overhead for the VIOS Advisor is minimal. An increase in CPU consumption of 0.1 cores was measured on a POWER7 server. Memory consumption will vary based on the number of physical I/O devices in the VIOS, but expect the advisor to consume 2-20 MB of memory."

I downloaded the advisor and extracted the files from the .zip file. Then I selected the vios_advisor_example file that was located in the newly created directory. This was the output in my browser:

I copied the vios_advisor file to my VIOS and ran chmod on it so that I could run the tool. Then I ran a quick test to make sure it worked:

$ chmod u+x vios_advisor
$ vios_advisor -v
vios_advisor  Version: 121211B

Then I ran:

 $ vios_advisor

 Usage: vios_advisor

      duration_in_minutes:
            Recommended monitoring time = >= 30 min
            Minimum monitoring time =         5 min  (only recommended for settings verification)
            Maximum monitoring time =      1440 min  (24 hours)

       -v :    Version

Since this was a test, I chose the minimum of five minutes to verify the settings.

$  vios_advisor 5

At the end of the test, I received a file called vios_advisor.xml. I copied that back to my PC, putting it in the directory that my vios_advisor.zip file was extracted to. Then I examined the report.

I'm sure IBM will continue to enhance the tool, helped along by user feedback.

So have you tried the VIOS advisor? How would you improve it?

DLPAR==Resolving an Issue with Dual HMCs

Recently, a customer was unable to run a DLPAR command against some of the LPARs on their frame. That in itself isn't unusual. Generally in these situations the network isn't communicating between the HMC and the LPAR, or perhaps RMC daemons need to be restarted somewhere.

This environment had dual HMCs connected to the managed system. HMC1 controlled some of the LPARs and HMC2 controlled others, but not by design. Although there was no rhyme or reason to it, for simplification let's say that HMC1 was controlling LPAR1 and LPAR3 and HMC2 was controlling LPAR2 and LPAR4. The correct setup would have been HMC1 and HMC2 controlling LPAR1, LPAR2, LPAR3 and LPAR4. In reality approximately 40 LPARs were on the frame, with each HMC controlling approximately half of the LPARs.

If you were on HMC1, you could DLPAR LPAR1 and LPAR3 with no issues. If you were on HMC2, you could DLPAR LPAR2 and LPAR4 with no issues. The problem was that the only way to know which HMC was controlling which LPAR was to either login to the HMC command line and run lspartition –dlpar, or use the HMC GUI and select HMC Management > View Network Topology. There was no way to know which HMC you needed to login to to manage which LPAR. This headache needed to be resolved.

Initially we did some troubleshooting with IBM Support. That resulted in us running things like:

            /usr/sbin/rsct/install/bin/recfgct
            /usr/sbin/rsct/bin/rmcctrl –p

We tried getting root access via pedbg. We also tried collecting a snap:

            /usr/sbin/rsct/bin/ctsnap

Eventually, once we escalated high enough up the support food chain, someone noticed a very basic HMC setup problem:

            The LPAR IBM.MgmtDomainRM default file shows this msg where it's attempting to create a    IBM.MCP entry for hmc1. It fails with Error number 14, duplicate key for localhost.

            2610-652 The specified time limit has been exceeded.
            Mon Feb 17 13:04:32 CST 2014(439849)      ../../../../../src/rsct/rm/MgmtDomainRM/MCP_cfg.c/01438/1.22  2613-034 Error number 14 was          returned when attempting to define an IBM.MCP resource.
            2610-014 The key token localhost is a duplicate.

During the initial build of the HMCs, they had been given their unique hostname and IP address, but somehow someone made a change that resulted in both hostnames being reset to localhost. Since these HMCs had the same hostname and ran on the same network, only one of the two was capable of managing LPARs at any given time. The other one would always fail.

Needless to say, if you run multiple HMCs, make sure they have unique hostnames. And in any environment, it's essential to establish good change control so people aren't making changes to systems without proper approvals and documentation

AIX 6L Network Installation Management (NIM)

                                                  

                                                

Course Summary

                                                                                                             Length: 2 days

Prerequisite: Introduction to AIX 6L System Administration

Recommendation Statement: The student should understand basic UNIX commands such as navigating directories, working with files and using the VI editor. In addition, the student should understand AIX file systems and how to install the AIX operating system from a local CD/DVD in a non-NIM environment.

Course Description: This course is for system administrators and operators that need to understand how to perform Network based installations of AIX 5L and 6L. You’ll learn how to perform NIM tasks such as configuring the NIM Master, NIM resources and NIM clients.

Upon Completion of this Course, you should be able to:

 Understand NIM concepts

 Understand NIM Resources including- SPOT, lpp_source, and mksysb

 Configuration of a Network Installation Manager Environment

 Configuring the NIM Master and Creating Basic Installation Resources

 Adding a NIM Client to the NIM Environment

 Performing a Base AIX O/S install onto a NIM Client

 Use a mksysb Image to Install the Base Operating System (BOS) on a NIM Client

 Perform a Non-prompted BOS Installation

 Define and Customize Additional Base Install Resources

 Understand NIM Network Issues

 Perform Maintenance and Updates on NIM resources

 Use NIM for Disaster Recovery

 Maintain Software in an LPP_SOURCE

 Boot a NIM client into maintenance mode from the boot image on a NIM server

AIX 6L Network Installation Management (NIM)

 Detailed Course Outline

Module 1. Network Installation Manager definitions and concept

 How NIM works

 Basic NIM configuration

 Planning the NIM Environment

 NIM file system hierarchy

Module 2. Basic configuration of a NIM environment

 Overview of the configuration process

 System requirements

 Install NIM filesets

 EZNIM vs. command Line vs. smit to configure the NIM environment

 Configure the NIM master

 Create file system for NIM

 Define basic resources

 Define the client

 Start the client installation

 Describe the NIM configuration files (/etc/niminfo, /etc/bootptab,

 Display NIM information

Module 3. Configure network services required to support NIM

 Inetd

 Tftp

 Bootp

 NFS

Module 4. The lpp_source

 lpp_source file system

 lpp_source naming convention

 Create a base level NIM resource from the CD/DVD

 Create lpp_source from AIX fileset image (using command line and smit)

 Display lpp_source information

Module 5. SPOT Objects

 SPOT file system

 Define a SPOT resource (using command line and smit)

 Create a SPOT from a mksysb resource

 Check a SPOT resource

 Remove a SPOT resource

Module 6. mksysb Resource

 mksysb image naming convention

 Define a mksysb resource

 Create a mksysb from a NIM client

Module 7. Define the Client

 Define the client from the command line and smit

 The /etc/niminfo file

 List a NIM client

 Remove a NIM client

Module 8. Start the Client Installation

 The /tftpboot directory

 The client.info file

 Watch the client installation status

 Perform a Non-prompted BOS installation

 The bosinst.data file

 The image.data file

Module 9. NIM Administrative Tasks

 Reset the NIM state

 Remove NIM machines and resources

 Deallocate resources

 Perform a maintenance boot from the master server

 Update the NIM lpp_source

 Update a NIM client

 Add software to the lpp_source

Module 10. Update or Clone a Client with alt_disk_install

Module 11. Troubleshoot a NIM Installation