Tanti Technology

My photo
Bangalore, karnataka, India
Multi-platform UNIX systems consultant and administrator in mutualized and virtualized environments I have 4.5+ years experience in AIX system Administration field. This site will be helpful for system administrator in their day to day activities.Your comments on posts are welcome.This blog is all about IBM AIX Unix flavour. This blog will be used by System admins who will be using AIX in their work life. It can also be used for those newbies who want to get certifications in AIX Administration. This blog will be updated frequently to help the system admins and other new learners. DISCLAIMER: Please note that blog owner takes no responsibility of any kind for any type of data loss or damage by trying any of the command/method mentioned in this blog. You may use the commands/method/scripts on your own responsibility. If you find something useful, a comment would be appreciated to let other viewers also know that the solution/method work(ed) for you.

Wednesday 26 February 2014

AIX troubleshooting : PART 1



IBM has announced "AIX5L".  It's essentialy AIX Version 5.  The 'L' stands
for "Linux Affinity".  A statement that AIX is going to support some of the
Linux API's and interfaces (for instance: the /proc filesystem)

Some changes to the filesystem limits, virtual IP's, dynamic dealocation
of swapspaces.

Using SMIT is probably very different from your normal way of doing
system administration, but could prove very useful in the long run. In
some areas, in particular TCP/IP, NFS, etc., you can also do things the
normal way, but it is unfortunately difficult to know exactly when the
normal way works. Again, always using SMIT is probably your best way
to go, even when you have to learn a new tool. 

What SMIT actually does is build up commands with all required options
to perform the functions requested and execute them. The commands
called and the output they produce are stored in the files smit.script
and smit.log in your home directory. Looking in smit.script may teach
you more about system administration.

How do I import an /etc/passwd or /etc/group file
                 from another box?

If the other box is non-AIX,  copy the password and group entries for
the non-system users into AIX's /etc/passwd and /etc/group files.
Then run /bin/pwdck -t ALL.  This will create the proper entries in
the shadow password file (/etc/security/users).  You should also run
usrck and grpck.

To duplicate the password and  group entries from another AIX box,
copy /etc/passwd, /etc/group, /etc/security/passwd, /etc/security/group,
/etc/security/user, /etc/security/limits, /etc/security/environ.  The
last three are optional unless you modified them.  If you modified
/etc/security/login.cfg, you should also copy that file.

How to fsck the root filesystem

You can run fsck either in maintenance mode or on mounted filesystems.
Try this:

  1. boot from diskette (AIX 3 only --- AIX 4 boot from CD or tape)
  2. select maintenance mode
  3. type /etc/continue hdisk0 exit (replace hdisk0 with boot disk if
     not hdisk0)
  4. fsck /dev/hd4

How can I unmount /usr to run fsck on it?

In order to fsck /usr, it has to be unmounted. But /usr cannot be
unmounted because /bin is symbolically linked to /usr/bin. Also
/etc/fsck is symbolically linked to /usr/sbin/fsck.

To work around this, when you boot from the boot/maintenance diskettes
and enter maintenance mode, enter "getrootfs hdisk0 sh" instead of
"getrootfs hdisk0" where hdisk0 is the name of the boot disk. Then run
"fsck /dev/hd2".

How do I see/change parameters like number of
                processes per user?

You can use SMIT as described below or simply use lsattr/chdev.
The former will list the current setting as in:

  # lsattr -E -l sys0 -a maxuproc
  maxuproc 40 Maximum # of processes allowed per user True

and you can then increase the maxuproc parameter:

  # chdev -l sys0 -a maxuproc=200
  sys0 changed

If you just type 'lsattr -E -l sys0' you will get a list of all
parameters, some of which can be changed but not others.

If you want to use smit, do as follows:

smit
  System Environments and Processes
    Change / Show Operating System Parameters
      - on this screen you can change by overtyping the following fields:
        - Maximum number of PROCESSES allowed per user
        - Maximum number of pages in block I/O BUFFER CACHE
        - Maximum Kbytes of real memory allowed for MBUFS
      - toggle fields exist for:
        - Automatically REBOOT system after a crash  (false/true)
        - Continuously maintain DISK I/O history (true/false)


How do I shrink the default paging space on hd6?

create a paging space to use temporarily
   mkps -s 20 -a rootvg
change default paging space hd6 so it is not used at next reboot
   chps -a n hd6
swapon /dev/paging00

sysdumpdev -p /dev/paging00

Update information in boot logical volume

   bosboot -ad /dev/hdisk0 

reboot
remove current hd6 and create a new one of smaller size
   rmps hd6
   mklv -y hd6 -t paging rootvg 

swapon /dev/hd6

change the dump device back to hd6:

   sysdumpdev -p /dev/hd6

Update information in boot logical volume

   bosboot -ad hdisk0

change current paging device (paging00) so it is inactive at next boot
   chps -a n /dev/paging00

shutdown, reboot, remove paging00 using the command:
    rmps paging00

You can check your paging space with `lsps -a`

The swapper seems to use enormous amounts of paging space, why?

When you run ps, you may see a line like:

USER   PID %CPU %MEM    SZ   RSS     TT STAT  TIME CMD
root     0 0.0%  14% 386528  8688      -    S 17:06 swapper

This is normal behavior, the swapper looks to ps like it has the entire
paging space plus real memory allocated.

How do I remove a committed lpp?

installp has a new option, uninstall (-u) which can be used to remove lpps.  BEWARE of pre-requisite chains.

How can I recover space after installing updates?

Note: If you are a /usr server, do not use this because the files
      mentioned below are needed by /usr clients and cannot be deleted.

Installp creates numerous files in /usr to clean up after failed/rejected installs and also for de-installing uncommitted lpps. Once you have COMMITted packages you can remove these files safely. Depending on your installation activity the numbers can be significant: hundreds-to-thousands of files, megabytes of data.

Files eligible for removal are associated with each "product" you have installed; the largest collection being due to bos. After COMMITting bos lpps, you may safely remove all files of the form:

/usr/lpp/bos/deinstl*
              /usr/lpp/bos/inst_U4*
              /usr/lpp/bosadt/deinstl*
        and   /usr/lpp/bosadt/inst_U4*

You may repeat this for all additional COMMITted products (e.g.,bostext1, bosnet, xlc) you have on your system.
This problem of lingering install files is a known defect in installp. If you have installed PTF U411711 (or any superseder of it: U412397, U413366, U413425) the deadwood in /usr will not be quite as prevalent. No single PTF currently available completely corrects this problem.

On my own 320, the following freed up 12.4M in /usr:

       # rm -R /usr/lpp/bos/deinstl*
       # rm -R /usr/lpp/bos/inst_U4*

Where are the AIX log files kept?

AIX logs messages as specified in /etc/syslog.conf.  Here's an
example

#
*.err;kern.debug;auth.notice;user.none          /dev/console
*.err;kern.debug;daemon,auth.notice;mail.crit;user.none /var/adm/messages
lpr.debug                                       /var/adm/lpd-errs

*.alert;kern.err;daemon.err;user.none           operator
*.alert;user.none                               root
*.emerg;user.none                               *

# for loghost machines, to have authentication messages (su, login, etc.)
# logged to a file, un-comment out the following line and adjust the 
# file name as appropriate.
#
# if a non-loghost machine chooses to have such messages 
# sent to the loghost machine, un-comment out the following line.
#
auth.notice                     /var/log/authlog
mail.debug                      /var/log/syslog

# following line for compatibility with old sendmails. they will send
# messages with no facility code, which will be turned into "user" messages
# by the local syslog daemon. only the "loghost" machine needs the following
# line, to cause these old sendmail log messages to be logged in the
# mail syslog file.
#
user.alert                       /var/log/syslog
#
# non-loghost machines will use the following lines to cause "user"
# log messages to be logged locally.
#
user.err                         /dev/console
user.err                         /var/adm/messages
user.alert                       `root, operator'
user.emerg                       *

How can I log information about ftp accesses to a file?

1) In /etc/syslog.conf, add the line:
   daemon.debug  /tmp/daemon.log

2) # touch  /tmp/daemon.log
   # refresh -s syslogd

3) Modify your inetd.conf so that ftpd is called with the "-l" flag.
   You may also want the "-d" flag. This can be done with 'smit inetdconf'.

All the syslog messages from various system daemons should now appear in
the file "/tmp/daemon.log".

How do I find a file name from the inode number?

ncheck -i nnnn /mntpoint

AIX troubleshooting : PART 2



How do I set the tty name associated with a physical port?

Let's say you wanted to make a tty on the s1 port and call it rs0000 and a tty on the s2 port and call it rs0001.

You could run:

mkdev -c tty -s rs232 -t tty -l rs0000 -p sa0 -w s1   # creates rs0000
and
mkdev -c tty -s rs232 -t tty -l rs0001 -p sa1 -w s2   # creates rs0001

How do I use mksysb to clone a system?

I use the following steps on the master machine to clone an AIX system:
1) Remove the password from root.
2) Remove the NIS line from the end of the /etc/group file (the last
   line with the +: )
3) Change most of the level '2' designations in /etc/inittab to level
   '3' to prevent them from being started up when the new system is
   booted (the minimum ones to change are rc.nfs and rc.tcpip)
4) Boot in service mode and change the name and ip address to a "spare"
   set to avoid address collision.
5) Clear /tmp, /usr/tmp and /usr/spool/lpd/stat.
6) Run mkszfile and edit it to be sure /usr is as small as possible;
   then mksysb from the command line.

The above changes allow me to boot in normal mode the first time, get in
as root, change the above files back and do the other things necessary
to configure the new system.

Then, of course, I go back and clean up and reboot my master machine.

Note: 1 and 2 lets you log in even if you can't get on the network. 
   It prevents the login process from trying to reach an NIS server. 
   Step 2 needed only if you use NIS.

How do I remove a non-existant physical volume?

To delete a phantom disk from the ODM use reducevg with the pvid instead of the disk name.  You are running some command such as lsvg or varyonvg and it is griping about a disk that is no longer findable right?  In that warning message, it should give you a pvid.  Try one of the following, (note: reducevg updates the VGDA but not the ODM).

reducevg -f  

ldeletepv -g VGid -p PVid
      -g Required, specify the VGid of the volume group you are
         removing the physical volume from
      -p Required, specify the PVid of the PV to be removed

How do I kill a process that ignores   kill -QUIT -KILL -STOP

If there is i/o pending in a device driver, and the driver does not catch the signal, you can't kill it - a reboot is the only way to clear it.

Furthermore, if the process stays hung for more than a few minutes, you can find out what device is wedged by doing this --

% echo trace -k $(expr  / 256) | crash | tee stack


How can I see "console" messages?

Use the swcons command to redirect the console to a file.  Or use chcons to do it permanently.

How do I merge my /etc/password and /etc/security/password for Crack?

/usr/sbin/mrgpwd.  You must have permissions to read /etc/security/password.

I lost the root password, what should I do?

Boot from boot diskettes, bootable tape, or bootable CD.  
At the Installation/Maint menu select item 4, "Start a limited function
 maintenance shell.
At the subsequent "#" prompt enter the command:
 getrootfs hdiskN
 (where "N" is replaced by the number of a disk on your system
 that is in rootvg.)
That will run for about a minute or so and you get a # prompt back.  At this
 point you are logged in as root in single user mode.
Change to /etc/security and edit the passwd file.  Delete the three lines
 under root: password, update time (or whatever it's called), and
 flags.  Save the file.
Then at the prompt, give root a new password.
Shutdown/reboot in normal mode.  Log in with new password.