Tanti Technology

My photo
Bangalore, karnataka, India
Multi-platform UNIX systems consultant and administrator in mutualized and virtualized environments I have 4.5+ years experience in AIX system Administration field. This site will be helpful for system administrator in their day to day activities.Your comments on posts are welcome.This blog is all about IBM AIX Unix flavour. This blog will be used by System admins who will be using AIX in their work life. It can also be used for those newbies who want to get certifications in AIX Administration. This blog will be updated frequently to help the system admins and other new learners. DISCLAIMER: Please note that blog owner takes no responsibility of any kind for any type of data loss or damage by trying any of the command/method mentioned in this blog. You may use the commands/method/scripts on your own responsibility. If you find something useful, a comment would be appreciated to let other viewers also know that the solution/method work(ed) for you.

Monday, 12 May 2014

Netstat Detailed


Netstat command displays various network related information such as network connections, routing tables, interface statistics, masquerade connections, multicast memberships etc.,
In this article, let us review 10 practical unix netstat command examples.

1. List All Ports (both listening and non listening ports)

List all ports using netstat -a

# netstat -a | more
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
udp        0      0 *:bootpc                *:*                                

Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6135     /tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     5140     /var/run/acpid.socket

List all tcp ports using netstat -at

# netstat -at
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
tcp        0      0 localhost:ipp           *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN

List all udp ports using netstat -au

# netstat -au
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 *:bootpc                *:*
udp        0      0 *:49119                 *:*
udp        0      0 *:mdns                  *:*

2. List Sockets which are in Listening State

List only listening ports using netstat -l

# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:ipp           *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN
udp        0      0 *:49119                 *:*

List only listening TCP Ports using netstat -lt

# netstat -lt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN

List only listening UDP Ports using netstat -lu

# netstat -lu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 *:49119                 *:*
udp        0      0 *:mdns                  *:*

List only the listening UNIX Ports using netstat -lx

# netstat -lx
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6294     private/maildrop
unix  2      [ ACC ]     STREAM     LISTENING     6203     public/cleanup
unix  2      [ ACC ]     STREAM     LISTENING     6302     private/ifmail
unix  2      [ ACC ]     STREAM     LISTENING     6306     private/bsmtp

3. Show the statistics for each protocol

Show statistics for all ports using netstat -s

# netstat -s
Ip:
    11150 total packets received
    1 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    11149 incoming packets delivered
    11635 requests sent out
Icmp:
    0 ICMP messages received
    0 input ICMP message failed.
Tcp:
    582 active connections openings
    2 failed connection attempts
    25 connection resets received
Udp:
    1183 packets received
    4 packets to unknown port received.
.....

Show statistics for TCP (or) UDP ports using netstat -st (or) -su

# netstat -st

# netstat -su

4. Display PID and program names in netstat output using netstat -p

netstat -p option can be combined with any other netstat option. This will add the “PID/Program Name” to the netstat output. This is very useful while debugging to identify which program is running on a particular port.
# netstat -pt
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        1      0 ramesh-laptop.loc:47212 192.168.185.75:www        CLOSE_WAIT  2109/firefox
tcp        0      0 ramesh-laptop.loc:52750 lax:www ESTABLISHED 2109/firefox

5. Don’t resolve host, port and user name in netstat output

When you don’t want the name of the host, port or user to be displayed, use netstat -n option. This will display in numbers, instead of resolving the host name, port name, user name.
This also speeds up the output, as netstat is not performing any look-up.
# netstat -an
If you don’t want only any one of those three items ( ports, or hosts, or users ) to be resolved, use following commands.
# netsat -a --numeric-ports

# netsat -a --numeric-hosts

# netsat -a --numeric-users

6. Print netstat information continuously

netstat will print information continuously every few seconds.

# netstat -c
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 ramesh-laptop.loc:36130 101-101-181-225.ama:www ESTABLISHED
tcp        1      1 ramesh-laptop.loc:52564 101.11.169.230:www      CLOSING
tcp        0      0 ramesh-laptop.loc:43758 server-101-101-43-2:www ESTABLISHED
tcp        1      1 ramesh-laptop.loc:42367 101.101.34.101:www      CLOSING
^C

7. Find the non supportive Address families in your system

netstat --verbose
At the end, you will have something like this.
 netstat: no support for `AF IPX' on this system.
 netstat: no support for `AF AX25' on this system.
 netstat: no support for `AF X25' on this system.
 netstat: no support for `AF NETROM' on this system.

8. Display the kernel routing information using netstat -r

# netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.1.0     *               255.255.255.0   U         0 0          0 eth2
link-local      *               255.255.0.0     U         0 0          0 eth2
default         192.168.1.1     0.0.0.0         UG        0 0          0 eth2
Note: Use netstat -rn to display routes in numeric format without resolving for host-names.

9. Find out on which port a program is running

# netstat -ap | grep ssh
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        1      0 dev-db:ssh           101.174.100.22:39213        CLOSE_WAIT  -
tcp        1      0 dev-db:ssh           101.174.100.22:57643        CLOSE_WAIT  -
Find out which process is using a particular port:
# netstat -an | grep ':80'

10. Show the list of network interfaces

# netstat -i
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0       1500 0         0      0      0 0             0      0      0      0 BMU
eth2       1500 0     26196      0      0 0         26883      6      0      0 BMRU
lo        16436 0         4      0      0 0             4      0      0      0 LRU
Display extended information on the interfaces (similar to ifconfig) using netstat -ie:
# netstat -ie
Kernel Interface table
eth0      Link encap:Ethernet  HWaddr 00:10:40:11:11:11
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:f6ae0000-f6b00000

VMSTAT - Virtual Memory Statistics



If someone asks me to check how Linux/UNIX system is performing now, first think I do it vmstat. Lot of people just checks for CPU and memory utilization statistics in vmstat. But in real, it gives more information that just CPU and memory information. In this posting, I am going to explain the detail of vmstat.

vmstat stands for virtual memory statistics; it collects and displays summary information about memory, processes, interrupts, paging and block I/O information. By specifying the interval, it can be used to observe system activity interactively.

Most commonly people will use 2 numeric arguments in vmstat; first is delay or sleep between updates and the second is how many updates you want to see before vmstat quits. Please note this is not the full syntax of vmstat and also it can vary between OSs. Please refer your OS man page for more information.

To run vmstat with 7 updates, 10 seconds apart type

#vmstat 10 7

Please note, in some systems, reported metrics might be slightly diffent. The heading that I am writing now are reported in Oracle Linux (Unbreakable Oracle Linux)


Process Block: Provides details of the process which are waiting for something (it can be CPU, IO etc; can be potentially for any resource)
r  -->  Processes waiting for CPU. More the count we observe, more processes waiting to run. If we just observe a spike in the count, we shouldn’t treat them as bottleneck. If value is constantly high (most people treats 2 * CPU count  as high), it hints that CPU is the bottleneck.
b  -->  Uninterruptible sleeping processes, also known as “blocked” processes. These processes are most likely waiting for I/O but could be for something else too
w  -->  number of processes that can be run but have been swapped out to the swap area. This parameter gives hint about memory bottleneck. Please remember, only some system reports this count

Memory Block: Provides detailed memory statistics
Swpd  -->  Amount of virtual memory or swapped memory used
Free  -->  Amount of free physical memory (RAM)
Buff  -->  Amount of memory used as buffers. This memory is used to store file metadata such as i-nodes and data from raw block devices
Cache  -->  Amount of physical memory used as cache (Mostly cached file).

Note: Most of the systems report memory block value in KB. Remember I said most; not all. So check man page.

Swap Block: Provided memory swap information
si  -->  Rate at which the memory is swapped back from the disk to the physical RAM
so  -->  Rate at which the memory is swapped out to the disk from physical RAM

Note: Most of the systems reports swap block value in KB. Check man page

I/O Block: I/O related information
bi  -->  Rate at which the system sends data to the block devices (in blocks/sec)
bo  -->  Rate at which the system reads the data from block devices (in blocks/sec)

System Block: System information
in  -->  Number of interrupts received by the system per second
cs  -->  Rate of context switching in the process space (in number/sec)

CPU block: Most used CPU related information
Us  -->  Shows the percentage of CPU spent in user processes. Most of the user/application/database processes comes under user processes category
Sy  -->  Percentage of CPU used by system processes, such as all root/kernal processes
Id  -->  Percentage of free CPU
Wa  -->  Percentage spent in “waiting for I/O”

Lot of people have problems here. Some people say us + sy +id + wa=100 and some other says us + sy +id =100. I stick to second (I/O doesn’t consume CPU). 

Interpretation with respect to performance:

The first line of the output is an average of all the metrics since the system was restarted. So, ignore that line since it does not show the current status. The other lines show the metrics in real time.

Ideally r/b/w values under procs block with close to 0 or 0 itself. If one or value counter values are constantly reporting high values, it means that system may not have sufficent CPU or Memory or I/O bandwidth.

If value of swpd under swap is too high and it keeps changing, the it means that system is running short of memory.

The data under “io” indicates the flow of data to and from the disks. This shows how much disk activity is going on, which does not necessarily indicate some problem(obviously data has to go to disk in order to be persistent). If we see some large number under “proc” and then “b” column (processes being blocked) and high I/O, the issue could be a I/O contention.

Rule of thumb in Performance
Adding more CPU, Memory or I/O bandwidth to the system is not the solution to the problem always; this is just postment of problem to future and it can blow anytime. The real solution is tune the application(every compoent in the architecture) as far as possible. Adding hardware capacity or buying a powerful hardware should be the last option.

IOSTAT



Iostat command reports CPU and I/O statistics. iostat command is used during performance analysis to narrow down the problematic areas in the system.
Sample output of iostat command and its explanation:
$ iostat
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          25.99    0.78    7.43   12.77    0.00   53.03

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              27.40       797.19       201.27     800902     202208
sr0               0.03         1.24         0.00       1248          0
The first line displays general information — Linux kernel version, hostname etc.,
Next two lines displays CPU statistics, in which the first 3 column displays average CPU usage. The 4th column displays the percentage of time the CPU were idle and waited for I/O response. 5th column displays percentage of waiting time of virtual CPU. 6th column displays the percentage of time the CPU is idle.
Remaining line displays the device utilization report. First line in that is headers, such as number of Transfer per second, number of blocks read per second, blocks write per second, total block reads, total block writes.


4 iostat Examples

Continuous execution of iostat with specific interval

Execute iostat continuously at specific interval, and up to N times
$ iostat interval count
Execute iostat every 10 seconds for 5 times.
$ iostat 10 5
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          24.24    1.51    7.49    4.97    0.00   61.79

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              14.06       249.94       121.22    1337998     648912
sr0               0.01         0.23         0.00       1248          0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          13.13    0.00    6.21    0.60    0.00   80.06

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               2.51         4.01        28.86         40        288
sr0               0.00         0.00         0.00          0          0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          11.11    0.00    5.71    0.00    0.00   83.18

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               0.30         0.00         3.20          0         32
sr0               0.00         0.00         0.00          0          0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          11.61    0.00    6.51    0.40    0.00   81.48

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               4.10         0.00        68.87          0        688
sr0               0.00         0.00         0.00          0          0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          15.03    0.00    5.91    0.20    0.00   78.86

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               1.50         0.00        30.46          0        304
sr0               0.00         0.00         0.00          0          0

View only the device report

View only the device report only once.
$ iostat -d
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              12.17       207.67       108.62    1352766     707584
sr0               0.00         0.19         0.00       1248          0
View the device report continuously for every 2 seconds, for 5 times.
$ iostat -d 2 5
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              11.12       186.33       100.49    1358494     732640
sr0               0.00         0.17         0.00       1248          0

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               2.00       108.00         0.00        216          0
sr0               0.00         0.00         0.00          0          0

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               5.00         0.00        96.00          0        192
sr0               0.00         0.00         0.00          0          0

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               1.01         0.00         8.04          0         16
sr0               0.00         0.00         0.00          0          0

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               0.00         0.00         0.00          0          0
sr0               0.00         0.00         0.00          0          0

View device statistics for particular devices

When you have multiple devices, and wanted information only about certain devices then you can do that with -p option.
$ iostat -p sda
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          23.71    0.91    8.63    3.20    0.00   63.55

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               9.72       154.75        90.78    1375518     806888
sda1              0.00         0.11         0.00        946          0
sda2              0.00         0.00         0.00         14          0
sda3              4.74       115.66        30.38    1028073     270024
sda4              0.01         0.17         0.00       1522          0
sda5              4.96        38.75        60.40     344427     536864
For viewing multiple devices statistics, you can use
$ iostat -p sda, sdb

View extended statistics report

You can view extended statistics report using -x option.
$ iostat -x sda
Linux 2.6.31-17-generic (sathiya-laptop)  05/25/10  _i686_ (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          23.20    0.73    8.62    2.75    0.00   64.70

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               3.65     7.00    4.99    3.64   131.34    85.14    25.10     0.35   40.25   4.55   3.93

Syntax and Options

iostat [ -c ] [ -d ] [ -N ] [ -n ] [ -h ] [ -k | -m ] [ -t ] [ -V ] [ -x ] [ -z ] [ device [...] | ALL ] [ -p [ device [,...] | ALL ] ] [ interval [ count ] ]
Short OptionOption Description
-cDisplay the CPU utilization report
-dDisplay the device utilization report
-hMake the NFS report displayed by option -n easier to read by a human
-kDisplay statistics in kilobytes per second instead of blocks per second. Data dis-played are valid only with kernels 2.4 and later
-mDisplay statistics in megabytes per second instead of blocks or kilobytes per sec-ond. Data displayed are valid only with kernels 2.4 and later
-NDisplay the registered device mapper names for any device mapper devices. Usefulfor viewing LVM2 statistics
-nDisplay the network filesystem (NFS) report. This option works only with kernel2.6.17 and later
-p [ { device [,...] | ALL } ]The -p option displays statistics for block devices and all their partitions that are used by the system. If a device name is entered on the command line, then statistics for it and all its partitions are displayed. Last, the ALL keyword indicates that statistics have to be displayed for all the block devices and partitions defined by the system, including those that have never been used. Note that this option works only with post 2.5 kernels
-tPrint the time for each report displayed. The timestamp format may depend on thevalue of the S_TIME_FORMAT environment variable (see below)
-VPrint version number then exit
-xDisplay extended statistics. This option works with post 2.5 kernels since it needs /proc/diskstats file or a mounted sysfs to get the statistics. This option may also work with older kernels (e.g. 2.4) only if extended statistics are available in /proc/partitions (the kernel needs to be patched for that)
-zTell iostat to omit output for any devices for which there was no activity duringthe sample period

 * Source Article from : Internet

SVMON


To print out global statistics, use the -G flag. In this example, we will repeat it
five timesat two-second intervals.

# svmon -G -i 2 5
       m e m o r y            i n  u s e            p i n        p g  s p a c e
  size inuse  free   pin   work  pers  clnt   work  pers  clnt     size   inuse
 16384 16250   134  2006  10675  2939  2636   2006     0     0    40960   12674
 16384 16254   130  2006  10679  2939  2636   2006     0     0    40960   12676
 16384 16254   130  2006  10679  2939  2636   2006     0     0    40960   12676
 16384 16254   130  2006  10679  2939  2636   2006     0     0    40960   12676
 16384 16254   130  2006  10679  2939  2636   2006     0     0    40960   12676

The columns on the resulting svmon report are described as follows:

memory
     Statistics describing the use of real memory, shown in 4 K pages.
     size
         Total size of memory in 4 K pages.
     inuse
         Number of pages in RAM that are in use by a process plus the number of
         persistent pages that belonged to a terminated process and are still resident
         in RAM. This value is the total size of memory minus the number of pages
         on the free list.
     free
         Number of pages on the free list.
     pin
         Number of pages pinned in RAM (a pinned page is a page that is always
         resident in RAM and cannot be paged out).
in use
     Detailed statistics on the subset of real memory in use, shown in 4 K frames.
    work
         Number of working pages in RAM.
     pers
         Number of persistent pages in RAM.
     clnt
         Number of client pages in RAM (client page is a remote file page).
pin
     Detailed statistics on the subset of real memory containing pinned pages, shown in
     4 K frames.
     work
         Number of working pages pinned in RAM.
     pers
         Number of persistent pages pinned in RAM.
     clnt
         Number of client pages pinned in RAM.
pg space
     Statistics describing the use of paging space, shown in 4 K pages. This data is
     reported only if the -r flag is not used. The value reported starting with operating
     system version 4.3.2 is the actual number of paging-space pages used (which
     indicates that these pages were paged out to the paging space). This differs from
     the vmstat command in that vmstat's avm column which shows the virtual
     memory accessed but not necessarily paged out.
     size
         Total size of paging space in 4 K pages.
     inuse
         Total number of allocated pages.

In our example, there are 16384 pages of total size of memory. Multiply this number
by 4096 to see the total real memory size (64 MB). While 16250 pages are in use,
there are 134 pages on the free list and 2006 pages are pinned in RAM. Of the total
pages in use, there are 10675 working pages in RAM, 2939 persistent pages in RAM,
and 2636 client pages in RAM. The sum of these three parts is equal to the inuse
column of the memory part. The pin part divides the pinned memory size into working,
 persistent and client categories. The sum of them is equal to the pin column of the
memory part.There are 40960 pages (160 MB) of total paging space, and 12676
pages are in use. The inuse column of memory is usually greater than the inuse
column of pg spage because memory for file pages is not freed when a program
completes, while paging-space allocation is.
=====================
Memory bottlenecks

      The following section describes memory bottleneck solutions with the
      following commands: vmstat, svmon, ps.

      1.vmstat

      Run the following command:

      vmstat 1

      NOTE: System may slow down when pi and po are consistently non-zero.
      pi number of pages per second paged in from paging space
      po number of pages per second paged out to paging space

      When processes on the system require more pages of memory than are
      available in RAM, working pages may be paged out to paging space and then
      paged in when they are needed again. Accessing a page from paging space
      is considerably slower than accessing a page directly from RAM. For this
      reason, constant paging activity can cause system performance degradation.

      NOTE: Memory is over-committed when the fr:sr ratio is high.
      fr number of pages that must be freed to replenish the free list or to
      accommodate an active process sr number of pages that must be examined
      in order to free fr number of pages

      An fr:sr ratio of 1:4 means for every one page freed, four pages must be
      examined. It is difficult to determine a memory constraint based on this ratio
      alone and what constitutes a high ratio is workload/application dependent.

      NOTE: Memory is over-committed to the point of thrashing when
      po*SYS>fr.

      The system considers itself to be thrashing when po*SYS > fr where SYS is a
      system parameter viewed with the schedtune command. The default value is 0
      if a system has 128MB or more. Otherwise,the default is 6. Thrashing occurs
      when the system spends more time paging than performing work. When this
      occurs, selected processes may be suspended temporarily, and the system
      may be noticeably slower.

      2.svmon

      As root run the following command:
      # svmon -Pau 10 | more

      Sample Output:

      Pid Command Inuse Pin Pgspace
      13794 dtwm 1603 1 449
      Pid: 13794
      Command: dtwm
      Segid Type Description Inuse Pin Pgspace Address Range
      b23 pers /dev/hd2:24849 2 0 0 0..1
      14a5 pers /dev/hd2:24842 0 0 0 0..2
      6179 work lib data 131 0 98 0..891
      280a work shared library 1101 0 10 0..65535
      181 work private 287 1 341 0..310:65277..65535
      57d5 pers code,/dev/hd2:61722 82 0 0 0..135

      This command lists the top ten memory using processes and gives a report
      about each one. In each process report, look where Type = work and
      Description = private. Check how many 4K (4096 byte) pages are used under
      the Pgspace column. This is the minimum number of working pages this
      segment is using in all of virtual memory. A Pgspace number that grows, but
      never decreases, may indicate a memory leak. Memory leaks occur when an
      application fails to deallocate memory.

      341 * 4096 = 1,396,736 or 1.4MB of virtual memory
      3. ps

      Run the following command:
      ps gv | head -n 1; ps gv | egrep -v "RSS" | sort +6b -7 -n -r
      size
      amount of memory in KB allocated from page space for the memory segment
      of Type = work and Description = private for the process as would be
      indicated by svmon.

      RSS
      amount of memory, in KB, currently in use (in RAM) for the memory segment
      of Type = work and Description = private plus the memory segment(s) of Type
      = pers and Description = code for the process as would be indicated by
      svmon.

      trs
      amount of memory, in KB, currently in use (in RAM) for the memory
      segment(s) of Type = pers and Description = code for the process as would
      be indicated by svmon.

      %mem
      RSS value divided by the total amount of system RAM in KB multiplied by
      100.