Paging Space Utilization

We received this alert this morning.  At the time, the paging space utilization exceeded 80%:

[yo@pd2]/>lsps -a
Page Space      Physical Volume   Volume Group Size %Used Active  Auto  Type Chksum
paging05        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging04        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging03        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging02        hdisk0            rootvg        1024MB    80   yes   yes    lv     0
paging01        hdisk0            rootvg        1024MB    79   yes   yes    lv     0
paging00        hdisk0            rootvg        1024MB    80   yes   yes    lv     0
hd6             hdisk0            rootvg        1024MB    80   yes   yes    lv     0

There is normally 4GB of paging space active, plus there is another 3GB standing by.  Occasionally AIX paging space will become filled with stale segments, especially when java is in use on the system, which in this case is mostly from WebSphere.  By deactivating and reactivating the active paging spaces, those get cleared out.  Note: it can take up to about 10 minutes per space to deactivate.

For the procedure:
1.    Activate at least one of the spare spaces
a.    swapon /dev/paging05
2.    Step through each of the other active spaces, and issue a swapoff and swapon
a.    swapoff /dev/hd6
b.    swapon  /dev/hd6
c.    swapoff /dev/paging00
d.    swapon  /dev/paging00
e.    swapoff /dev/paging01
f.    swapon  /dev/paging01
g.    swapoff /dev/paging02
h.    swapon  /dev/paging02
3.    Release the standby space(s) activated earlier
a.    swapoff /dev/paging05

Following the procedure, we went from 80% to 5%:

[yo@pd2]/home/yo>lsps -a
Page Space      Physical Volume   Volume Group Size %Used Active  Auto  Type Chksum
paging05        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging04        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging03        hdisk0            rootvg        1024MB     0    no    no    lv     0
paging02        hdisk0            rootvg        1024MB     2   yes   yes    lv     0
paging01        hdisk0            rootvg        1024MB     4   yes   yes    lv     0
paging00        hdisk0            rootvg        1024MB     6   yes   yes    lv     0
hd6             hdisk0            rootvg        1024MB     9   yes   yes    lv     0

Alert:

Subject: ALERT:Warning PctTotalPgSpFree at 19.926 on pd2

Host % Total Paging Space Free PctTotalPgSpFree for pd2 triggered PctTotalPgSpFree < 20 at 19.926

Alert detail:
ERRM_DATA_TYPE=CT_FLOAT64
ERRM_RSRC_CLASS_NAME=Host
ERRM_ATTR_NAME=% Total Paging Space Free
ERRM_COND_SEVERITYID=0
ERRM_NODE_NAMELIST={pd2}
ERRM_COND_NAME=PG_Warning
ERRM_ATTR_PNAME=PctTotalPgSpFree
ERRM_COND_HANDLE=0x6004 0xffff 0xd20fd739 0x873cfd56 0x126a2686 0xfbdb1899
ERRM_RSRC_HANDLE=0x6008 0xffff 0xd20fd739 0x873cfd56 0x122601f8 0x619e2352 ERRM_TYPE=Event
ERRM_ER_HANDLE=0x6006 0xffff 0xd20fd739 0x873cfd56 0x122601f1 0x70eec021
ERRM_RSRC_NAME=pd2
ERRM_RSRC_CLASS_PNAME=IBM.Host
ERRM_ATTR_NUM=1
ERRM_TYPEID=0
ERRM_TIME=1339082122,353161
ERRM_RSRC_TYPE=0
ERRM_COND_SEVERITY=Informational
ERRM_EXPR=PctTotalPgSpFree < 20
ERRM_VALUE=19.926
ERRM_COND_BATCH=0
ERRM_NODE_NAME=pd2
ERRM_ER_NAME=Warning notifications