Trouble shooting Application slowness issue
Linux performance related commands
top - to check cpu and memory usage
top - 03:32:44 up 1 day, 23:09, 6 users, load average: 0.00, 0.05, 0.32
Tasks: 157 total, 1 running, 156 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.1%us, 0.3%sy, 0.0%ni, 97.7%id, 0.8%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 1011508k total, 1001500k used, 10008k free, 3528k buffers
Swap: 2097148k total, 333960k used, 1763188k free, 69648k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10 root 20 0 0 0 0 S 3.7 0.0 0:19.80 rcu_sched
13415 oracle 20 0 1286m 293m 4116 S 1.8 29.7 0:29.02 java
1 root 20 0 19408 376 200 S 0.0 0.0 0:04.57 init
System activity report
sar -r 2 5
sar -r 2
[oracle@OEL6 ~]$ sar -r 2 2
Linux 3.8.13-98.2.2.el6uek.x86_64 (OEL6) 02/10/2016 _x86_64_ (1 CPU)
03:33:57 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit
03:33:59 AM 11020 1000488 98.91 2692 69448 2032144 65.37
03:34:01 AM 11020 1000488 98.91 2692 69448 2032144 65.37
Average: 11020 1000488 98.91 2692 69448 2032144 65.37
Input and out put statistics
iostat 2 5
iostat 2
[oracle@OEL6 ~]$ iostat 2 1
Linux 3.8.13-98.2.2.el6uek.x86_64 (OEL6) 02/10/2016 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.12 0.00 0.37 0.85 0.00 97.66
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
scd0 0.00 0.00 0.00 544 0
sda 4.91 160.92 43.61 27073268 7336356
fd0 0.00 0.00 0.00 16 0
[oracle@OEL6 ~]$ free
total used free shared buffers cached
Mem: 1011508 999864 11644 2328 3064 60936
-/+ buffers/cache: 935864 75644
Swap: 2097148 334344 1762804
df -h - Disk usage
[oracle@OEL6 ~]$ df -h /home/oracle/
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 57G 23G 32G 42% /
jmap -heap will display the snap shot of heap usage
[oracle@OEL6 ~]$ /home/oracle/jdk1.7.0_21/bin/jmap -heap 12659
Attaching to process ID 12659, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 23.21-b01
using thread-local object allocation.
Mark Sweep Compact GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 268435456 (256.0MB)
NewSize = 1310720 (1.25MB)
MaxNewSize = 17592186044415 MB
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 268435456 (256.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 80543744 (76.8125MB)
used = 75933440 (72.415771484375MB)
free = 4610304 (4.396728515625MB)
94.27602471521563% used
Eden Space:
capacity = 71630848 (68.3125MB)
used = 71377800 (68.07117462158203MB)
free = 253048 (0.24132537841796875MB)
99.6467332063415% used
From Space:
capacity = 8912896 (8.5MB)
used = 4555640 (4.344596862792969MB)
free = 4357256 (4.155403137207031MB)
51.112904268152576% used
To Space:
capacity = 8912896 (8.5MB)
used = 0 (0.0MB)
free = 8912896 (8.5MB)
0.0% used
tenured generation:
capacity = 178978816 (170.6875MB)
used = 101403672 (96.7060775756836MB)
free = 77575144 (73.9814224243164MB)
56.65680121607241% used
Perm Generation:
capacity = 130940928 (124.875MB)
used = 130640608 (124.58859252929688MB)
free = 300320 (0.286407470703125MB)
99.7706446681056% used
69269 interned Strings occupying 8985680 bytes.
Validation
Welogic Server check
Check for the weblogic server status
Number of free vs available threads and stuck threads
Timeouts in logs ( Application logs and server logs)
Check java Heap memory size ( Free memory)
Unix server Check
Check I/O statistics - iostat
Check the memory,swap Bottleneck - free
To check top process CPU utilizations -top
Check the disk space - df -h
DB Server check
Check for hung sessions
Check for long running queries
Check for CPU usage – If beyond a certain threshold need to take an action
Stale Stats
top - to check cpu and memory usage
top - 03:32:44 up 1 day, 23:09, 6 users, load average: 0.00, 0.05, 0.32
Tasks: 157 total, 1 running, 156 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.1%us, 0.3%sy, 0.0%ni, 97.7%id, 0.8%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 1011508k total, 1001500k used, 10008k free, 3528k buffers
Swap: 2097148k total, 333960k used, 1763188k free, 69648k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10 root 20 0 0 0 0 S 3.7 0.0 0:19.80 rcu_sched
13415 oracle 20 0 1286m 293m 4116 S 1.8 29.7 0:29.02 java
1 root 20 0 19408 376 200 S 0.0 0.0 0:04.57 init
System activity report
sar -r 2 5
sar -r 2
[oracle@OEL6 ~]$ sar -r 2 2
Linux 3.8.13-98.2.2.el6uek.x86_64 (OEL6) 02/10/2016 _x86_64_ (1 CPU)
03:33:57 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit
03:33:59 AM 11020 1000488 98.91 2692 69448 2032144 65.37
03:34:01 AM 11020 1000488 98.91 2692 69448 2032144 65.37
Average: 11020 1000488 98.91 2692 69448 2032144 65.37
Input and out put statistics
iostat 2 5
iostat 2
[oracle@OEL6 ~]$ iostat 2 1
Linux 3.8.13-98.2.2.el6uek.x86_64 (OEL6) 02/10/2016 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.12 0.00 0.37 0.85 0.00 97.66
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
scd0 0.00 0.00 0.00 544 0
sda 4.91 160.92 43.61 27073268 7336356
fd0 0.00 0.00 0.00 16 0
[oracle@OEL6 ~]$ free
total used free shared buffers cached
Mem: 1011508 999864 11644 2328 3064 60936
-/+ buffers/cache: 935864 75644
Swap: 2097148 334344 1762804
df -h - Disk usage
[oracle@OEL6 ~]$ df -h /home/oracle/
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 57G 23G 32G 42% /
jmap -heap will display the snap shot of heap usage
[oracle@OEL6 ~]$ /home/oracle/jdk1.7.0_21/bin/jmap -heap 12659
Attaching to process ID 12659, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 23.21-b01
using thread-local object allocation.
Mark Sweep Compact GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 268435456 (256.0MB)
NewSize = 1310720 (1.25MB)
MaxNewSize = 17592186044415 MB
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 268435456 (256.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 80543744 (76.8125MB)
used = 75933440 (72.415771484375MB)
free = 4610304 (4.396728515625MB)
94.27602471521563% used
Eden Space:
capacity = 71630848 (68.3125MB)
used = 71377800 (68.07117462158203MB)
free = 253048 (0.24132537841796875MB)
99.6467332063415% used
From Space:
capacity = 8912896 (8.5MB)
used = 4555640 (4.344596862792969MB)
free = 4357256 (4.155403137207031MB)
51.112904268152576% used
To Space:
capacity = 8912896 (8.5MB)
used = 0 (0.0MB)
free = 8912896 (8.5MB)
0.0% used
tenured generation:
capacity = 178978816 (170.6875MB)
used = 101403672 (96.7060775756836MB)
free = 77575144 (73.9814224243164MB)
56.65680121607241% used
Perm Generation:
capacity = 130940928 (124.875MB)
used = 130640608 (124.58859252929688MB)
free = 300320 (0.286407470703125MB)
99.7706446681056% used
69269 interned Strings occupying 8985680 bytes.
Validation
Welogic Server check
Check for the weblogic server status
Number of free vs available threads and stuck threads
Timeouts in logs ( Application logs and server logs)
Check java Heap memory size ( Free memory)
Unix server Check
Check I/O statistics - iostat
Check the memory,swap Bottleneck - free
To check top process CPU utilizations -top
Check the disk space - df -h
DB Server check
Check for hung sessions
Check for long running queries
Check for CPU usage – If beyond a certain threshold need to take an action
Stale Stats
Comments
Post a Comment