Optimizing WebLogic Server Performance: JVM Tuning

Understanding garbage collection

It uses for a free memory occupied by objects that are no longer being used by the application. The Java Language Specification (JLS) does not require a JVM to have a garbage collector.All of the commonly used JVMs have garbage collectors, and most garbage collectors use similar algorithms to manage their memory and perform collection operations. we need to understand how the JVM performs garbage collection so that we can identify the tuning . The garbage collection algorithms and implementations, it is possible to tune the application and gc behaviour to max its  performance. 

Garbage collectors will identify an object is eligible for collection by objects are being referenced by any active objects in the system tor not. They are garbage tho general approaches for this are :

• Reference counting 
• Object reference traversal 

We have  Some garbage collectors used by modern JVMs are:

• Mark-and-sweep collector 
• Mark-sweep-compact collector 
• Copying collector 
• Incremental collector 
• Generational collector 
• Concurrent collector 
• Parallel collector Object lifetimes

Performance wise:

There are two primary types of garbage collection performance:

1. Throughput is the percentage of total time not spent in garbage collection.
2. Pauses are the times when an application appears unresponsive because garbage collection is occurring.

The performance wise we need to choose one of the two optimisations:

1. Max throughput - Java threads get as much as possible CPU resources.
2. Min pauses - Response time of the application is maximized.

To optimize we have to iterate through the following steps:

• Gather garbage collection information:
• Use this information to adjust JVM parameters.
• Check if the desired result is obtained, if not, gather garbage collection information.

Some of the Troubleshooting issues with commands

 For Setting a Class Path

WL-Home\servers\bin SetwlsEnv.cmd (windows)

WL-Home\servers\bin SetwlsEnv.sh (Unix)

For Setting a set Domain

WL-Home\servers\bin SetDomainEnv.cmd(win)

WL-Home\servers\bin SetDomainEnv.sh(Unix)

To increase WLS Memory

 set minimum and maximum to same size

$ Java ...-ms32m -mx32m      -> it will allocate 32 megabytes.

            :Xms2048m - Xmx2048m

 Increase Permgenspace
-XX :Maxpermsize=256m (default =64m).

To Enable verboseGC

Java_Options = -"Xverbose:Memory,gcreport,gcpause-xverbosetimestamp"

Enable core dump

sun JVM;  -xx:+ShowMessageBoxOnError

Jrockit JVM: -Djrockit.WaitOnError

Windows : DrWatson

Server is listening on the specified port number or not

telnet <IP> <Port>

Ex: telnet 199.129.212.1 8080

To check server is alive or not

Ping <IP>

Ex: Ping 199.129.212.1

start Managed Server Independence Mode?

nohup./StartManagedWeblogic_ManagedServername.sh&

weblogic server Process ID

usr/ucb/ps -auxwww | grep java

Thread dump:

Ps -ef | grep java

kill -3 <PID>             
ctl + Break

jstack <PID>   

1. 
   
   
   
   
   
   
   
   
   

Performance tuning parameters

Web Logic Server performance tuning ranges over an extremely broad area and includes the tuning of 

1. Operating system Tuning (CPU and Native IO)
2. JVM Tuning (Heap and GC tuning)
3. Weblogic server Tuning ( Cluster,Thread pools and stuck threads)
4. Workmangers Tuning. (Define workmangers)
5. Java DataBase Tuning (Connection Pools ,Data sources,Transactions )
6. JMS Tuning
7. Application Tuning and Web services Tuning (Servelets, JSP's)

Setting Java Parameters for Starting WebLogic Server
The important performance tuning parameters in these files are the JAVA_HOME parameter and the Java heap size parameters: 

• Change the value of the variable JAVA_HOME to the location of your JDK. For example:
  set JAVA_HOME=C:\bea\jdk150_03 
• For higher performance throughput, set the minimum java heap size equal to the maximum heap size. For example:
  "%JAVA_HOME%\bin\java" -server – Xms512m – Xmx512m -classpath %CLASSPATH% - 

Work Managers and Execute Queues

Execute Queues: incoming requests are put into a default execute queue or a user-defined execute queue. Each execute queue has an associated execute queue manager that controls an exclusive, dedicated thread-pool with a fixed number of threads in it.

Work managers provide you the ability to better control thread utilization (server performance) than execute-queues, primarily due to the many ways that you can specify scheduling guidelines for the priority-based thread pool. These scheduling guidelines can be set either as numeric values or as the capacity of a server-managed resource, like a JDBC connection pool.

Tuning the Stuck Thread Detection Behavior

WebLogic Server diagnoses a thread as stuck if it is continually working (not idle) for a set period of time. You can tune a server’s thread detection behavior by changing the length of time before a thread is diagnosed as stuck, and by changing the frequency with which the server checks for stuck threads a thread in an execute queue becomes “stuck.” Because a stuck thread cannot complete its current work or accept new work

WebLogic Server Clusters to Improve Performance

WebLogic Servers instances that together provide fail-over and replicated services to support scalable high-availability operations for clients within a domain.Scalability expands the capacity of the middle tier beyond that of a single WebLogic Server or a single computer,New WebLogic Servers can be added to a cluster dynamically to increase capacity but it must be able to communicate by IP multicast.

Session Replication

User session data can be stored in two standard ways in a J2EE application: stateful session EJBs or HTTP sessions.

HTTP Seession replication and Replica aware stub.

high-availability by using the redundancy of multiple servers to insulate clients from failures. The same service can be provided on multiple servers in a cluster. If one server fails, another can take over. The ability to have a functioning server take over from a failed server increases the availability of the application to clients.

Java Heap Size and Garbage Collection

Java heap is repository for live objects, dead objects, and free memory. When an object can no longer be reached from any pointer in the running program, it is considered “garbage” and ready for collection.

Garbage collectors usually have to stop all other activity for some portion of the garbage collection process. This stop-the-world approach means all application-related work stops while the garbage collector runs. As a result, any in-flight requests will experience an increase in their response time by the amount of time taken by the garbage collector. Some garbage collectors used by modern JVMs are:

Types of Collectors

•    Mark-and-sweep collector – traverses the object graph and marks reachable objects. Next, the heap is scanned for unmarked objects and their memory added to a list of available memory segments. This collector uses a single thread to do its work and is a stop-the-world collector. 
•    Mark-sweep-compact collector – uses the same marking phase as a mark-and-sweep collector. During the second phase, the heap is compacted by copying marked objects to a new area of the heap. These collectors are stop-the-world collectors. 
•    Copying collector – divides the heap into two areas. Only one area at a time is used. When the garbage collector runs, all reachable objects are copied to the other area, thus compacting the heap as the live objects are copied. All dead objects are left behind. This algorithm works well for short-lived objects, but the expense of continually copying long-lived objects makes it less efficient. This is a stop-the-world collector. 
•    Incremental collector – divides the heap into multiple areas and collects garbage from only one area at a time. This can create much smaller, though more frequent, pauses in the application. Numerous approaches exist for defining how the actual collection is handled from traditional mark-and-sweep to algorithms designed explicitly for use with multiple smaller areas. 
•    Generational collector – divides the heap into two or more areas that are used to store objects with different lifetimes. The JVM generally creates all new objects in one of these areas. Over time, the objects that continue to exist get tenured and move into another area for longer-lived objects. Generational collectors often use different algorithms for the different areas to optimize performance. 
•    Concurrent collector – runs concurrently with the application, typically as one or more background threads. These collectors typically have to stop-the-world at some point to complete certain tasks, but the amount of time they halt all processing is significantly reduced because of their background work. 
•    Parallel collector – uses one of the traditional algorithms but uses multiple threads to parallelize their work on multiprocessor machines. This can dramatically improve the scalability of an application.

New features in Weblogic 12c (12.1.2).

WebLogic 12c (12.1.2) is the latest version released. It is robust, provides unmatched performance and high availability.

This can be downloaded from the below URL.

http://www.oracle.com/technetwork/middleware/fusion-middleware/downloads/index.html

Below are few new features that are introduced in WebLogic 12.1.2.

1. Dynamic Cluster Support:

It is one of the most striking features of WebLogic 12.1.2. This release introduces use of the dynamic cluster for a highly scalable systems, high availability environment.  Dynamic cluster allows the managed servers to be automatically added based on a server template.

2. Support with Oracle Database 12c.

This release of WebLogic Server adds support for Oracle Database 12c

3. JMS enhancements:

Supports clustered targeted JMS Servers for providing high availability  eliminating the need to configure many JMS resources for every single server.

4. Enhanced maven support.

Support for POMs, Maven 3.0.4.

5. WebSocket support.

Supports for WebSocket Protocol (RFC 6455),which provides two-way, full-duplex communication over a single TCP connection between clients and servers, where each side can send data independently from the other. The WebSockets communication model occurs in real-time and promotes user interaction.

Weblogic - Identity store


The identity store contains the definitions of:

users

groups and

groups hierarchies

required for authentication.

It's:

the embedded LDAP server

and User profile derived from other LDAP server

JVM

JVM(Java Virtual Machine)

JVM Architecture: JVM is the heart of any Java based Application Server. We face most of the issues due to incorrect JVM tuning. It is very important to understand the Overall architecture of the JVM in order to trouble shoot different JVM tuning related issues. Here we are going to discuss the Architecture and the Major parts of a Java Process And the Java Heap Division.

The Following Diagram is just a basic overview of a Java Process in a 2 GB process Size Machine. Usually in 32 bit Windows Operating Systems the default process size will be 2 GB (In Unix based 64 bit operating Systems it can be 4GB or more). So i draw the following Diagram of Java Process to explain the Java Process partitions in a 2Gb process size machine.

Java Process Architecture Diagram

In the above diagram we will find different partitions of a Java Process. Please compare the above diagram with below descriptions.

.

1) Just for Example we can see that Process Size is 2048 MB (2GB)

2) The Java Heap Size is 1024MB (means 1GB)   -Xmx1024m

3) Native Space = ( ProcessSize – MaxHeapSize – MaxPermSize) It means around 768 MB of Native Space.

4) MaxPermSpace is around -XX:MaxPermSize=256m

5) Young Generation Space is around    40% of Maximum Java Heap.

Different parts of JVM:

1) Eden space: Eden Space is a Part of Java Heap where the JVM initially creates any objects, where most objects die and quickly are cleaned up by the minor Garbage Collectors (Note: Full Garbage Collection is different from Minor Garbage Collection). Usually any new objects created inside a Java Method go into Eden space and the objects space is reclaimed once the method execution completes. Where as the Instance Variables of a Class usually lives longer until the Object based on that class gets destroyed. When Eden fills up it causes a minor collection, in which some surviving objects are moved to an older generation.

2) Survivor Spaces: Eden Space has two Survivor spaces. One survivor space is empty at any given time. These Survivor Spaces serves as the destination of the next copying collection of any living objects in Eden and the other survivor space.

The parameter SurvivorRatio can be used to tune the size of the survivor spaces.

-XX:SurvivorRatio=6 sets the ratio between each survivor space and Eden to be 1:6

If survivor spaces are too small copying collection overflows directly into the tenured generation.

3) Young Generation: (-XX:MaxNewSize): Till JDK1.3 and 1.4 we used to set the Young Generation Size using -XX:MaxNewSize. But from JDK1.4 onwards we set the YoungGeneration size using (-Xmn) JVM option.

Young Generation size is controlled by NewRatio.  It means setting -XX:NewRatio=3 means that the ratio between the Old Generation and the Young Generation is  1:3. Similarly -XX:NewRatio=8 means that 8:1 ratio of tenured and young generation.

NewRatio: NewRatio is actually the ratio between the (YoungGenaration/Old Generations) has default values of 2 on Sparc , 12 on client Intel, and 8 everywhere else.

NOTE: After JDK 1.4 The Young Generation Size can be set using  (-Xmn) as well.

1) Virtual space-1:(MaxNewSize – NewSize): The First Virtual Space is actually shows the difference between the -XX:NewSize and -XX:MaxNewSize.  Or we can say that it is basically a difference between the Initial Young Size and the Maximum Young Size.

JavaHeapArea:( -Xmx and –Xms): Java Heap is a Memory area inside the Java Process which holds the java objects.  Java Heap is a combination of Young Generation Heap and Old Generation Heap. We can set the Initial Java Heap Size using -Xms JVM parameter similarly if we want to set the Maximum Heap Size then we can use -Xmx JVM parameter to define it.

Example:

-Xmx1024m —> Means Setting the Maximum limit of Heap as 1 GB

-Xms512m —> Means setting Java Heap Initial Size as 512m

.

NOTE-1): It is always recommended to set the Initial and the Maximum Heap size values as same for better performance.

NOTE-2): The Theoretical limitation of Maximum Heap size for a 32 bit JVM is upto 4GB. Because of the Memory Fragmentation, Kernel Space Addressing, Swap memory usages and the Virtual Machine Overheads are some factors JVM does not allow us to allocate whole 4GB memory for Heap in a 32 bit JVM. So usually on 32-bit Windows Operating Systems the Maximum can be from 1.4 GB to 1.6 GB.

If we want a larger memory allocation according to our application requirement then we must choose the 64-bit operating systems with 64 bit JVM. 64-bit JVM provides us a larger address space. So we can have much larger Java Heap  with  the increased number of Threads allocation area. Based on the Nature of your Operating system in a 64 bit JVM you can even set the Maximum Heap size upto 32GB.

Example:        -Xms32g -Xmx32g -Xmn4g

2) Virtual Space-2: (MaxHeapSize – InitialHeapSize): The Second Virtual Space is actually the Difference between the Maximum Heap size (-Xmx)and the Initial Heap Size(-Xms). This is called as virtual space because initially the JVM will allocate the Initial Heap Size and then according to the requirement the Heap size can grow till the MaxHeapSize.

PermGen Space: (-XX:MaxPermSize): PermGen is a non-heap memory area where the Class Loading happens and the JVM allocates spaces for classes, class meta data,  java methods and the reference Objects here. The PermGen is independent from the Heap Area. It can be resized according to the requirement using -XX:MaxPermSize and -XX:PermSize  JVM Options. The Garbage collection happens in this area of JVM Memory as well. The Garbage collection in this area is called as “Class GC”. We can disable the Class Garbage Collection using the JVM Option -noclassgc. if  ”-noclassgc” Java Option is added while starting the Server. In that case the Classes instances which are not required will not be Garbage collected.

Native Area: Native Memory is an area which is usually used by the JVM for it’s internal operations and to execute the JNI codes. The JVM Uses Native Memory for Code Optimization and for loading the classes and libraries along with the intermediate code generation.

The Size of the Native Memory depends on the Architecture of the Operating System and the amount of memory which is already commited to the Java Heap. Native memory is an Process Area where the JNI codes gets loaded or JVM Libraries gets loaded or the native Performance packs and the Proxy Modules gets loaded.

There is no JVM Option available to size the Native Area. but we can calculate it approximately using the following formula:

NativeMemory = (ProcessSize – MaxHeapSize – MaxPermSize)