Business Object Parsing Modes

A new business object framework runtime is provided in version WebSphere Version 7, which is built on a new high speed XML infrastructure that enables optimized XML processing and enhanced XML fidelity.

The following should be taken into account when choosing the parsing mode:

- A module can be configured to run in one parsing mode.

- Modules that were created in a version of IID prior to version 7 will run in the eager parsing mode without any changes required.

- By default, modules and libraries that are created in IID version 7 and later versions will be given the most suitable parsing mode depending on the parsing mode of

- existing projects in the workspace
- dependent projects in the workspace
- other projects in the same solution

- For better performance, avoid using mixed mode (eager, lazy) for application modules that frequently interact with each other.

- The interaction between modules with diff erent parsing modes takes place through serialization and deserialization, which can negatively affect performance.

- If using a mixed mode is unavoidable, starting an application from an eager parsing module to a lazy parsing module is more efficient than the other way around.

The WebSphere runtime uses the RMI/IIOP for the synchronous  invocations.

Assuming we have the following scenario:

SCA synchronous intermodule communication (two servers)

where the SCA messages are passed by value across different servers using RMI/IIOP. The SCA messages are transferred from one SCA at runtime to the SCA runtime on the second server by using the RMI/IIOP protocol.

In this case there is a slowdown in speed due to the serialization and deserialization of the data between the two servers. A slowdown is introduced also if the parsing modes of the two modules are different, as show below:

An attractive alternative solution for these kind of invocations (intermodule between two different clusters) could be the following:

where the SCA cross JVM invocation has been replaced by a Web Service binding invocation.

(Note that the module M1A acts as a static single access point for multiple consumers that want access to the services provided by the Cluster B modules.)

By bench marking the above solutions (40 parallel requests every 2 seconds) with the same sample XML (~3KB) we show that the average response time for the second scenario (Web Services) was 171 msec, whereas for the first scenario (SCA) the average response time was 260 msec.

References

[1] IBM Information Center- “Consideration when choosing the business object parsing mode"  Date of access: October 2014. http://www-01.ibm.com/support/knowledgecenter/#!/SSFPJS_7.5.1/com.ibm.wbpm.main.doc/bo/topics/rboparse.html

Out of Memory during Passivation of View Object with LOV

Have you ever faced an Out Of Memory Error (OOME) with your ADF application? You probably had so please go ahead and read the rest of this post...

Sometimes it seems so very easy to create problems, but it is so damn hard to find them out and fix them. In this post we will start from the easy part: how to create a problem. The main ingredients of the recipe that may lead to some serious problems in production environments are summarized here:

• An old code base that was migrated from earlier versions of JDeveloper (e.g. 11.1.1.4)
• A View Object with an Attribute that has a List of Values (LOV) defined
• The View Object has a View Accessor that is used in the LOV
• The List Data Source is a View Object based on SQL Query. The View Object is configured to Passivate State Including All Transient Attributes

View Object - Bad Tuning Example

In addition to the above we should also consider the fact that the LOV was build based on a table with thousands of rows in the production environment. No provision was made in order to filter data and limit the results the LOV could fetch from the database.

If you forget for the moment that all the above configuration smells some kind of trouble and should have been avoided at the beginning, think that was just something in production without any tool or procedure (code review, testing etc) ever noticing it.

The result of the above was either very long (a day or so...) running requests and sometimes an OOME error. The worst part was that this problem happened only twice... A "stuck thread" and an OOME incident and no more... No one could re-produce the problem. That was the worst part since another fix included some modifications through ADF customization that really buried the problem for the specific installation. This was proved just by accident...

In the rest of the post I will try to explain the ADF behavior with the above configuration. In another post I will show how to take advantage of any trace of the problem you may have in your hands like a JFR.

In the following figure you can have a look of the UI that I have created in order to demonstrate the behavior described above.

In order to reproduce the problem I have created two View Objects: EmployeeVO and DepartmentsVO. Based on the DepartmentsVO I have created a ViewAccessor and an LOV for the DepartmentId attribute.

For both the ViewAccessor and the LOV I have left the default values that JDeveloper generates, but for the LOV I have chosen the "Input Text with List of Values".

Now if you choose to disable Application Module (AM) pooling and run the application you would get a nice AM activation passivation cycle. So far so good, the application is running fast and each time we press one of the form's buttons we are happy with the result, but shall we?...

Let's dig a little more and study what's going on behind the scenes. In order to get directly to the point of this post, let start from checking what is going on with the activation/passivation cycle, since we have deliberately disabled AM pooling. (By the way this is also recommended by Oracle while developing). In my case I have increased the logging level for the oracle.jbo.server.Serializer to FINEST. At the log file the Serializer has produced the following XML during the passivation of the AM.

There are two ViewObjects that get passivated: EmployeeVO1 and _LOCAL_VIEW_USAGE_com_soapplied_blog_ap_model_EmployeesVO_DepartmentsVA. For the first things look normal, but for the second things are not so pretty... Just image what may happen if the VO was not querying the Departments table, but something quite bigger!

So in a case where the query behind the Departments VO could fetch thousands of records from the DB, this would lead to an enormous XML document that the Serializer has to build and persist to the DB. This is a very CPU and memory hungry procedure that would certainly cause trouble to a production environment. To add to this very bad configuration we have left the fetch size of the VOs to 1. This means that all records of the Departments VO will be fetched one by one from the DB. This is also a very slow procedure with thousands of round-trips to the DB.

References:
https://blogs.oracle.com/ADFProgrammers/entry/restrict_the_number_of_items

http://adfpractice-fedor.blogspot.gr/2013/07/passivation-and-activation-of-view.html

Print active transactions

Get list of active transactions.

Create the AdminClient class in order to provide the client side APIs for the remote AdminService.

Get the TransactionService

For each TransactionService MBean print its active transactions:

References

[1] IBM Information Center - “TransactionService MBean"  Date of access: July 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=%2Fcom.ibm.websphere.javadoc.doc%2Fpublic_html%2Fmbeandocs%2FTransactionService.html

Querying multiple databases using one-phase capable JDBC resources

1-2 Phase Commit (PC)

Within a one phase commit a transaction manager (responsible for managing the transactional operations of transactional components) sends out the commit message after a unit of work is executed.

In a two phase commit the transaction manager first checks with each local resource manager regarding whether they are ready to commit the transaction (sends a prepare for commit message) and based on the response from each resource manager, the transaction manager instructs all resource managers to commit the transaction if all agree, or to roll back the transaction if at least one disagrees. 

Using more than one 1PC resources in the same transaction 

Assuming the following scenario:

Figure 1 : Querying more than one 1PC resources example

Where:

Figure 2 : Querying more than one 1PC resources Beans detail

Both figures above illustrate the following:

• WebSphere is responsible for managing the transactional operations of transactional components (components involved in a transaction).
• All Session Beans do not have a TransactionManagementType annotation declared at the class level which indicates that all  have CMT transaction (default for session beans)
• All Session Beans do not have a TransactionAttributeType annotation declared at the class level which indicates that all have the REQUIRED value for all methods.This means:
• Methods must always execute in a transaction context.
• If there is a transaction already running, the bean participates in that transaction. 
• If there is no transaction, the EJB container starts a transaction on behalf of the bean.
• The method in FacadeBean runs in a propagated transaction if present, otherwise in a new global transaction. This means:
• The changes that are made by the services invoked to the transactional resources (from BeanA and BeanB) are persisted when the global transaction commits.
• The JDBC driver implementation class used for DS1 is com.microsoft.sqlserver.jdbc.SQLServerConnectionPoolDataSource (i.e One-Phase Transaction Support - 1PC).
• The JDBC driver implementation class used for DS2 is com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource (i.e One-Phase Transaction Support - 1PC).
• All resources get enlisted by WAS on the current global transaction so that is can be managed.

The result of the above configuration is the following exception:

These errors occur because there are more than one one-phase capable (1PC) resources within a global transaction.

Two alternative configurations are illustrated below in order to resolve the above problem.

Last Participant Support

With Last Participant Support (LPS), it is allowed to have one one-phase capable resource with any number of two-phase capable resources in the same global transaction.

Figure 3 : One-phase resource participates in a two-phase transaction with one two-phase resources

Also using the administrative console:

Figure 4 : Enable last participant support extension

Resource manager local transaction (RMLT)

A resource manager local transaction (RMLT) is a resource manager's view of a local transaction; that is, it represents a unit of recovery on a single connection that is managed by the resource manager. A local transaction containment is used to define the application server behavior in an unspecified transaction context. One example is the coordination of multiple one-phase resource managers that do not support XA transaction coordination. A session bean starts starts an ActivitySession and perform its RMLTs within the scope of that ActivitySession The ActivitySession service provides an alternative unit-of-work scope to the scope that is provided by global transaction contexts. 

The above figure illustrates the following:

• The TransactionAttributeType annotation at the methodB in BeanB, overwrites the class level declaration and indicates that this specific method has NOT_SUPPORTED transaction value. This means:
• Method cannot be involved in a transaction.
• Method will always execute outside the transaction context. 
• If there is a transaction running, it is suspended and will be resumed after the end of those methods.
• In the deployment descriptor of the BeanB, in the Local Transactions section, the Resolver attribute is set to Application and the Unresolved action attribute is set to Rollback. This means:
• The component processing occurs within a WebSphere local transaction containment  that is managed by the application.
• The Resolver option define how the local transaction is to be resolved before the local transaction context ends. The application option indicates that is the responsibility of the application to commit or roll back transactions. 
• The unresolver-action options are commit or rollback and specify the action the container will take if the resources are uncommitted by an application in a local transaction.
• BeanB starts an ActivitySession and perform its RMLTs within the scope of that ActivitySession.
• The code is responsible for forcing commitment by using the javax.resource.cci.LocalTransaction.begin() and javax.resource.cci.LocalTransaction.commit() i.e. code delineates the transaction (another option is to have setAutocommit(true) - each statement will run in its own transaction).
• In case the code does javax.resource.cci.LocalTransaction.begin() but does not call the javax.resource.cci.LocalTransaction.commit()  or sets setAutocommit(false) after obtaining a JDBC connection and not add any code to control the transaction, then the state of the transaction at the end of the method would be ambiguous. To resolve this problem the  unresolver-action option is set. 

References

[1] IBM Support Portal - “WTRN0062E and J2CA0030E Errors While Trying To Do Last Participant Support Extensions"  Date of access: July 2013. http://www-01.ibm.com/support/docview.wss?uid=swg21244805

[2] IBM Redbooks - Ueli Wahli, Giuseppe Bottura, Jacek Laskowski, Nidhi Singh - “WebSphere Application Server Version 6.1 Feature Pack for EJB 3.0." September 2008. Date of access: April 2013. http://www.redbooks.ibm.com/abstracts/sg247611.html

[3] Oracle Documentation - "The Java EE 6 Tutorial" . January 2013. Date of access: June 2013. http://docs.oracle.com/javaee/6/tutorial/doc/bncih.html

[4] IBM Information Center - “JDBC provider summary"  Date of access: July 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=/com.ibm.websphere.nd.multiplatform.doc/info/ae/ae/udat_minreq.html

[5] IBM Information Center - “Local transaction containment"  Date of access: June 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Frjta_useltran.html

DB2 - Export database contents

Script for export all DB2 database tables data - useful for fast search and comparison

In case there is a need to search or compare database contents (without knowing at the beginning in which table to search for - such that to use sql statement(s)) then you could export database tables data in delimiter file format as well as export  all LOB and XML data. The script below illustrates this scenario.

WebSphere distributed transactions with SQL Server Linked Server

SQL Server Linked Server

Linked server enable the SQL Server Database Engine to execute commands against OLE DB data sources outside of the instance of SQL Server. Typically linked servers are configured to enable the Database Engine to execute a Transact-SQL statement that includes tables in another instance of SQL Server, or another database product such as Oracle. Linked servers offer the following advantages:

• The ability to access data from outside of SQL Server.
• The ability to issue distributed queries, updates, commands, and transactions on heterogeneous data sources across the enterprise.
• The ability to address diverse data sources similarly.

Invoking a linked server stored procedure from a WebSphere environment using Global Transaction and REPEATABLE_READ

Assuming the following scenario:

Figure 1 : Global Transaction Example with WebSphere as the transaction manager

The above figure illustrates the following:

• WebSphere is responsible for managing the transactional operations of transactional components (components involved in a transaction).
• The TransactionManagementType annotation at the class level indicates that BeanA has CMT transaction.
• The TransactionAttributeType annotation at the class level indicates that BeanA has the SUPPORTS value for all methods. This means:
• The bean participates in a running transaction but does not require it.
• If there is no transaction, the method executes without a transaction.
• The TransactionAttributeType annotation at the methodA in BeanA, overwrites the class level declaration and indicates that this specific method has REQUIRED transaction value. This means:
• Methods must always execute in a transaction context.
• If there is a transaction already running, the bean participates in that transaction.
• If there is no transaction, the EJB container starts a transaction on behalf of the bean.
• The methodA of BeanA runs in a propagated transaction if present, otherwise in a new global transaction.
• The changes that are made by the service invoked to the transactional resources are persisted when transaction of BeanA commits.
• If one of the operations within each transaction fails, then all of them are rolled-back so that the application is returned to its prior state.
• The JDBC driver implementation class used is the com.microsoft.sqlserver.jdbc.SQLServerConnectionPoolDataSource (i.e One-Phase Transaction Support - 1PC).
• The default isolation level in WebSphere is TRANSACTION_REPEATABLE_READ for all databases that support the RepeatableRead isolation level. This means:
• repeated reads of the database result with the same data values.
• used when we have to update the database records often.
• this prevents data from being modified by other concurrent transactions.
• The SQL query is a distributed query executed through a linked server. This means:
• SQL Server parses the command and sends requests to OLE DB

The result of the above configuration is the following exception:

Two alternative configurations are illustrated below in order to resolve the above problem.

Invoking a linked server stored procedure from a WebSphere environment using Local transaction containment

Figure 2 : Local transaction containment

The above figure illustrates the following:

• The TransactionAttributeType annotation at the methodA in BeanA, overwrites the class level declaration and indicates that this specific method has NOT_SUPPORTED transaction value. This means:
• Method cannot be involved in a transaction.
• Method will always execute outside the transaction context. 
• If there is a transaction running, it is suspended and will be resumed after the end of those methods.
• In the deployment descriptor of the BeanA, in the Local Transactions section, the Resolver attribute is set to Application and the Unresolved action attribute is set to Rollback. This means:
• The component processing occurs within a WebSphere local transaction containment  that is managed by the application.
• The Resolver option define how the local transaction is to be resolved before the local transaction context ends. The application option indicates that is the responsibility of the application to commit or roll back transactions. 
• The unresolver-action options are commit or rollback and specify the action the container will take if the resources are uncommitted by an application in a local transaction.
• The code is responsible for forcing commitment by using the javax.resource.cci.LocalTransaction.begin() and javax.resource.cci.LocalTransaction.commit() i.e. code delineates the transaction (another option is to have setAutocommit(true) - each statement will run in its own transaction).
• In case the code does javax.resource.cci.LocalTransaction.begin() but does not call the javax.resource.cci.LocalTransaction.commit()  or sets setAutocommit(false) after obtaining a JDBC connection and not add any code to control the transaction, then the state of the transaction at the end of the method would be ambiguous. To resolve this problem the  unresolver-action option is set. 

Invoking a linked server stored procedure from a WebSphere environment using TRANSACTION_READ_COMMITTED

Figure 3 : READ COMMITTED Isolation level

The above figure illustrates the following:

• The isolation level in WebSphere is TRANSACTION_READ_COMMITTED. This means:
• transactions will read committed data only
• data read is always consistent.
• used for report-generating programs that use the current state of the database at the time of report generation.

SCA Java Component Configuration

The following configuration should be used if a Java SCA component invokes the SQL Server Linked Server (instead of the Session Bean described in the above examples).

Figure 4 : Java SCA Component Configuration

References

[1] The Microsoft Developer Network - “Linked Servers (Database Engine)"  Date of access: June 2013. http://msdn.microsoft.com/en-us/library/ms188279.aspx

[2] IBM Redbooks - Ueli Wahli, Giuseppe Bottura, Jacek Laskowski, Nidhi Singh - “WebSphere Application Server Version 6.1 Feature Pack for EJB 3.0." September 2008. Date of access: April 2013. http://www.redbooks.ibm.com/abstracts/sg247611.html

[3] IBM Support Portal - “Changing the default isolation level for non-CMP applications and describing how to do so using a new custom property webSphereDefaultIsolationLevel"  Date of access: June 2013. http://www-304.ibm.com/support/docview.wss?uid=swg21224492

[4] IBM Information Center - “Local transaction containment"  Date of access: June 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Frjta_useltran.html

[5] IBM Information Center - "EJB 3.0 application bindings overview" Date of access: June 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Fcejb_bindingsejbfp.html

JMX Programs

Using JMX interface to manage SIB queues

A service integration bus provides location transparent messaging, meaning that an application can connect to any bus member and send to (or receive from) a destination deployed to any bus member. The destination does not need to be deployed to the same bus member that the application connected to,

the bus routes the message to (or from) the destination. This section will describe how to use the Java application programming interfaces (APIs) in order to create a tool to navigate through the messaging components of a SIBus.

The code below will show how to view the current depth and message contents of all queues from all messaging engines. One can use this code via the AdminClient API (if the code will be used as a standalone) or via the MBeanServer API (if the code will be run inside a WebSphere JVM).

For a standalone application the first step is to create the AdminClient class in order to provide the client side APIs for the remote AdminService.  This is shown below:

If the code will run inside a WebSphere JVM (i.e. as an MBean) then the following code should be used:

It should be noted in this regard that the above code should be run inside the Messaging infrastructure JVM if a deployment environment is used (not a single server in a single cell).

The next step is to find all SIBMessagingEngines:

Then for each SIBMessagingEngine we are looking for all attached SIBQueuePoints:

For each SIBQueuePoint we are quering for the following attributes:

• depth
• state
• id
• highMessageThreshold
• sendAllowed

If the current depth of this SIBQueuePoint is greater than zero (could be greater than a specific threshold if you need to implement a simple monitor solution) then we get the queue messages:

For each message within a queue we print message attributes:

Then we print the queue message details:

Finally we print the message contents:

References

[1] IBM Information Center - “WebSphere Application Server Version 7.0 -Interface AdminClient"  Date of access: June 2013. http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.javadoc.doc%2Fweb%2Fapidocs%2Fcom%2Fibm%2Fwebsphere%2Fmanagement%2FAdminClient.html