Why you should monitor the SAP XI/PI adapter engine

1. Architecture overview

XI older than 7.3 are dual stack systems. The adapter engine with most of the adapters are running on the Java stack (JMS, HTTP, SOAP, …) while the adapters to connect XI to SAP systems are running on the abap stack.

Beside the adapter engine you find also the SLD (system landscape directory) and the RTW (runtime workbench) on the Java side.

Java and Abap stack are communicating with each other using technial users. If those user get locked (for example due to some security features which lock users if unauthorized access was detected with wrong password) the internal communication between the java and abap stack is broken.

The Java and the Abap stack have it’s own separte database and it is not possible to access the java database from abap directly or vice versa.

2. Implications of broken internal communication

If for a certain time the internal communication was broken, some of the messages may not be processed correctly. I detected such situation for example for inbound wholesaler orders.

If you are processing a lot messages it is easy to find out which orders are not processed correctly. Depending on the used adapter you may have to look in every message payload via the runtime workbench.

3. Automatic Adapter Engine Message Monitoring

To make the monitoring easier I have implemented a Java Enterprise application (EAR/WAR) for XI which has access to the java database where the runtime workbench messages are stored. By providing a webbased technical interface to access the java database tables, all messages can be continuously downloaded and processed in a external monitoring tool so that not processed order numbers can be displayed as alerts to the responible persons. The relevant database table is: XI_AF_MSG

Attention: Before you can access the java database of your XI system, you have to define some datasource in Visual Administrator.

If you want to centralize your data in a data lake it is now easy to write the data flow to hadoop where XI data and ERP/CRP/BW data can be stored so that you have the possibility to analize them together!

 

 

Google trends analysis of ESB products

For enterprises with many interfaces it is very helpful to make use of ESB (enterprise servicebus) systems. A quick analysis with google trends for the most used ESB systems leads to the following diagramm:

Apache Camel is integrated in Apache Servicemix which is based on OSGi so that hot deployment and concurrent use of different component versions at the same time is no problem.

Usually such systems make extensive use of MOM (message oriented middleware) to be able to get a loose coupling between the processing steps (especially input and output). As a consequence such systems generate a huge number of threads in the system. In SMP or CMP systems it is important to check that the JVM is not just using on CPU/core at a time to get best performance results.

As it is described very nicely in the blog Parallel Processing and Multi-Core Utilization with Java it is impotant to not just use

new Thread( )

but instead make use of the methods of the standard jdk class

java.util.concurrent.Executors

especially of its method

newFixedThreadPool(int nThreads)

In Apache Servicemix / Camel there are possiblities to make use of all CPU cores in several ways:

  • Thread pool profiles (see also: Camel Threading Model)
  • Apache Servicemix Clustering (in this case the ESB can even be spread over several different servers)

 

I replaced for a company SAP XI by Apache Servicemix with the effect of several advantages:

  • Even the core of the ESB is open source so that we had a chance to find and correct bugs in the core systems by our own.
  • Apache Servicemix / Camel allows to create interface OSGi bundles in a very flexible and lightweight way which leads to a decrease of the development efforts
  • Because Apache Servicemix is using the newest java version (which is not the case in all SAP XI versions – some still use JDK 1.4) it is possible to make use of the huge amount of available java libraries and packages available on the market (many are free)
  • Many of the Enterprise Integration Patterns described in the book with the same title are already ready to use as features in Camel.

Because such system make use of message queue systems the availability of such systems is very important. Really satisfying HA solutions for JMS message queue systems are not that easy to set up. To get at least rid of the problem to loose messages it is quite easy to set up several stand alone MQ Servers and configure the client with a HA URL so that in case of a problem with one server the client will deliver the message to another mq server.

If someone has a good solution for the synchronization problem between the mq servers please let me know.

martin[at]dres-menzel[DOT]de