An introduction to the Oracle Service Bus

We are in the process of designing a new system for a telecommunication provider where we have looked at the Oracle Service Bus (OSB) to be used as the enterprise service bus. One of the first plus points for me was the amazing tooling support it encompasses. Oracle has integrated all their enterprise integration software stack into a cohesive whole by bundling it up as the Oracle SOA Suite. In this article, the focus would be on the Oracle OSB 11g which is part of the Oracle SOA Suite 11g. There are considerable changes that has been done with the new Oracle SOA Suite 12c which we will not delve into in this article. However, one feature I love about the new Oracle SOA Suite 12c is the fact that the developers can use JDeveloper to develop BPEL(Business process execution language) and OSB code in one IDE(Integrated Development Environment).

Couple of main components one needs to be aware of with the OSB is as follows;

Proxy Service

A proxy service as its name implies, is a service that is hosted to the external parties which acts as a facade for an internal service. By having a proxy service, you have more control over the changes in your internal services as the proxy service can do the required transformations if your internal services ever change.

Business Service

A business service, in terms of the OSB, represents an internal application service. It can be a WebService, JMS queue/topic, REST service, FTP service and many more. The business service will encompass the functionality to call the actual service.

So the scenario we will focus on this article is as follows;

1. We have an internal service that returns subscriber information if the user passes in either the MSISDN or the SIM Card number and depending on the input, data will be fetched and returned.
2. This service will have to be exposed to the external party in a more meaningful manner by making use of a proxy service.

The sample project can be downloaded here.

First of all, we create the business service which will act as the facade to our internal service. In your OSB project,  create the following four folders;

• proxy
• business
• transformation
• wsdl

Then we need to copy the internal service WSDL and the proxy service WSDL created for this example into the “wsdl” folder.

Configuring the business service

Right click on the “business” folder and select New->Business Service. When the business service is created, you will then first be presented with the “General” tab. In this we do the following;

• Select “WSDL Web Service” and click on browser. Then select “Browse”, select the WSDL file and you will be presented with two options. Select the one ending with “(port).

• Then go the “Transport” tab and change the URI as http://localhost:8088/mockInstalledBaseSubscriberClassificationQueryWSServiceSoapBinding. This is because we will use the SOAPUI mock service feature to test this out and the URI represents the mock service endpoint of SOAPUI for the service represented by the WSDL. 

• The SOAPUI project use for this example can be downloaded from here.

That is all we need to do to configure our business service. Then we move onto our proxy service where all the action takes place.

Configuring the proxy service

• Right click on the “proxy” folder created, select New->Proxy Service and provide a valid name. 
• In the “General” tab, select “WSDL Web Service” and click on browse.
• Now in the proxy service, you need to select the proxy WSDL file we have created which will be exposed to the external clients.

• Go to the “Message Flow” tab. In that tab, first drag a “Route” element from the “Design Palette” on the right side. 
• Afterwards, drag a “Routing” element into the “Route” element.
• Click on the “Routing” element and in the bottom pane, go into the “Properties” tab where you will provide the business service that this proxy service will access and the operation name.

• The result will be as follows;

• Then drag a “Replace” action into the “Request Action” component.

• Before we provide the information on the “Properties” tab for the “Replace” action, we need to create the XQuery transformation files which will map the proxy service request to the business service request and then the business service response back to the proxy service response.
• Right click on the “transformation” folder and select New->XQuery Transformation. Enter a valid name. This should be done for both the request and response transformation files.
• The request transformation file used is as follows;

 
(:: pragma bea:global-element-parameter parameter="$fetchSubscriber1" element="ns2:FetchSubscriber" location="../wsdl/SubscriberProxyService.wsdl" ::)
(:: pragma bea:local-element-return type="ns1:InstalledBaseSubscriberClassificationQuery/ns0:InstalledBaseSubscriberClassificationQuery" location="../wsdl/subscriber_classfication.wsdl" ::)

declare namespace ns2 = "http://www.example.org/SubscriberProxyService/";
declare namespace ns1 = "http://www.openuri.org/";
declare namespace ns0 = "http://mtnsa.co.za/si/IB/IBSubscriberClassificationQuery";
declare namespace xf = "http://tempuri.org/OSB%20training%201/transformation/subscriberrequest/";

declare function xf:subscriberrequest($fetchSubscriber1 as element(ns2:FetchSubscriber))
    as element() {
     <ns1:InstalledBaseSubscriberClassificationQuery>
        <ns0:InstalledBaseSubscriberClassificationQuery>
            <ns0:Request>
              
                    {
                        if (data($fetchSubscriber1/EquipmentType) = "MSISDN") then
                           <ns0:MSISDN>  { (data($fetchSubscriber1/EquipmentValue))}</ns0:MSISDN>
                        else 
                           <ns0:SIMCard> { data($fetchSubscriber1/EquipmentValue)}</ns0:SIMCard>
                    }

            </ns0:Request>
        </ns0:InstalledBaseSubscriberClassificationQuery>
        </ns1:InstalledBaseSubscriberClassificationQuery>
};

declare variable $fetchSubscriber1 as element(ns2:FetchSubscriber) external;

xf:subscriberrequest($fetchSubscriber1)

Here as you can see, we check if the equipment type is equal to “MSISDN” and then set the appropriate element on the business service.

• The response transformation file used is as follows;

 
(:: pragma bea:global-element-parameter parameter="$installedBaseSubscriberClassificationQueryResponse1" element="ns1:InstalledBaseSubscriberClassificationQueryResponse" location="../wsdl/subscriber_classfication.wsdl" ::)
(:: pragma bea:global-element-return element="ns2:FetchSubscriberResponse" location="../wsdl/SubscriberProxyService.wsdl" ::)

declare namespace ns2 = "http://www.example.org/SubscriberProxyService/";
declare namespace ns1 = "http://www.openuri.org/";
declare namespace ns0 = "http://mtnsa.co.za/si/IB/IBSubscriberClassificationQuery";
declare namespace xf = "http://tempuri.org/OSB%20training%201/transformation/subscriberresponse/";

declare function xf:subscriberresponse($installedBaseSubscriberClassificationQueryResponse1 as element(ns1:InstalledBaseSubscriberClassificationQueryResponse))
    as element(ns2:FetchSubscriberResponse) {
        <ns2:FetchSubscriberResponse>
            <TradeCustomerCode>{ data($installedBaseSubscriberClassificationQueryResponse1/ns0:InstalledBaseSubscriberClassificationQuery/ns0:Response/ns0:Subscriber/@ServiceProviderCode) }</TradeCustomerCode>
            <PackageCode>{ data($installedBaseSubscriberClassificationQueryResponse1/ns0:InstalledBaseSubscriberClassificationQuery/ns0:Response/ns0:Subscriber/ns0:Package/@ProductCode) }</PackageCode>
            <PaymentOption>{ data($installedBaseSubscriberClassificationQueryResponse1/ns0:InstalledBaseSubscriberClassificationQuery/ns0:Response/ns0:Subscriber/@PaymentOption) }</PaymentOption>
        </ns2:FetchSubscriberResponse>
};

declare variable $installedBaseSubscriberClassificationQueryResponse1 as element(ns1:InstalledBaseSubscriberClassificationQueryResponse) external;

xf:subscriberresponse($installedBaseSubscriberClassificationQueryResponse1)

This is a simple transformation where we map the response elements to the proxy response elements as required.

Now we move back to our proxy service, click on the “Replace” action, go to the “Properties” tab.

• In the “In Variable” insert the value “body”.
• Click on the “Expression” link. Go to the “XQuery Resources” tab, click on “Browse” and select the request transformation file.
• In the “Variable Structures” component on the right side, expand the “body” element, and then select the request element and drag and drop it into the “Binding” text box as follows;

• Then select “OK” which will take you back to the “Properties” tab.
• Select “Replace node contents” radio button. The end result will look as follows;

• Now let us drag and drop a “Replace” action to the “Response Action” component.
• Same as before, select the response transformation “$body/ins:InstalledBaseSubscriberClassificationQueryResponse”.
• You will now get an error stating that the “ins” namespace is not recognized.
• In order to resolve that, in the same “Properties” tab, select the tab “Namespaces” and click on add. Enter the prefix as “ins” and the URI as “http://www.openuri.org/”. 

And that is it. Now we can test out the functionality. Before you do, remember to first start the mock service created on SOAP UI. 

Now let us log into the service bus console, go to the proxy service and launch the test console. This is the result that I got by running a sample;

You can see a trace of what exactly happened if you go further down on the same screen within the “Invocation Trace” section. The request and response transformation done by the OSB can be seen as follows;

That ends our introduction to the Oracle Service Bus. If you do have any queries on the same, please do not hesitate to leave a comment by and I will respond to it as soon as possible. Also, if there are any areas of improvement you may see, kindly leave your feedback as well which is always much appreciated.

Performance monitoring and profiling – Part 1

Performance monitoring and profiling are two different aspects. The former takes a more of a proactive measure whereas the latter is a reactive approach. In my experience, performance is an afterthought in most cases rather than being built into the software development life cycle. I see the same thing in most cases when it comes to security as well but let us not go there today as that lends itself to a different post.

Performance monitoring

In order to monitor your application, you first need to understand what performance aspects the end users of your application expects. This would include throughput, response time, uptime etc. After you collate these information, the next step would be to create scripts for load testing your application. Coming primarily from a Java background, I normally use JMeter which comes with a user friends UI to create load testing depending on your use case be it database, Web Service, JMS etc.

So you have your load testing scripts and it is running fine. What next?

Well now that you have the load scripts in place, the next step would be note down what parameters you should monitor. Generally you would start out monitoring the CPU and Memory usage while your load test is in progress.

When you monitor CPU, you would be looking at the CPU usage as the application is being load tested. If there is high CPU usage, you need to further drill down to find out what issue might cause this. If there is low CPU usage, then probably you can try and increase the load to see what the optimum load that uses efficient CPU.

If you are on a Windows machine, by default you will open the task manager and go into the Performance tab in order to find out what your CPU is up to. A snippet of mine is as follows;

I’m running on a Core i5 with 8GB of RAM. In the upper right hand corner you can see the CPU usage history. The four boxes here represent the four CPUs. You will get the same value if you run Runtime.getRuntime().availableProcessors() within a Java program. 

By default it only shows the total CPU time. But you can easily add the kernel CPU time by clicking onn View-> Show Kernal Times. You will then be presented with the following graphs with a red line;

The space between the red line and the green line is the user CPU time. So what is the difference between the user and the kernel CPU time you might ask.

User CPU Time : The amount of time the CPU spends on running your application code.

Kernel CPU Time : The amount of time the CPU spends on operating system related activities. For example if your application does a lot of reads/writes to the disk, you will see high kernel CPU as it requires the operating system related functionality to be invoked.

But what if you needed further information on what your CPU is doing at any given moment of time? PerfMon to the rescue. Open up a Run tab and type in perfmon and you will be presented with the following screen;

You can add any counter which you are interested in monitoring by right clicking on the graph area and selecting the Add Counters menu option. If you want to monitor the user CPU and kernel CPU, select the Performance object and under that you will find the User time and Privileged time (kernel CPU time) counters.

Another key indicator to watch out for when monitoring CPU is the CPU schedule run queue. This is where all light weight processes needed to run are queued up waiting for CPU. As a rule of thumb, if your run queue depth is more than four times the available virtual processors on your machine, then you need to investigate your application further more to see what could be done. In the era of high memory machines, people often overlook the fact of using and writing efficient algorithms and data structures as there is plenty of memory available on high end servers. But the issue here is that many a times CPU is limited. If your algorithms/data structures do not scale well with added load, you will end up over loading your CPU with the only alternative being is to scale your application into different servers. So if you are having issues with the scheduler run queue depth, it is always best to see the possibility of writing certain sections of your application code in a more efficient manner so as to utilize your CPU better. The run queue depth can also be monitored using Perfmon by selecting Add Counters->System-> Processor Queue Length.

Now that we have covered some of the important factors to look into in terms of CPU when monitoring your application, in the next post we will look at what aspects we need to consider when monitoring the memory usage.

Please do leave by your comments and suggestions which is as always much appreciated as I love learning from the experience of others which I deem as invaluable.

SuperMan bound by Java Monitors

Photo Taken from : http://goo.gl/2B5Sj

Its a dark time in the life of Super Man. Jor-El wants him to go on a voyage to prepare him for his ultimate destiny. Yet the Earth is faced with dooms-day and the Justice League needs their Man of Steel in action to save the world. But you cant do both at the same time since we have just one SuperMan. Also he cannot fight dooms day without first fulfilling his destiny and realizing his true powers. How do we call upon Superman without making the man go bonkers on what to do. This should be done in an orderly manner where one has to wait until the voyage is done.

We will make use of Java Monitors to help SuperMan listen to his Kryptonian father as well as come back in time to save the world from dooms day. First of all we define the Man of Steel;

/**
 * The awesome kryptonian man is represented by this class
 * 
 * @author Dinuka Arseculeratne
 *
 */
public class SuperMan {

 private boolean onVoyage = false;

 /**
  * Schedule a voyage for Superman. Note that this method first checks whether he is
  * already on a voyage, and if so calls the wait() method to hault the current thread
  * until notify is called and onVoyage is set to false.
  */
 public synchronized void goOnVoyage() {

  if (onVoyage) {
   try {
    System.out.println("SuperMan is already on a voyage. Please wait until he returns from his quest.");
    wait();
    System.out.println("His goyage is over, time for him to go on a new voyage....");
   } catch (InterruptedException e) {
    System.out.println(" I am SuperMan, i do not handle these petty exceptions");
   }

  }
  onVoyage = true;
  notify();

 }

 /**
  * This method calls Superman back from his current voyage. Again the method
  * checks whether Super man is not already on a voyage. If so the current thread is
  * Halted until he is schedule to go on a voyage because he needs to be on a voyage
  * to be called back in the first place.
  */
 public synchronized void returnFromVoyage() {

  if (!onVoyage) {
   try {
    System.out.println("SuperMan is not yet on a voyage. Please Wait.");
    wait();
    System.out.println("Great he has gone on a voyage, time to call him back!!");
   } catch (InterruptedException e) {
    System.out.println(" I am SuperMan, i do not handle these petty exceptions");
   }
  }
  onVoyage = false;
  notify();
 }
}

So we have defined SuperMan. Note that he has two methods defined. One which allows him to go on a voyage and another to call him back from his current voyage. As you can see SuperMan does not handle exceptions because, well.......... He is SuperMan and he is the exception :). You can see that before each call we check the boolean indicating whether he is on a voyage or not and depending on the method called, the wait() of the Object is called in order to halt the current thread that is calling the method until notify() is called by the thread that is currently operating on the object. Note that wait() and notify() should be called inside a synchronized method or block for it to work accurately. Because you first need to acquire a lock in order to halt or notify it.

Getting back to the previous issue, we know that both the Justice League and Jor-El need SuperMan but for different purposes. Lets see how this battle unravels with the following code snippet;

public class Test {

 public static void main(String[] args) {
  SuperMan superMan = new SuperMan();
  
  JusticeLeague justiceLeague = new JusticeLeague(superMan);
  justiceLeague.start();
  
  JorEl jorEl = new JorEl(superMan);
  jorEl.start();
  
 }

 

}

class JusticeLeague extends Thread{
 
 private SuperMan superMan = null;
 
 public JusticeLeague(SuperMan superMan)
 {
  this.superMan = superMan;
 }
 
 @Override
 public void run() {
  superMan.returnFromVoyage();
 }
}

class JorEl extends Thread{
 
 private SuperMan superMan = null;
 public JorEl(SuperMan superMan)
 {
  this.superMan = superMan;
 }
 
 @Override
 public void run() {
  superMan.goOnVoyage();
 }
 
}

Note that here we have JorEl and the JusticeLeagure operating on two different threads trying to access SuperMan concurrently. As you can see from our main method, the JusticeLeague wants to call back SuperMan in order to save the world. But fortunately he is not yet on a voyage so its illegal to ask him to return. Then comes JorEl asking his son to go on a voyage to fulfill his true destiny. It is only after this voyage that he can return to save planet Earth. If you run this now you can see that the JusticeLeague thread is halted until SuperMan goes on the voyage and notify is called. Just for fun try to comment out the notify() method and you will see the application will hang because now one thread will wait indefinitely until it is notified of the completion of the process.

If not for Java Monitors, SuperMan would have failed since he would have gone to face doomsday without first going on his voyage and fulfilling his destiny. And Java saves the world again.

Note : The story is fictional yet Java Monitors are real


Thank you for reading everyone. And have a great day ahead. If you feel like it, please do leave by a comment. Cheers!!

Locking with a semaphore : An example

Concurrency is one aspect that brings along interesting challenges along with it. If not correctly handled, it brings about race conditions that will baffle people because those issues just pop up from time to time and work flawlessly sometimes.

The Java language gives many ways of handling race conditions when dealing with concurrent threads accessing a common resource. Some include;

1. Using the volatile keyword
2. Using classes available in java.util.concurrent and java.util.concurrent.atomic 
3. Synchronized blocks
4. Using a Semaphore

Of course there might be many more that i might not be aware of. For today, the example i want to show you all is the one using a Semaphore. This was introduced from JDK 1.5, and provides the developer with the ability to acquire and release locks in a seamless way. Also the example i will be showing is a hypothetical scenario which i used just to depict what can be achieved using a semaphore and therefore please do not look at the intrinsic details of the code :)..

So the scenario as such, there is an in-memory cache holding objects of type "Person". Users can insert and retrieve records using the cache. The issue here is we are going to control concurrent access to our in-memory cache using semaphores. Now i do not want to bore you with more text so lets get to business and show some code;

 
import java.util.concurrent.Semaphore;

/**
 * This class will allow thread to acquire and release locks as required
 * 
 * @author dinuka.arseculeratne
 * 
 */
public class PersonLock {

 /**
  * We do not want multiple lock objects lying around so we make ths class
  * singleton
  */
 private PersonLock() {

 }

 /**
  * Bill Pugh's way of lazy initializing the singleton instance
  * 
  * @author dinuka.arseculeratne
  * 
  */
 private static class SingletonHolder {
  public static final PersonLock INSTANCE = new PersonLock();
 }

 /**
  * Use this method to get a reference to the singleton instance of
  * {@link PersonLock}
  * 
  * @return the singleton instance
  */
 public static PersonLock getInstance() {
  return SingletonHolder.INSTANCE;
 }

 /**
  * In this sample, we allow only one thread at at time to update the cache
  * in order to maintain consistency
  */
 private Semaphore writeLock = new Semaphore(1);

 /**
  * We allow 10 concurrent threads to access the cache at any given time
  */
 private Semaphore readLock = new Semaphore(10);

 public void getWriteLock() throws InterruptedException {
  writeLock.acquire();
 }

 public void releaseWriteLock() {
  writeLock.release();
 }

 public void getReadLock() throws InterruptedException {
  readLock.acquire();
 }

 public void releaseReadLock() {
  readLock.release();
 }
}

This class will handle the process of obtaining and releasing locks required to make our cache thread safe. I have used two separate locks here for reading and writing. The rationale behind this was to allow users to read data though it might be stale at the time of reading.

Note that i have used "ten" here which denotes that ten thread can simultaneously obtain locks and access the cache for read purposes. Next up you can see in the write lock, i have used the "one" which signifies that only one thread can access the cache at a time to put items to it. This is important in order to maintain consistency within the cache. That is, i do not want multiple threads trying to insert items to the map which would result in unpredictable behavior ( at least in some instances). There are mainly two ways by which you can acquire a lock using a semaphore.

 1. acquire() : is a blocking call which waits until the lock is released or the thread is interrupted
2.  tryAcquire() : is a non-blocking call which will return immediately and return true or false signifying whether the lock was obtained or not.

Here i have used the blocking acquire call because i want the thread to wait until the lock is available. Of course this will depend on your use case. You can also define a timeout period in the tryAcquire() method so  that the thread will not wait indefinitely for a lock.

Next up the storage class below shows how i have used the lock class to insert and read data within the cache.

 
import java.util.HashMap;
import java.util.Map;

/**
 * A mock storage to hold the person objects in a map
 * 
 * @author dinuka.arseculeratne
 * 
 */
public class PersonStorage {

 private Map<Integer, Person> personCache = new HashMap<Integer, Person>();

 private int counter = 0;

 /**
  * This class is made singleton and hence the constructor is made private
  */
 private PersonStorage() {

 }

 /**
  * Bill Pugh's way of lazy initializing the singleton instance
  * 
  * @author dinuka.arseculeratne
  * 
  */
 private static final class SingletonHolder {
  public static final PersonStorage INSTANCE = new PersonStorage();
 }
 
 /**
  * Use this method to get a reference to the singleton instance of
  * {@link PersonStorage}
  * 
  * @return the singleton instance
  */
 public static PersonStorage getInstance()
 {
  return SingletonHolder.INSTANCE;
 }

 /**
  * Inserts the person into the map. Note that we use defensive copying so
  * that even if the client changes the object later on, those changes will
  * not be reflected in the object within the map
  * 
  * @param person
  *            the instance of {@link Person} to be inserted
  * @return the key which signifies the location of the person object
  * @throws InterruptedException
  */
 public int putPerson(Person person) throws InterruptedException {
  
  Person copyPerson = person.copyPerson();
  personCache.put(++counter, copyPerson);
  
  return counter;
 }

 /**
  * Here as well we use defensive copying so that the value of the object
  * reference within the map is not passed in to the calling party.
  * 
  * @param id
  *            the id representing the location of the object within the map
  * @return the instance of the {@link Person} represented by the key passed
  *         in
  * @throws InterruptedException
  */
 public Person retrievePerson(int id) throws InterruptedException {
  PersonLock.getInstance().getReadLock();
  if (!personCache.containsKey(id)) {
   throw new RuntimeException("Key is not found");
  }
  PersonLock.getInstance().releaseReadLock();
  return personCache.get(id).copyPerson();
 }

}

Obviously the code will work without the locks as well, but the issue is that the application will be inconsistent and provide different results at each run. This is not something you want your application to do and hence with locks you guarantee your application works consistently.

And lastly a small test class to show how it will behave; not that in here we obtain the lock before calling the putPerson() method and release the lock within the finally block in order to guarantee the release of the lock.

 
/**
 * A test class to demonstrate the locking at work
 * 
 * @author dinuka.arseculeratne
 * 
 */
public class TestLock {

 public static void main(String[] args) throws InterruptedException {

  Thread t1 = new Thread(new Runnable() {

   @Override
   public void run() {
    
    Person p1 = new Person(1L, "Test1", "XYZ");
    try {
    PersonLock.getInstance().getWriteLock();
PersonStorage.getInstance().putPerson(p1);
    } catch (InterruptedException e) {
     // Exception handling need to be done
     e.printStackTrace();
    }
   finally{
          PersonLock.getInstance().releaseWriteLock();
    }
   }
  });

  Thread t2 = new Thread(new Runnable() {

   @Override
   public void run() {
    
    Person p2 = new Person(2L, "Test123", "ABC");

    try {
     PersonLock.getInstance().getWriteLock();

     PersonStorage.getInstance().putPerson(p2);
    } catch (InterruptedException e) {
     // Exception handling need to be done
    }
 finally{
          PersonLock.getInstance().releaseWriteLock();
    }
    
   }
  });

  t1.start();
  t2.start();

  System.out.println(PersonStorage.getInstance().retrievePerson(2));
 }
}

That concludes my short introduction to using Sempahores to make your code thread safe.For anyone who wants to play around with the code, you can obtain it from here. Try to remove the locks in the Storage class and see how it behaves on each run. You will see possible race conditions taking place.

Appreciate your comments and feedback on the same as always and thank you for reading.

Cheers....