Sunday, October 23, 2016

Adding a new xml element to the payload - WSO2 ESB

In this blog post I'm going to explain how we can insert a single new xml element in to the payload in WSO2 ESB.

Why not payloadFactory Mediator ?
If you are familiar with WSO2 ESB you may know that if we want to change the entire payload into a different payload or build a new payload by extracting properties using the existing payload, we can use payloadFactory mediator. But for this requirement that I'm going to describe, payload factory mediator will not be the ideal mediator due to different reasons. One major reason is, if the current payload is bit lengthy one, you will have to build the other parts of the payload even though you don't need to change them at all.

If not payloadFactory then what?
It will be very convenient to get this requirement done using Enrich Mediator. Enrich Mediator will take your desired OMElement (xml element) using the configuration that you state in the source section and then will insert it to the target location you state.

I will explain the configuration using a sample usecase. Say I receive some request into ESB and depending on some properties in the payload I want to set some element in the payload. For an example assume I need to insert a new element into the payload if the received payload is not a valid payload.

In the above example I have set the incoming request to ESB in to a property (INCOMING_REQUEST) during a previous mediation. And hence using the first enrich mediator I am replacing the body of the current payload using that property.
So the second mediator is the one actually does the job.
It will take the OMElement given under source which is
and will insert as a child element of /BookingRequest/Booking xpath location in the INCOMING_REQUEST xml.

Incoming Request

After enrich mediator
In this example I have modified the payload by adding only one xml element. Even if you need to add more elements also you can follow the same approach.

Many thanks to Rajith from WSO2 for the tip.

Thursday, February 20, 2014

Crowning of WSO2 ESB as "Usain Bolt" in the ESB space !!!

With the latest round of performance testing results published for WSO2 ESB 4.8.1, it is unveiled that the fastest ESB in the ESB space is WSO2 ESB. You can find the complete article here.

The tests has been conducted against number of leading open source ESB's and it is proven that WSO2 ESB surpasses rest in almost all the scenarios and crowned as the No 1.


The table and the graph below shows the summarized results.

WSO2 ESB 4.8.1 has continued to outperform all other compared ESBs in almost all scenarios !!!


Tuesday, February 11, 2014

WSO2 User Engagement Server 1.5.0 -Milestone-1 Released !

Let me start this post with a saying.
                 "Data is worthless if you don't communicate It correctly"

I'm sure everybody reading this post will agree on the above saying. WSO2 User Engagement Server or simply known as WSO2 UES was released with the aim of providing a set of tools that will ease your data visualization.

If it is the first time that you are hearing about WSO2 UES, I would suggest you to peep a bit into WSO2 documentation about UES 1.0.0 here. Also if you are a developer who is interested in how the things have been done, UES git repo is the place for you.

UES 1.0.0 release was done sometime back and recently we released the milestone one of version 1.5.0 of the product. Milestone 1 mainly contains the initial implementation of Gadget generation tool which will simply take user through a wizard and will draw a graphing gadget visualizing his data.

Are you interested ? Let me walk you through the lanes of UES. First of all download the WSO2 UES 1.5.0 Milestone 1 pack from here.

   PRODUCT_HOME that I will be referring from here on-wards is the directory where you unzipped the downloaded pack. 

OK. Now we are ready to make our hands dirty with our data :)

Place your RDBMS driver at <PRODUCT_HOME>/repository/components/lib/
For an example if your data resides in a mysql database, place the mysql driver at the above location.
(IMPORTANT : If you have started the server already, you have to restart after placing the driver.)

Start the downloaded pack as you start any of the WSO2 products.
  • On Windows: <PRODUCT_HOME>\bin\wso2server.bat
  • On Linux/Solaris: sh <PRODUCT_HOME>/bin/

You can find more information on how to start the product here.

Go to the browser and open the link http://localhost:9763/portal and login.
You will see a page similar to following.

Find the sign in button on the top right corner of the page. For the moment let's login with following credentials.
                 Username : admin
                 Password: admin
Please note that if you need to create a new user and login with that user, you can do so.

Once you logged in, you will be redirected to the dashboard creation page.

Click Create on top left corner and select Dashboard.

Give a preferred name to the dashboard and select a layout. Then click on Create new app.
Layout defines the arrangement of gadgets in your dashboard.

Then you will be redirected to the dashboard designer window where you can add gadgets to the created dashboard.

Click on "Add Gadget" button on any of the blocks visible and follow the wizard.

Select your preferred data-source and click Next.
    Since this is milestone 1 of the tool, we have completed the implementation only for RDBMS. So please bear with us, we will support rest of the data-source types in coming releases.

Enter the connection details of your data-source and click Next. 
See the image below where I have entered mysql connection details of the database where I want to visualize my data. My database is "std" and both the username and password for the database is "test". 

Also note that the information of the data-source needs to be of the driver that you placed in the lib folder as stated in first step.
Ex: If you place mysql driver in lib folder then connection URL, username and password fields should be of a mysql database.

Enter your SQL query to get the data from the respective table of the database and Click Next.
This SQL query can be any complex query which return the columns that you want to visualize in your gadget. 
NOTE : Do not add the semi-colon at the end of the query.

Go to "Store Items" and click on your preferred graph type.
   As I mentioned above, at the moment we only support the chart types that I have caged with a rectangle  (Line chart and Bar chart). Await for more chart types in future.

Let's say I select the chart type as "Bar Chart".

Do the data mapping of the columns as needed and click Next.

  • Depending on the SQL query that you submitted in the previous step, the columns retrieved from that particular query will be visible as drop down for each axis. Select the column names from them.
  • Give the axis label names that you want in your chart.
  • You can set your chart to be updated at a given interval by setting the Update interval value. Note that update interval is in seconds.
  • Click on the + mark (shown in the cage), if you want to add more series to the same graph.

Once you click on the + button, you can add multiple series as follows.
Let's say I want to show both Maths marks and Science marks of the students in the same graph. For the first series I have selected Maths marks. Now let's add Science marks for the second series.

Like wise you can add any number of series in the same graph.

Once you have finish adding the required series click on Next button and you will see your data been visualized into a bar chart as follows.

Note that you can customize your chart (change colors, bar width etc) using a simple configuration file to meet your desire.

Click on the title "Bar Chart" on the top of the added gadget and change the title of your chart.

Enjoy adding as many as gadgets you need in to your dashboard.

Once you have completed adding gadgets click on Save on top right corner to save  your dashboard.

Gadget gen tool feature is in its very early stages, you will encounter number of bugs and feature limitations, we are working on improving these in future milestones.

  • Only supports connecting to RDBMS data source at the moment.
  • Line chart and Bar chart templates have been re-factored and rest of the charts will be done in future milestones.
  • Wizard UI will to be styled and polished in coming releases.
  • Error handling and validations are yet to be done.
  • Supporting text based x-axis ticks has been done and other special data types (ex : date/time) has not been done yet. Because by default flot (graphing library used for gadgets) supports only integer based x-axis ticks)

More good things are yet to come !!!
Be with us, tool will be improved more and more in future releases to cater all your data visualization needs.

Tuesday, January 7, 2014

Struggling with NoSuchProviderException: smtp or UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed ?

If you have faced with any of the above exceptions this blog post may be a little help for you to clear out some dark areas as I will list out some helpful posts/forums/articles that I ran through to chase away above two nasty fellows. But the main reason for writing this blog post is as a note to myself how I got through the above scary exceptions. 

I came across the above two exceptions when I was asked to go through this issue. The jaggery pom file that this jira is referring was this. As said in the jira when the jaggery level email feature is enabled, Axis2 email transport is not working and throwing the above said exception of 

ERROR {org.apache.axis2.transport.mail.MailTransportSender} - Error creating mail message or sending it to the configured serverjavax.mail.NoSuchProviderException: smtpat javax.mail.Session.getService( javax.mail.Session.getTransport( javax.mail.Session.getTransport(

As commented in the jira when I remove the below three lines in jaggery pom file under the export package header, Axis2 email transport started to function without any issue.


But after removing those lines I tested the jaggery level email feature with a sample application and it had break the jaggery email feature with the same exception as NoSuchProvider found in jaggery Rhino Engine as follows.

ERROR {org.jaggeryjs.scriptengine.engine.RhinoEngine} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.NoSuchProviderException: smtp (/Test//Test.jag#13)
[2013-12-23 10:09:21,600] ERROR {org.jaggeryjs.jaggery.core.manager.WebAppManager} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.NoSuchProviderException: smtp (/Test//Test.jag#13)

So that made sense when these exports are there, Axis2 email transport picks the wrong bundle at the class loading which is exported from jaggery. ( Where axis2 needs those bundles of version 1.4.0 it picks that of version 2.0.0.wso2v1) 

Then I thought of going ahead with fixing jaggery level email feature to work without the above three exports. Because even though those bundles were exported, they were used no where in jaggery. So there must be some misery why we have exported these in jaggery pom file.

To explore what is going behind the scene I played a bit with the pom file by removing all mail related imports/exports and etc as all those are coming from axis2 email transport. And ended up with the following exception.

ERROR {org.jaggeryjs.scriptengine.engine.RhinoEngine} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.MessagingException: IOException while sending message;
  nested exception is:
javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed
boundary="----=_Part_0_1755306561.1387772655287" (/Test//Test.jag#13)

So then my journey started to chase after this exception and came across this forum post. Go through the below comment by Oleg which shed some light in to the problem.
(I will add a screen shot of part of the reply for you to identify which one I'm talking about as there are many replies by Oleg )  

Go ahead reading the complete answer by Oleg. Although that answer ensured that this is some problem with class loading in OSGI level and also why we have exported those in jaggery level, still it didn't provide a solution to the problem.

Then this post came in to play.

Now the problem is more clear that this is due to buggy behavior of javax.activation and javax.mail in osgi level, due to issues in dynamic class loading. I could not go with the suggested approach due to some platform specific reasons of our code.

Finally the comments in this post became savior of mine.

So the send code is changed as given below.

         public void jsFunction_send() throws ScriptException { 
             ClassLoader classLoader = Thread.currentThread().getContextClassLoader();                                              Thread.currentThread().setContextClassLoader( javax.mail.Session.class.getClassLoader() );
             try {
             } catch (MessagingException e) {
                  throw new ScriptException(e); 
             } finally

Done with the self note :)  In-case you need to go through the current code, my pom file is here and java class is here.

Many thanks to Pradeep, Kishanthan and Rajith from WSO2 for their help.

Run Your Tests With GreenMail - A simple Mail Server

Let me start this post with some background information why I had to use a simple mail server like GreenMail.

Background story

I came across the need of an email or some mechanism that can be used in email related testcases. For an example if I need to send an email within my test case what is the mail account that I can use .

Simplest way is create some mail account with G-mail, Yahoo, Gmx or what ever your preferred free email provider and use credentials of that account and provider's smtp severs.

See the example below which is written in Jaggery. You can find a detailed sample code here.

var sender = new email.Sender("", "25", "", "pw", "tls");

In the above example we have used the smtp mail server of google ( and a some dummy account that we have created with G-mail. Similarly you can do the same with your preferred free email provider.

But the problem with this approach is when you run your test repeatedly for sometime (may be a year or so) test case starts failing. 

Recently I had to go through the same issue and I was amazed when I found out that it is a problem with the credentials that I used in the code which used to be pass the test without any issue earlier. (In the above example "" and password "pw") So I tried login to that gmail account and Google complained with the following message.

"Please change your password
We've detected suspicious activity on your Google Account. Please choose a new password to finish signing in. "

Google is suspicious with the account due to reasons listed here and the reason for this particular account may be "Messages that you didn't send appear in Sent Mail ".I could have got this issue fixed by resetting the password of this account as instructed in the above message. But someday I might face the same issue as this is a temporary solution.

So this solution of simple mail servers popped up with this background. Apart from GreenMail, there are many other simple mail servers available such as Apache James, Dumbster and so on. But GreenMail was the one that fitted most in to my requirement and I decided to move on with GreenMail.

How to get GreenMail mail server in your testcase

Add the relevant maven dependency in your pom file. You can find GreenMail dependencies for Maven projects here.

I added following snippet in my pom file.


And then added the following imports in my test java class file.

       import com.icegreen.greenmail.util.GreenMail;
       import com.icegreen.greenmail.util.ServerSetupTest;

Afterwards you have to initialize and start the GreenMail server as stated below in the same test class.

     GreenMail greenMail = new GreenMail ( ServerSetupTest.SMTP );
     greenMail.start ();

Then include what ever your sender code. In my case my sender code is written in Jaggery as below in a separate jaggery (.jag) file. (My test class is pointing to the jaggery file after the Green Mail initialization and start above. )

      var from = "";
      var to = "";
      var subject = "Test Subject";
      var content = "Test Content";

      var msg = require('email');
      var email = new msg.Sender("localhost", "3025", "", "frompw", "tls"); 

     email.from = from; = to;
     email.subject = subject;
     email.text = content;

     print("email successfully sent");

You may note in the above code that you don't need any free email provider credentials, but simply and some random password. So your test case will not fail due to credential issues.

Also note that without specifying the sender information as in the above code you can also embed that information in your configuration files as described in the answer here.

Your next step is upon message delivery and you need to  check whether the message received is as same as the message sent.

Given below is how you test whether the sent message subject is as same as the received message subject. You can do the same for mail content as well.

     String subject = greenMail.getReceivedMessages()[0].getSubject();
     assertEquals ( subject , "Test Subject");

Donot forget to stop the GreenMail server at the end.


We are done :) Run your test and see whether your test get passed. 

If you need any reference you can find my pom file here , test class here and my jaggery file here.

Try out more complex test cases with GreenMail !!!

Thursday, December 12, 2013

Flot - Get Your Ticks Rotated With Flot-Tickrotor

Once again this blog post is some simple but useful feature that I came across while playing out with Flot.

Have you ever got into trouble by your x-axis ticks overlapping on each other as you have many to be occupied within a small place-holder or because ticks are too long? It will make your chart look like a mess. 

As a workaround you can increase the width of your place-holder. But you will face into trouble if your place-holder has already occupied the full length of your page or your place-holder size is fixed and you don't have control over it.

Best possible solution
Probably you might have thought of rotating them in some angle to get this solved so that the visibility of the ticks get improved. Yes, that is the best possible solution that one can think of. Rotate the x-axis ticks !!!

So in this blog post I will explain how you can rotate your ticks or in other words display your ticks in some preferred angle.

It is just simple three step process.

1. Clone Flot-Tickrotor from here and place the jquery.flot.tickrotor.js along with other flot plugins.
2. Import the script file in your chart file as follows. 

     <script language="javascript" src="lib/jquery.flot.tickrotor.js" type="text/javascript"></script>

If you are using jquery.flot.axislabels.js then make sure that you place your jquery.flot.tickrotor.js before the jquery.flot.axislabels.js. Because jquery.flot.tickrotor.js completely redefines X-axis label height and then jquery.flot.axislabels.js modifies it.

3. In the options section of the xaxis specify the angle that you need to get your ticks rotated as rotateTicks.

                xaxis: {
                        axisLabel: 'Application',
                          ticks: ticks,
                        < your other x-axis options goes here >
                        rotateTicks: 135

We are ready to go !!! Now our x-axis ticks have rotated with an angle of 135 degrees.

You need to use the jquery.flot.js file from the flot-branch master as there is some bug fixed done in the master, related to ticks rotation.

If not, you will end up with your ticks getting repeated as shown in the below image.

You can find the full sample code of the above graph here .

- Thanks Mark Cote for the flot plugin to get the ticks rotated.

Inter Gadget Communication with WSO2 UES

The main objective of this blog post is to explain how to implement inter-gadget communication with WSO2 User Engagement Server (UES). 

Inter - Gadget Communication
Inter-gadget communication that we are talking here is the capability of one gadget to talk with some other gadget. In simple words "is Gadget A capable of sending some data to Gadget B" and "is Gadget B capable of updating itself with the received data". The message paradigm that is used underneath is the publisher - subscriber model.

Publisher - Subscriber Model
As shown in the image given below, publisher publishes the message to a channel and subscribers listen to that channel. Upon a new message in the channel by the publisher, subscribers fetch the message and update their selves accordingly.

In a complex scenario, there can be one publisher publishing to multiple channels and also number of subscribers listening to multiple channels. 

The role of WSO2 User Engagement Server is to act as the Message Broker in this paradigm.

I will explain the implementation of this publisher - subscriber (pub-sub) model with a simple example scenario.

Let's say publisher-gadget publishes a random number to a channel and that random number will be shown up in the subscriber-gadget each time that publisher-gadget will publish.

Gadgets usually communicate with the gadget-container via features and those features are declared in the the ModulPrefs section of gadget.xml file.

Gadgets convey their interest to either publish or subscribe to a channel via the gadgets.Hub.* API. Therefore the feature titled "pubsub-2" needs to be imported in-order to load the necessary API fragments . As I mentioned earlier this feature can be declared in the ModulePrefs section of our gadget.

OK. Now let's make our hands dirty with the sample that I described earlier.

Hint : Once we are done with the implementation we will have two gadgets with the following folder/file structure.

Publisher Gadget
Let's see how we can declare this in the publisher - gadget.xml file.

       <Require feature="pubsub-2">
                <Param name="topics">
                               <Topic title="randomNumber name="my-channel" publish="true"/>

Pay attention to the sections I have marked in red in the above code snippet.  Feature that we have to import is pubsub-2. And my-channel is the channel that the publisher wishes to publish into. To indicate that this is the publisher gadget we have set publish="true".

In the sample that I'm going to describe in this post, when somebody clicks the button named "Publish a random number" in the publisher gadget, a random number will be published to my-channel. So let's have a button declared in our gadget.xml file.

              <input type="button" value="Publish a random number" onclick="publish()"/>

The function call that will be triggered by this button click is publish() and we define it in the js file of our publisher - gadget. (in publisher-gadget.js)

          function publish() {
               var message = Math.random();
               gadgets.Hub.publish("my-channel", message);
               document.getElementById("output").innerHTML = "Number that is published to the channel : " + message;

Once again pay your attention to the red line which says publish the random number to my-channel which we declared in the ModulePrefs section.

Subscriber Gadget
As we did in the publisher gadget, we need to import the pubsub-2 feature and declare this gadget as a subscriber gadget in the subscriber-gadget.xml file.

          <Require feature="pubsub-2">
            <Param name="topics">
<Topic title="randomNumber" name="my-channel"
description="Subscribes to random number channel" type="object"

And then let's have a simple div in the subscriber-gadget.xml file which we can use to set the fetched value from the publisher's channel.

          <div id="output"> </div>

After that we need to add the following snippet in the subscriber-gadget.js to activate the subscription to "my-channel" channel.

         gadgets.HubSettings.onConnect = function () {
                     gadgets.Hub.subscribe("my-channel", callback);

The function that we want to invoke in the subscriber end, upon a message publishing by the publisher is written in the javascript callback function as follows which is also in the subscriber-gadget.js file.

         function callback(topic, obj, subscriberData) {
               document.getElementById("output").innerHTML = "Number that is fetched from the channel : " + obj;

OK. We are done with the pub-sub implementation.

Finally what we need to do is add these two gadgets to a WSO2 UES dashboard and check out the functionality.

For that you can follow WSO2 UES documentation on how to create a gadget and how to add gadgets to dashboards.

Or else there is a bit hacky way also ;) (Not recommended. But if you want to check whether your two gadgets working fine, you can try this out.)

1. Copy your two gadgets in UES_HOME/repository/deployment/server/jaggeryapps/portal/gadgets/
2. Then create a dashboard (let's say our dashboard is pub-sub) using any two of the existing gadgets (NOT our two gadgets) in UES dashboard.
3. Open up UES_HOME/repository/deployment/server/jaggeryapps/pub-sub and open up the index.jag
4. Search for word .xml and you will find two of them.
5. Replace those two urls with the following and save the index.jag file.

So your index.jag will have two snippets as follows.

     <li class="layout_block " data-sizey="2" data-sizex="2" data-col="1" data-row="2" 
                data-wid="5" data-prefs="{}" data-title="">

     <li class="layout_block " data-sizey="2" data-sizex="2" data-col="3" data-row="2"
                data-wid="6" data-prefs="{}" data-title="">

6. And then access the dashboard as https://localhost:9443/pub-sub/
7. You'll see a dashboard as given in the below image.

8. Click on "Publish a random number" button in the publisher gadget and observe that the same number getting printed in the subscriber gadget as well.

9. Try clicking on the same button several times and observe the subscriber gadget updating its content accordingly.

You can download the complete code of the above two gadget from here.

Real World Scenario
Let's say we have a time slider as the publsiher gadget. Upon a time period selection on the slider, we need to display the number of builds per apps within that period in a seperate gadget. So the later gadget acts as the subscriber by listening to the time slider gadget's channel.

You can do whole lot of things with WSO2 UES pub-sub model :)
Enjoy !!!

- Thanks to the WSO2 Gadget Server ( currently deprecated )  documentation on pub-sub model.