Saturday, October 29, 2016

Aggregate multiple responses using WSO2 ESB

This blog post explains how we can aggregate multiple responses from different endpoints.
In my previous post, I explained how we can call multiple endpoints in parallel using WSO2 ESB. So in this post we are aggregating the responses we got from those service calls.

Synapse Configuration


Explanation

In this example, we have aggregated the two responses coming from our two service calls and have wrap them using My_Response tag. So your final message after the aggregation will look like below.

Note the following.
  • When aggregating we have used the same id (which is "BOOKING_RESPONSE") that we set when we are calling the two services
  • Inside "onComplete" element you can write the logic you want to perform after the aggregation. It can contain sequences or another call to an endpoint.
  • In this example we have aggregated the bodies of the two responses without any condition
  • "Aggregated_Responses" property defined before the aggregate mediator is used to accumulate the aggregated messages inside a single property. Note we have given the name of the property in the "enclosingElementProperty" tag

Apart from aggregating the responses of the service calls sent from clone mediator, we can use the aggregate mediator to aggregate responses of the service calls from iterate mediator as well.

Call multiple endpoints in parallel using WSO2 ESB

In this blog post I'm going to explain how to call multiple endpoints in parallel using the Clone Mediator.

Synapse Configuration


Explanation

In the above example configuration, we are calling the two endpoints defined in the two call templates in parallel. Similarly by adding multiple "target" elements to the same configuration you can call multiple endpoints in parallel. When using the above configuration same message is cloned and two identical copies of the message is send to the two endpoints.

Note the following.
  • "sequential" attribute is set to false. Hence the two calls are happening in parallel
  • A value is set to "id" so that later we can use this id to aggregate the responses from two services
  • By default "continueParent" is set to false
  • Instead of the call template you can also use a sequence inside the "target"

In a future post I will explain how we can aggregate the response from the two services when the clone mediator is used to call endpoints.

Sunday, October 23, 2016

Adding a new xml element to the payload - WSO2 ESB

In this blog post I'm going to explain how we can insert a single new xml element in to the payload in WSO2 ESB.

Why not payloadFactory Mediator ?
If you are familiar with WSO2 ESB you may know that if we want to change the entire payload into a different payload or build a new payload by extracting properties using the existing payload, we can use payloadFactory mediator. But for this requirement that I'm going to describe, payload factory mediator will not be the ideal mediator due to different reasons. One major reason is, if the current payload is bit lengthy one, you will have to build the other parts of the payload even though you don't need to change them at all.

If not payloadFactory then what?
It will be very convenient to get this requirement done using Enrich Mediator. Enrich Mediator will take your desired OMElement (xml element) using the configuration that you state in the source section and then will insert it to the target location you state.

Usecase
I will explain the configuration using a sample usecase. Say I receive some request into ESB and depending on some properties in the payload I want to set some element in the payload. For an example assume I need to insert a new element into the payload if the received payload is not a valid payload.

Configuration
In the above example I have set the incoming request to ESB in to a property (INCOMING_REQUEST) during a previous mediation. And hence using the first enrich mediator I am replacing the body of the current payload using that property.
So the second mediator is the one actually does the job.
It will take the OMElement given under source which is
and will insert as a child element of /BookingRequest/Booking xpath location in the INCOMING_REQUEST xml.

Incoming Request


After enrich mediator
In this example I have modified the payload by adding only one xml element. Even if you need to add more elements also you can follow the same approach.

Acknowledgement
Many thanks to Rajith from WSO2 for the tip.


Thursday, February 20, 2014

Crowning of WSO2 ESB as "Usain Bolt" in the ESB space !!!

With the latest round of performance testing results published for WSO2 ESB 4.8.1, it is unveiled that the fastest ESB in the ESB space is WSO2 ESB. You can find the complete article here.

The tests has been conducted against number of leading open source ESB's and it is proven that WSO2 ESB surpasses rest in almost all the scenarios and crowned as the No 1.

Observations

The table and the graph below shows the summarized results.
Conclusion

WSO2 ESB 4.8.1 has continued to outperform all other compared ESBs in almost all scenarios !!!


References

Tuesday, February 11, 2014

WSO2 User Engagement Server 1.5.0 -Milestone-1 Released !

Let me start this post with a saying.
                 "Data is worthless if you don't communicate It correctly"

I'm sure everybody reading this post will agree on the above saying. WSO2 User Engagement Server or simply known as WSO2 UES was released with the aim of providing a set of tools that will ease your data visualization.

If it is the first time that you are hearing about WSO2 UES, I would suggest you to peep a bit into WSO2 documentation about UES 1.0.0 here. Also if you are a developer who is interested in how the things have been done, UES git repo is the place for you.

UES 1.0.0 release was done sometime back and recently we released the milestone one of version 1.5.0 of the product. Milestone 1 mainly contains the initial implementation of Gadget generation tool which will simply take user through a wizard and will draw a graphing gadget visualizing his data.

Are you interested ? Let me walk you through the lanes of UES. First of all download the WSO2 UES 1.5.0 Milestone 1 pack from here.

NOTE:
   PRODUCT_HOME that I will be referring from here on-wards is the directory where you unzipped the downloaded pack. 

OK. Now we are ready to make our hands dirty with our data :)

Place your RDBMS driver at <PRODUCT_HOME>/repository/components/lib/
For an example if your data resides in a mysql database, place the mysql driver at the above location.
(IMPORTANT : If you have started the server already, you have to restart after placing the driver.)

Start the downloaded pack as you start any of the WSO2 products.
  • On Windows: <PRODUCT_HOME>\bin\wso2server.bat
  • On Linux/Solaris: sh <PRODUCT_HOME>/bin/wso2server.sh

You can find more information on how to start the product here.

Go to the browser and open the link http://localhost:9763/portal and login.
You will see a page similar to following.



Find the sign in button on the top right corner of the page. For the moment let's login with following credentials.
                 Username : admin
                 Password: admin
Please note that if you need to create a new user and login with that user, you can do so.

Once you logged in, you will be redirected to the dashboard creation page.

Click Create on top left corner and select Dashboard.





Give a preferred name to the dashboard and select a layout. Then click on Create new app.
Layout defines the arrangement of gadgets in your dashboard.


Then you will be redirected to the dashboard designer window where you can add gadgets to the created dashboard.

Click on "Add Gadget" button on any of the blocks visible and follow the wizard.



Select your preferred data-source and click Next.
NOTE :
    Since this is milestone 1 of the tool, we have completed the implementation only for RDBMS. So please bear with us, we will support rest of the data-source types in coming releases.




Enter the connection details of your data-source and click Next. 
See the image below where I have entered mysql connection details of the database where I want to visualize my data. My database is "std" and both the username and password for the database is "test". 

Also note that the information of the data-source needs to be of the driver that you placed in the lib folder as stated in first step.
Ex: If you place mysql driver in lib folder then connection URL, username and password fields should be of a mysql database.




Enter your SQL query to get the data from the respective table of the database and Click Next.
This SQL query can be any complex query which return the columns that you want to visualize in your gadget. 
NOTE : Do not add the semi-colon at the end of the query.




Go to "Store Items" and click on your preferred graph type.
NOTE:
   As I mentioned above, at the moment we only support the chart types that I have caged with a rectangle  (Line chart and Bar chart). Await for more chart types in future.



Let's say I select the chart type as "Bar Chart".

Do the data mapping of the columns as needed and click Next.


  • Depending on the SQL query that you submitted in the previous step, the columns retrieved from that particular query will be visible as drop down for each axis. Select the column names from them.
  • Give the axis label names that you want in your chart.
  • You can set your chart to be updated at a given interval by setting the Update interval value. Note that update interval is in seconds.
  • Click on the + mark (shown in the cage), if you want to add more series to the same graph.


Once you click on the + button, you can add multiple series as follows.
Let's say I want to show both Maths marks and Science marks of the students in the same graph. For the first series I have selected Maths marks. Now let's add Science marks for the second series.



Like wise you can add any number of series in the same graph.

Once you have finish adding the required series click on Next button and you will see your data been visualized into a bar chart as follows.




Note that you can customize your chart (change colors, bar width etc) using a simple configuration file to meet your desire.

Click on the title "Bar Chart" on the top of the added gadget and change the title of your chart.

Enjoy adding as many as gadgets you need in to your dashboard.


Once you have completed adding gadgets click on Save on top right corner to save  your dashboard.

IMPORTANT
Gadget gen tool feature is in its very early stages, you will encounter number of bugs and feature limitations, we are working on improving these in future milestones.

      Limitations
  • Only supports connecting to RDBMS data source at the moment.
  • Line chart and Bar chart templates have been re-factored and rest of the charts will be done in future milestones.
  • Wizard UI will to be styled and polished in coming releases.
  • Error handling and validations are yet to be done.
  • Supporting text based x-axis ticks has been done and other special data types (ex : date/time) has not been done yet. Because by default flot (graphing library used for gadgets) supports only integer based x-axis ticks)

More good things are yet to come !!!
Be with us, tool will be improved more and more in future releases to cater all your data visualization needs.

Tuesday, January 7, 2014

Struggling with NoSuchProviderException: smtp or UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed ?

If you have faced with any of the above exceptions this blog post may be a little help for you to clear out some dark areas as I will list out some helpful posts/forums/articles that I ran through to chase away above two nasty fellows. But the main reason for writing this blog post is as a note to myself how I got through the above scary exceptions. 

I came across the above two exceptions when I was asked to go through this issue. The jaggery pom file that this jira is referring was this. As said in the jira when the jaggery level email feature is enabled, Axis2 email transport is not working and throwing the above said exception of 

ERROR {org.apache.axis2.transport.mail.MailTransportSender} - Error creating mail message or sending it to the configured serverjavax.mail.NoSuchProviderException: smtpat javax.mail.Session.getService(Session.java:782)at javax.mail.Session.getTransport(Session.java:708)at javax.mail.Session.getTransport(Session.java:651)
..............................

As commented in the jira when I remove the below three lines in jaggery pom file under the export package header, Axis2 email transport started to function without any issue.

com.sun.mail.imap;version="2.0.0.wso2v1", 
com.sun.mail.pop3;version="2.0.0.wso2v1", 
com.sun.mail.smtp;version="2.0.0.wso2v1", 

But after removing those lines I tested the jaggery level email feature with a sample application and it had break the jaggery email feature with the same exception as NoSuchProvider found in jaggery Rhino Engine as follows.

ERROR {org.jaggeryjs.scriptengine.engine.RhinoEngine} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.NoSuchProviderException: smtp (/Test//Test.jag#13)
[2013-12-23 10:09:21,600] ERROR {org.jaggeryjs.jaggery.core.manager.WebAppManager} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.NoSuchProviderException: smtp (/Test//Test.jag#13)
..........................................

So that made sense when these exports are there, Axis2 email transport picks the wrong bundle at the class loading which is exported from jaggery. ( Where axis2 needs those bundles of version 1.4.0 it picks that of version 2.0.0.wso2v1) 

Then I thought of going ahead with fixing jaggery level email feature to work without the above three exports. Because even though those bundles were exported, they were used no where in jaggery. So there must be some misery why we have exported these in jaggery pom file.

To explore what is going behind the scene I played a bit with the pom file by removing all mail related imports/exports and etc as all those are coming from axis2 email transport. And ended up with the following exception.

ERROR {org.jaggeryjs.scriptengine.engine.RhinoEngine} -  org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: javax.mail.MessagingException: IOException while sending message;
  nested exception is:
javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed
boundary="----=_Part_0_1755306561.1387772655287" (/Test//Test.jag#13)


So then my journey started to chase after this exception and came across this forum post. Go through the below comment by Oleg which shed some light in to the problem.
(I will add a screen shot of part of the reply for you to identify which one I'm talking about as there are many replies by Oleg )  





Go ahead reading the complete answer by Oleg. Although that answer ensured that this is some problem with class loading in OSGI level and also why we have exported those in jaggery level, still it didn't provide a solution to the problem.

Then this post came in to play.




Now the problem is more clear that this is due to buggy behavior of javax.activation and javax.mail in osgi level, due to issues in dynamic class loading. I could not go with the suggested approach due to some platform specific reasons of our code.

Finally the comments in this post became savior of mine.

So the send code is changed as given below.


         public void jsFunction_send() throws ScriptException { 
             ClassLoader classLoader = Thread.currentThread().getContextClassLoader();                                              Thread.currentThread().setContextClassLoader( javax.mail.Session.class.getClassLoader() );
             try {
                   message.setContent(multipart); 
                  Transport.send(message); 
             } catch (MessagingException e) {
                  throw new ScriptException(e); 
             } finally
                  Thread.currentThread().setContextClassLoader(classLoader);
             } 
      }



Done with the self note :)  In-case you need to go through the current code, my pom file is here and java class is here.

Acknowledgement
Many thanks to Pradeep, Kishanthan and Rajith from WSO2 for their help.


Run Your Tests With GreenMail - A simple Mail Server

Let me start this post with some background information why I had to use a simple mail server like GreenMail.

Background story

I came across the need of an email or some mechanism that can be used in email related testcases. For an example if I need to send an email within my test case what is the mail account that I can use .

Simplest way is create some mail account with G-mail, Yahoo, Gmx or what ever your preferred free email provider and use credentials of that account and provider's smtp severs.

See the example below which is written in Jaggery. You can find a detailed sample code here.

 
var sender = new email.Sender("smtp.gmail.com", "25", "mytestmail@gmail.com", "pw", "tls");


In the above example we have used the smtp mail server of google (smtp.gmail.com) and a some dummy account that we have created with G-mail. Similarly you can do the same with your preferred free email provider.

But the problem with this approach is when you run your test repeatedly for sometime (may be a year or so) test case starts failing. 

Recently I had to go through the same issue and I was amazed when I found out that it is a problem with the credentials that I used in the code which used to be pass the test without any issue earlier. (In the above example "mytestmail@gmail.com" and password "pw") So I tried login to that gmail account and Google complained with the following message.

"Please change your password
We've detected suspicious activity on your Google Account. Please choose a new password to finish signing in. "

Google is suspicious with the account due to reasons listed here and the reason for this particular account may be "Messages that you didn't send appear in Sent Mail ".I could have got this issue fixed by resetting the password of this account as instructed in the above message. But someday I might face the same issue as this is a temporary solution.

So this solution of simple mail servers popped up with this background. Apart from GreenMail, there are many other simple mail servers available such as Apache James, Dumbster and so on. But GreenMail was the one that fitted most in to my requirement and I decided to move on with GreenMail.

How to get GreenMail mail server in your testcase

Add the relevant maven dependency in your pom file. You can find GreenMail dependencies for Maven projects here.

I added following snippet in my pom file.


       <dependency>
            <groupId>com.icegreen</groupId>
            <artifactId>greenmail</artifactId>
            <version>1.3.1b</version>
        </dependency>


And then added the following imports in my test java class file.

   
       import com.icegreen.greenmail.util.GreenMail;
       import com.icegreen.greenmail.util.ServerSetupTest;


Afterwards you have to initialize and start the GreenMail server as stated below in the same test class.


     GreenMail greenMail = new GreenMail ( ServerSetupTest.SMTP );
     greenMail.start ();


Then include what ever your sender code. In my case my sender code is written in Jaggery as below in a separate jaggery (.jag) file. (My test class is pointing to the jaggery file after the Green Mail initialization and start above. )

 
      var from = "from@localhost.com";
      var to = "to@localhost.com";
      var subject = "Test Subject";
      var content = "Test Content";

      var msg = require('email');
      var email = new msg.Sender("localhost", "3025", "from@localhost.com", "frompw", "tls"); 

     email.from = from;
     email.to = to;
     email.subject = subject;
     email.text = content;

     email.send();
     print("email successfully sent");


You may note in the above code that you don't need any free email provider credentials, but simply xxx@localhost.com and some random password. So your test case will not fail due to credential issues.

Also note that without specifying the sender information as in the above code you can also embed that information in your configuration files as described in the answer here.

Your next step is upon message delivery and you need to  check whether the message received is as same as the message sent.

Given below is how you test whether the sent message subject is as same as the received message subject. You can do the same for mail content as well.


     String subject = greenMail.getReceivedMessages()[0].getSubject();
     assertEquals ( subject , "Test Subject");


Donot forget to stop the GreenMail server at the end.


    greenMail.stop();


We are done :) Run your test and see whether your test get passed. 

If you need any reference you can find my pom file here , test class here and my jaggery file here.

Try out more complex test cases with GreenMail !!!