Messaging Systems
MULE
MULE is a messaging framework that implements some sort of messaging protocol, and
facilitates the message passage in a scalable manner. Additionally, MULE is highly regarded in many
circles, as well as being fully open source, so no bothers with licensing fees. To the rest of the services, it
looks only like a black box, with the addition of message senders/receivers (which would be designed to
handle several types of messages, and/or be extensible in design).
MULE
Message Sender
Message Receiver
In actual practice, however, it would look something more like:
MULE INSTANCE
Message
Sender
UMO
MULE INSTANCE
…
UMO
Message
Receiver
UMO
Here, each UMO represents some sort of messaging protocol, or a stage of one. As can be seen in the
diagram, it is horizontally scalable, allows for disparate messaging systems, and thus grants us
extensibility. In particular, when the final product is handed off to JPMorgan, they will be able to adjust
to their own messaging systems, and allow for easy interoperability. Of mention as well is that it is
Maven-buildable. Most protocols are operable within MULE, including JMS and SOAP, and thus it is
highly recommended that we look into this as we are building the various architectures that will
(eventually) require communication with each other while on separate machines.
AMQP
AMQP defines a messaging protocol – something that can be implemented within the MULE
messaging framework. It basically helps give structure to the black box noted above. The key part of
AMQP is that it utilizes a messaging queue stored on a database for heavy loads, to ensure stability and
recoverability. An example diagram is as follows:
Incoming
Message
Message
Hub
Message
Queue
Message Processor
Message Sender
Outgoing
Message
In this diagram, the Message Hub acts as a receiver for all sent messages, which are placed into the
Message Queue (kind of like a mailbox). The Message Processor examines the message, determines
where it needs to go, and the Message Sender forwards the message to the requisite recipients (like the
driver of a UPS truck). Of course, this still is a very general notion of what is going on within the
messaging server, but the key things to note are that with this system it is possible to have multiple
messaging hubs, processors, and senders. In addition, by utilizing visitor patterns we would be able to
decouple the type of message from the actual message sending within the processor, allowing for full
extensibility as we would desire. Of benefit to note would be that since we are dealing with timesensitive data, it would be easy to have methods analyze the message queue, and disregard messages
that are beyond the scope of time-sensitivity.
There are, however, two major faults to AMQP that we cannot ignore. The first is that there is
no current existing plug-in or module (that isn’t thoroughly tested or out of beta) for AMQP in
conjunction with MULE. This is something that we would have to write ourselves, and would take a fair
amount of time – it may be worth asking if JPMorgan has already developed something along these
lines. Secondly, scalability exists, but we are looking at an estimated (according to the AMQP
specification) 16 servers to get a throughput of 1000 messages per second. It would thus be beneficial
for JPMorgan to use later on since they already have these resources dedicated to AMQP, but we should
seriously consider these drawbacks while planning our architecture.
QPID
The current implementation of QPID is created by Apache – and if we get Maven, would be very
easy to work. Otherwise, we would need to utilize Python (Maven uses the aptly named Jython). There
are no records of it hooking into MULE easily at all, thus I recommend against it as it does not suit our
requirement of being interchangeable. Again, the concerns against AMQP must be taken into
consideration with QPID relative to the work that we are doing.
Recommendation
My recommendation is as follows: for our purposes within the scope of this project, we only
need implement at most MULE as a framework for carrying messages between systems. In fact, we
should implement it, as it will provide two key components within the messaging system – extensibility
and scalability. The scalability is had through MULE’s inherent design, as shown above, whereas the
extensibility is had in that MULE provides a framework with which we can swap out various messaging
protocols (including AMQP).
Do we need, or stand to benefit from AMQP at this point? No – as discussed earlier, while
AMQP is scalable it is not valuable to us at the moment due to its relatively high entry fee. This does
not, however, exclude the possibility of utilizing AMQP in the near future, so long as the modules are
designed properly to be replaceable. For our purposes, and within the scope of the project, a more
naïve messaging protocol would be recommended – JMS could work well, even. What is more
important is that we provide a framework so that these additions can be implemented later on – this is
something MULE affords us.
References
Apache QPID - http://cwiki.apache.org/qpid/qpid-developer-documentation.html
AMQP Specification – http://www.amqp.org
MULE Documentation - http://mule.mulesource.org/display/REG/Home
Evaluation on MULE - http://www.theserverside.com/tt/articles/article.tss?l=CaseStudyMule
JMS - http://java.sun.com/products/jms/