Pre-requisite

Download Apache Vysper from http://mina.apache.org/vysper/downloads.html

Vysper can be run in standalone mode as well as in an embedded mode.

Running as Standalone Server

Goto the bin directory and execute

run.sh

The Server shall start

Running in an Embedded Mode

Embedding Vysper into your own App is easy. Here is a glimpse

XMPPServer server = new XMPPServer("myembeddedjabber.com");
server.start();

There is still some glue code needed to get the Server function completely. Rather than me duplicating the stuff, please refer http://mina.apache.org/vysper/embed-into-another-application.html

Looks Interesting ! Give Vysper a try and let us know about your experience.

• Both are NIO frameworks
• Trustin has been part of both Frameworks, lately contributing to Netty (Thanks to Emmanuel for pointing this out)
• Both uses Annotation, though in MINA this is more in area of MINA State Machine, whereas Netty, has used annotations for configuring the IO Chains as well.
• Both Frameworks are working aggressively towards improving their Documentation

Hmm.. that's it for this post. Will be back soon with more


Before I continue, the opinion expressed here are purely mine, and other can tend to disagree. I am a MINA committer myself, so may have a little bias towards MINA. Shall try to keep my discussion unbiased, as much as possible. Having used MINA for a while, have a natural comfort with the framework.

What to Expect

Shall try to evaluate both the frameworks, as if I need to use them in one of my own projects and try to provide rational for my thoughts. User comments shall provide more indepth to the posts. The details shall be covered as I move along on my journey. This post is just the beginning.

Latest Releases

MINA - MINA 2.0 M4
Netty - 3.0.2 GA

Community Activity

Having an active community in an Open Source project is very important. It assures that the project is actively being looked after.

Source of Data

MINA - http://mina.markmail.org/
Netty - http://netty.markmail.org/

MINA community is very active, and scores well above Netty. Though Trustin is very active in Netty, and I hope Netty community shall have active support from more users/developers.

Documentation

MINA and Netty both have sufficient documentation to start with, but there is need for improvement. Netty Documentation is better organized. MINA Documentation needs a better organization. (Blame me for this, I am supposed to take care of this).

Coming up Next

Working of examples on both the frameworks and try to come up with Application Architectures soon.

Creating Auto Expanding Buffer

IoBuffer class in MINA has a property autoExpand, which when set makes the IoBuffer expand on the need basis, very much similar to StringBuffer class.
Let’s see, how we can achieve this

 
IoBuffer buffer = IoBuffer.allocate(8);
buffer.setAutoExpand(true);
 
buffer.putString("12345678", encoder);
// Add more to this buffer
buffer.put((byte)10);
 

When we set this property, the buffer will expand as and when needed. Under the hood, it essentially allocated a new IoBuffer and returns the new instance with the copied data.

So, can we shrink the buffer as well. Yes, if we set the autoshrink property to true.

 
IoBuffer buffer = IoBuffer.allocate(8);
buffer.setAutoShrink(true);
buffer.put((byte)7);
System.out.println("Buffer size = "+buffer.capacity());
buffer.shrink();
buffer.capacity(16);
System.out.println("Buffer size = "+buffer.capacity());
buffer.shrink();
System.out.println("Buffer size = "+buffer.capacity());
 

Here we set autoShrink property to true. To manually shrink the capacity, we use shrink() method. Caution, shrink(), won’t reduce capacity below the minimum capacity (the capacity with which the buffer was created). The second call to shrink(), will reduce the capacity to 8.

Has updated the MINA’s documentation page on IoBuffer today. So please feel free to find more details here

Useful API’s

setBlacklist(InetAddress[] addresses) – Sets a list of IP Addresses that needs to be blocked. This API call clears all the previously added Addresses

setSubnetBlacklist(Subnet[] subnets) – Sets a List of Subnets to be blocked. This API call clears all the previously added Addresses

setBlacklist(Iterable<InetAddress> addresses) – Sets a list of IP Addresses that needs to be blocked. This API call clears all the previously added Addresses

setSubnetBlacklist(Iterable<Subnet> subnets) - Sets a List of Subnets to be blocked. This API call clears all the previously added Addresses

block(InetAddress address) - Adds a remote address to the blacklist

block(Subnet subnet) – Adds a Subnet to be blocked

unblock(InetAddress address) – Removes remote address from the blacklist

unblock(Subnet subnet) – Removes a Subnet from the blacklist

Under the Hood

For each of the events, the Filter checks for the remote address, in the blacklisted Address. If found, the session is closed. The isBlocked() API, checks for the remote address in the configured list and returns true if found.

Blacklist Filter

How to Use Blacklist Filter

To use a blacklist filter, we need to follow some simple steps listed below

• Create an instance of Blacklist filter
• Add IP Addresses/Subnets to be blacklisted
• Add the Backlist filter to the Filter chain

Code Snippet

public BlacklistFilter createBlacklistFilter() throws UnknownHostException {
        BlacklistFilter filter = new BlacklistFilter();
 
        // Get the IP Addresses to be blocked
        // from a config file or from DB, whatever
        InetAddress[] blockedAddresses = null;
 
        // populate the list
        // Ensure that the list if populated correctly as per requirement
        blockedAddresses = new InetAddress[1];
 
        // block all connection from Localhost
        blockedAddresses[0] = InetAddress.getByName("127.0.0.1");
        filter.setBlacklist(blockedAddresses);
 
        return filter;
    }

The code is fairly straight forward. We create an instance of Blacklist Filter and add the list of Addresses to be blocked. Then we need to add the Filter instance in the Filter chain of our implementation.

Usage Scenario’s

• Conditionally allow traffic from only selected sources

Think Differently

• The default MINA implementation stores the blacklisted Addresses in a List, Custom data structure can be used to speed up search
• Can extend the Filter’s functionality to allow traffic only from selected sources.

So, how do we use this Decoder

Extend from the class CumulativeProtocolDecoder and implement the doDecode().

Conditions to take care of

1. read the IoBuffer and decode as per yur logic
2. If the data is sufficient to create your message, return true, else return false

This is how our application is structure. To see details of this application, please refer to the post Implementing Trap Receiver in 30 minutes using Apache MINA

To integrate with Spring, we need to do following

• One Handler
• Two Filter – Logging Filter and a ProtocolCodec Filter
• NioDatagram Socket

This is how our code looks like for the standalone application

public void initialize() throws IOException {
 
	// Create an Acceptor
	NioDatagramAcceptor acceptor = new NioDatagramAcceptor();
 
	// Add Handler
	acceptor.setHandler(new ServerHandler());
 
	acceptor.getFilterChain().addLast("logging",
				new LoggingFilter());
	acceptor.getFilterChain().addLast("codec",
				new ProtocolCodecFilter(new SNMPCodecFactory()));
 
	// Create Session Configuration
	DatagramSessionConfig dcfg = acceptor.getSessionConfig();
        dcfg.setReuseAddress(true);
        logger.debug("Starting Server......");
        // Bind and be ready to listen
        acceptor.bind(new InetSocketAddress(DEFAULT_PORT));
        logger.debug("Server listening on "+DEFAULT_PORT);
}

To integrate with Spring, we need to do following

1. Set the IO handler
2. Create the Filters and add to the chain
3. Create the Socket and set Socket Parameters

public void initializeViaSpring() throws Exception {
	new ClassPathXmlApplicationContext("trapReceiverContext.xml");
}

Apache MINA has wonderful concept of ProtocolDecoder to process Decoding protocol specific messages. XML is one of the most widely used format for EDA. Lets see how can we implement a Protocol Decoder for Apache MINA.

Algorithm

The picture below describes the basic algorithm that we need to use to construct an XML message from bytes.

The logic is simple, keep reading the bytes till the XML message is balanced. Balanced here means, that end of the root element has been achieved. For eg. If the xml document has root element as , we have to read the bytes till we received .

Its very particular to note that large XML packets when sent over TCP, may get fragmented and we shall received the same amount of read events while using Apache MINA low level API’s.

This type of situations where, we need to wait to data to completely arrive, calls for the use of CumulativeProtocolDecoder. As the name signifies, the decoder waits till, we get the balanced xml. Once the balanced XML is found, we write the parsed object to the output, to be processed further.

Lets see the code. My apologies for the unformatted code

public abstract class XMLDecoder extends CumulativeProtocolDecoder  {
 
/*
* As per XML specification 1.0, http://www.w3.org/TR/REC-xml
*/
private static final char XML_START_TAG = '<';
private static final char XML_END_TAG = '>';
private static final char XML_PI_TAG = '?';
private static final char XML_COMMENT_TAG = '!';
 
protected static enum ParseState {ELEMENT_START, ELEMENT_END, COMMENTS, ENDELEMENT, PI, UNDEFINED};
 
protected static final int ELEMENT_START = 1;
protected static final int ELEMENT_END = 2;
 
private static Logger logger = LoggerFactory.getLogger(XMLDecoder.class);
 
@Override
protected boolean doDecode(IoSession session, IoBuffer ioBuffer,
ProtocolDecoderOutput decoderOutput) throws Exception {
 
int startPosition = ioBuffer.position();
 
if(!ioBuffer.hasRemaining()) {
logger.debug("NO bytes to read keep waiting...");
return false;
}
 
// Continue to read the bytes and keep parsing
char currentChar = '0', previousChar = '0';
 
boolean rootElementStarted = false;
boolean rootElementPresent = false;
boolean isBalanced = false;
 
int rootStartPosition, rootEndPosition;
 
ParseState parsingState = ParseState.UNDEFINED;
logger.debug("Lets start decoding the XML");
 
String root = null;
 
boolean markedForEndElement = false;
 
while(ioBuffer.hasRemaining()) {
previousChar = currentChar;
currentChar = (char)ioBuffer.get();
 
switch (parsingState) {
case ELEMENT_START:
if(currentChar == XML_PI_TAG){
logger.debug("Got PI Element");
parsingState = ParseState.PI;
} else if(currentChar == XML_COMMENT_TAG) {
logger.debug("Got Comment Element");
parsingState = ParseState.COMMENTS;
} else if((currentChar == ' ' || currentChar == XML_END_TAG)
&& rootElementStarted && !rootElementPresent) {
rootEndPosition = ioBuffer.position();
rootElementPresent = true;
 
// Copy the Root Element
int cPos = ioBuffer.position();
int mPos = ioBuffer.markValue();
 
char[] rootChar = new char[cPos - mPos];
for(int i = mPos - 1, j =0; i < cPos - 1; i++) {
rootChar[j++] = (char)ioBuffer.get(i);
}
 
root = new String(rootChar);
logger.debug("Root Element = "+ root);
parsingState = ParseState.ELEMENT_END;
logger.debug("Root Element detection completed "+rootEndPosition);
} else if(currentChar == XML_END_TAG) {
parsingState = ParseState.ELEMENT_END;
} else if(!rootElementStarted && !rootElementPresent) {
rootStartPosition = ioBuffer.position();
ioBuffer.mark();
rootElementStarted = true;
logger.debug("Got the root element at "+rootStartPosition);
} else if (currentChar == '/') {
// Change state
if(previousChar == XML_START_TAG) {
parsingState = ParseState.ENDELEMENT;
}
}
break;
 
case ENDELEMENT:
if(currentChar == XML_END_TAG) {
parsingState = ParseState.ELEMENT_END;
 
int cPos = ioBuffer.position();
int mPos = ioBuffer.markValue();
 
char[] el = new char[cPos - mPos];
for(int i = mPos - 1, j =0; i < cPos - 1; i++) {
el[j++] = (char)ioBuffer.get(i);
}
markedForEndElement = false;
if(root.equalsIgnoreCase(new String(el))) {
logger.debug("XML is balanced."+root);
isBalanced = true;
}
 
break;
} else if (currentChar == ' ') {
continue;
} else {
 
// mark the position, we need to compare the it to see that if its the end element
if(!markedForEndElement) {
ioBuffer.mark();
markedForEndElement = true;
}
}
break;
 
case ELEMENT_END:
if(currentChar == XML_START_TAG) {
parsingState = ParseState.ELEMENT_START;
}
break;
 
case UNDEFINED:
if(currentChar == XML_START_TAG) {
parsingState = ParseState.ELEMENT_START;
}
break;
 
case COMMENTS:
if (currentChar == '-') {
previousChar = currentChar;
} else if (previousChar == '-' && currentChar == '>') {
parsingState = ParseState.ELEMENT_END;
}
break;
 
case PI:
if (currentChar == '?') {
previousChar = currentChar;
} else if (previousChar == '?' && currentChar == XML_END_TAG) {
parsingState = ParseState.ELEMENT_END;
}
break;
 
default:
break;
}
}
 
if(isBalanced) {
decoderOutput.write(parserXML(ioBuffer));
}
 
if(isBalanced && !ioBuffer.hasRemaining()) {
logger.debug("No more bytes to process");
return true;
}
 
ioBuffer.position(startPosition);
return false;
}
 
/**
* Extending classes can implement their custom XML parsing to create Objects
* from XML and use them appropriately in Handler
*
* @param xmlBuffer
* @return
*/
public abstract Object parserXML(IoBuffer xmlBuffer);
}

The implementation is pretty straight forward. We take each character and try to match the characters as specified in XML specification.

Some keys things in the implementation:
1. The Decode function just collects the bytes till we get the balanced XML document
2. Once we get the balanced XML document, we shall call the abstract function parseXML(). The function has been kept abstract, so that its easy to implement custom parsing using desired XML library like JAXB, JIBX etc
3. We have to return true from doDecode(), the moment we have balanced XML. Return type true indicates to the framework that we are not waiting for any more data. A false, forces the framework to keep accumulating the data, till we write it to the output. Now it must be clear why, its called Cumulative decoder.

Still have Queries, please leave a comment and I shall revert back to you.

Will continues to post articles about MINA, to be updated,  Subscribe in a reader

Yes, we are going to implement a SNMP trap receiver in less than 30 minutes. If you have been following my post on Apache MINA, this would be a natural extension to it. In this article, we shall bring together all our components to build the system.

 Pre-requisites:

 Please read through these articles. You can find all these posts on Apache MINA

• What is Apache MINA
• Apache MINA based Server Application Architecture
• Implementing UDP Server using Apache MINA
• Implementing SNMP4J Decoder for Apache MINA

 To execute the code, you need following jars

• mina-core-2.0.0-M1.jar
• slf4j-api-1.5.0.jar
• slf4j-log4j12-1.5.0.jar
• log4j-1.2.15.jar
• SNMP4J.jar

 I am assuming that you have read my previous posts and jumping straight to the implementations. The high level architecture is explained in the post Apache MINA based Server Application Architecture

Architecture 

Let’s understand the flow here and see what all we need to do to process a trap

• Receive the trap over UDP
• Decode the Trap
• Dump the Trap

Lets see how each of the these maps to the code we have already written

It's a now simple to understand that we have reused all the components to create the Server. Let me not write too much, and jump straight to the code J

 The Server Code

The only change that we did to the UDP Server code was to add a Protocol Codec. At line 36, we added our custom ProtocolCodecFiler. The SNMPCodecFactory return the instance of our SNMP4J codec. There is not much code and this construct is fairly simple, and explained very well in MINA’s documentation. From the main method, we just need to call the method initialize() and our trap receiver starts.

References

• http://mina.apache.org/
• http://www.snmp4j.org/