In this small post we shall explore Terracotta, a leading pure Java Scalability platform. The discussion is based on the AtomicInteger example from Terracotta site, which shows how to implement a Cluster wide id generator (Actually it’s the Sequencer example, but to keep my steps simple had used AtomicInteger). The reason why I choose this example was coz of a very similar requirement that I had to implement in Clustered J2EE application.
Terracotta is well known and needs no introduction

About problem Statement

Well, I needed a simple solution to have
• Cluster wide unique id’s
• Less frequent access to these id’s
• Optional Persistence

NOTE: Please note that the current example is slightly modified version of example from terracotta.org.

Pre-requisite

To run this example, you need to have following installed
• Terracotta
• And JDK offcourse
Let’s take a look at the sample code first

import java.util.concurrent.atomic.*;

public class IdGenerator
{

private AtomicInteger masterCounter = new AtomicInteger(0);

private AtomicInteger slaveCounter = new AtomicInteger(1024);

public int getNewMasterId()
{
int newId = masterCounter.getAndIncrement();
System.out.println("Master Id is: " + newId);
return newId;
}

public int getNewSlaveId()
{
int slaveId = slaveCounter.getAndIncrement();
System.out.println("Slave Id is: " + slaveId);
return slaveId;
}

public static void main(String[] args)
{
new IdGenerator().getNewMasterId();
}
}

The code has two fields ids master and slave, of which master id need to be unique cluster wide. The code is pretty straight forward, and there is nothing to explain. The id’s are generated by simple getAndIncrement() API.

Lets see the tc-config.xml

<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">

<application>
<dso>
<roots>
<root>
<field-name>IdGenerator.masterCounter</field-name>
</root>
</roots>
</dso>
</application>
</tc:tc-config>

Since only master Id needed to be unique in this case, hence the configuration

Let’s look at the deployment architecture

Deployment Architecture

The application was to run on 3 nodes (3 JVM’s on same machine) and one Terracotta server running.
Before we get into running this application, lets take a look at some useful scripts provided by Terracotta
• dso-java (bat|sh) – startup script that bootstraps Terracotta libraries to your application
• start-tc-server (bat|sh) – Script to start Terracotta Server. It is mandatory to start Terracotta Server before running Clients

Let’s run the application
1. Compile the application. There is no dependency on TC libraries.
2. First Start the terracotta Server using script start-tc-server
3. Run 3 separate JVM with command “dso-java IdGenerator”
That’s it. Our first application is running. If we observe the console output, you can see that we have unique id’s for each invocation.

References:

Source code for Terracotta examples - http://svn.terracotta.org/svn/forge/cookbook

Pre-requisite

The article assumes that you have downloaded MPXJ release and related dependencies. If not here are the links for download

• MPXJ - http://sourceforge.net/project/showfiles.php?group_id=70649
• Dependencies - http://mpxj.sourceforge.net/dependencies.html

Now let’s get back to the original intent of reading MPP files

Lets see the simplest form of reading an MPP file

try {
    ProjectWriter writer = ProjectWriterUtility.getProjectWriter("test.mpx");
    ProjectFile project = new ProjectFile();
 
    // Operations on Project File
    writer.write(project, "test.mpx");
} catch (InstantiationException e) {
    e.printStackTrace();
} catch (IllegalAccessException e) {
    e.printStackTrace();
} catch (IOException ioEx) {
    ioEx.printStackTrace();
}

In the first part, let’s explore what are the different ways of reading a MPP file in MPXJ.

Pre-requisite

The article assumes that you have downloaded MPXJ release and related dependencies. If not here are the links for download

• MPXJ - http://sourceforge.net/project/showfiles.php?group_id=70649
• Dependencies - http://mpxj.sourceforge.net/dependencies.html

Now let’s get back to the original intent of reading MPP files

Lets see the simplest form of reading an MPP file

try {
     ProjectReader reader = ProjectReaderUtility.getProjectReader(fileName);
     ProjectFile projectFile = reader.read(fileName);

     // ... process Project file here
} catch (Exception e) {
     e.printStackTrace();
}

ProjectReaderUtility.getProjectReader(), provides an appropriate Reader class, based on the version of MS Project file. This is the simplest way to read an MPP file. ProjectFile is the class that contains details about the Schedule, we can iterate and extract information from it.

Alternative ways

• For reading MPX file, you can use net.sf.mpxj.mpx.MPXReader
• For reading MPP,MPT file, use net.sf.mpxj.mpp.MPPReader
• For reading MSPDI, use net.sf.mpxj.mspdi.MSPDIReader

This shall be the logical flow of the implementation

• Get the encoded Trap Data
• Send the Trap 

The code snippet above shows a simple way of creating and encoding a Trap PDU. Essentially, we create an instance of PDU class and sets the type as Trap. This is important, else SNMP4J shall throw an exception. Thereafter, we can set the trap parameters. Here, we have hardcoded the parameters, there can be custom implementations that can take these from config files or from UI. After setting the parameters, we just call the encode function passing the Output stream and collect the byte array to be sent.

Sending part is even simpler     

The send code is preety straight forward. Here we have used Datagram Socket, we can use Apache MINA UDP Client implementation to send the trap as well.

References

• http://www.snmp4j.org/
• Implementing UDP Client using Apache MINA 

Will continues to post articles about MINA, to be updated,  Subscribe in a reader

Yes, we are going to implement a SNMP trap receiver in less than 30 minutes. If you have been following my post on Apache MINA, this would be a natural extension to it. In this article, we shall bring together all our components to build the system.

 Pre-requisites:

 Please read through these articles. You can find all these posts on Apache MINA

• What is Apache MINA
• Apache MINA based Server Application Architecture
• Implementing UDP Server using Apache MINA
• Implementing SNMP4J Decoder for Apache MINA

 To execute the code, you need following jars

• mina-core-2.0.0-M1.jar
• slf4j-api-1.5.0.jar
• slf4j-log4j12-1.5.0.jar
• log4j-1.2.15.jar
• SNMP4J.jar

 I am assuming that you have read my previous posts and jumping straight to the implementations. The high level architecture is explained in the post Apache MINA based Server Application Architecture

Architecture 

Let’s understand the flow here and see what all we need to do to process a trap

• Receive the trap over UDP
• Decode the Trap
• Dump the Trap

Lets see how each of the these maps to the code we have already written

It's a now simple to understand that we have reused all the components to create the Server. Let me not write too much, and jump straight to the code J

 The Server Code

The only change that we did to the UDP Server code was to add a Protocol Codec. At line 36, we added our custom ProtocolCodecFiler. The SNMPCodecFactory return the instance of our SNMP4J codec. There is not much code and this construct is fairly simple, and explained very well in MINA’s documentation. From the main method, we just need to call the method initialize() and our trap receiver starts.

References

• http://mina.apache.org/
• http://www.snmp4j.org/

Steps to create a UDP Server using java.net API's

1. Create a Socket and listen for incoming connection
2. Process each packet in a separate thread (I hate this, unfortunately need this to have high processing rate)
3. Parse and process the request and optionally send response (Lets omit this to keep things simple)

That's it

• Our protocol is carrying Strings in UDP packet
• We shall not do any transformation on the packets received. We shall just dump the content

The Handler

 

The Handler is in its basic form. Our only method of interest is messageReceived. Since we know that we are getting "String" message without any transformation, we could easily create a new string from the bytes received. A wonderful thing to note about this API is the parameter "message". This makes the API generic enough to cater to any kind of objects. If we had used a ProtocolCodec in the chain and had transformed the byte[] into a custom Object, we would have type-casted message to that object.

That's all in the handler.

 

The Server

Lets see the main Server code

 

 

The Server is even simpler than client

We create an instance of NioDatagramAcceptor and add our custom handler to it. We then bind to the port desired. Here I have made it bind to the default port, code can be customized to bind to any port desired.

So what does 31 does. There is wonderful description of this option at http://www.unixguide.net/network/socketfaq/4.5.shtml
The main function is pretty simple. The Server is ready. You can run it, using the Client from the post Implementing UDP Client using Apache MINA

References

• http://mina.apache.org/udp-tutorial.html

In this post, I have tried to capture briefly on how to implement a UDP Client using Apache MINA. We shall concentrate on keeping our scope limited to sending data over UDP. In Subsequent posts, we shall see how we can enhance this UDP Client to send SNMP Traps.

In brief, these are the steps we need to perform to send a UDP packet

1. Create a Datagram Socket
2. Create a Datagram Packet
3. Send the packet

1. Create a NioDatagramConnector instance
2. Add an IoHandler (we can use an IoHandlerAdapter, as for this example we don't need to use all the API's)
3. Connect the NioDatagramConnector
4. Get the session
5. Write the data onto the session

There is a slight difference in which the MINA API's work. The API's work asynchronously. A call to connect() on NioDatagramConnector, return a reference to ConnectFuture and the operation is kicked off in a new thread. Lets see how the Code looks like and see it in details.

 

 

Lets see the Code in detail now. We have maintained the Session at the class level, assuming that we may reuse the session for sending packets, if session is still active. At line 25, we check if the session is active or not. If inactive we initiate the connection. 

Line 28, creates an instance of NioDatagramConnector

Line 31, add an IoHandler to the NioDatagramConnector on Line 28. In MINA terminology, IoHandler contains the code where Business logic of an Application resides. In our case its a pure Adapter, with only log statements inside the functions.

Line 32, a call to connect(), returns an reference to ConnectFuture. The call, starts the connection operation, in a separate thread and returns to the caller. To cater to this asynchronous behaviour, we add an IoFutureListener, to returned ConnectFuture reference. Out there (Line 38-43), we just check if the connection is OK, we get the session from it. Line 39, signals that connection is connected and its safe to get the session for writing data.

The connect code is complete. Lets see how to send the data.

The send function is pretty simple. We just create an IoBuffer and call write on the session. We have called connect() on Line 59, just to ensure that the connection is alive. If it isn't, it shall create a connection and return a session back.

NOTE: This code is in its very simple form. There can be various mechanism to handle Session (based on destination etc). Also, please be aware of the asynchronous behaviour of the NioDatagramConnector connect() API. If the connection is not complete in time, you might see a null pointer at Line 67 . But this is usually the case. For production code, it is advisable to ensure that session is never null.

That's it, out UDP client is ready. To create an SNMP Trap sender, we just need to pass Trap byte array to send() and we are done. Will try to post that article soon.

References

• http://mina.apache.org/udp-tutorial.html

Lets have a look how the application architecture looks like

Lets briefly see what the Major components are

IO service - The API's at this level are responsible for performing actual IO operation over the network

IO Filters - Filters are the mechanism by which MINA lets a User plugin its code to manipulate the byte stream from the network. Filters are arranged in Filter chain and are called by MINA in the order in which they are present. Filters are mainly used for transforming byte stream into high level objects.

IO Handler - IO Handler is the heart of application logic. Here is where the whole application logic is concentrated.

MINA is JMX enabled, hence Filters can be dynamically added/removed from the Filter chain.

The framework comes with samples. Please feel free to try them out.

This is how a static mock code looks like

Static Mock

Even when using a mocking framework, we have to define the state of the object we want to return, which means compiling and redploying especially when testing deployed applications. Can frameworks be smart enough to take care of all data variation? unfortunately, the answer is no. Frameworks provide various ways of reducing the effort, but can't make it zero. Hope its always true, else Develoeprs like me shall loose their jobs.

Coming back to the topic. With a little overhead we can make this mock dynamic, so that we can change the state of the object without compiling and redeplying

Let me introduce beanshell. Beanshell is a framework that supports lightweight scripting in Java and it now JSR274. It allows to write scripts in Java syntax, with type relaxation. Now lets see how can we make out Status mock, a dynamic mock by using beanshell script. many IDE's support beanshell scripting as well. Other options available are JRuby and Groovy.

Lets change the Mocked Object code to support beanshell

We create an instance of BSH Interpreter and call the file containing the implementation of the function. I prefer seperate file for seperate functions, to have exercise more control. The difference here is, we have moved the main logic from compiled code to interpreted code. This way, we can change the logic at runtime. We just change the bsh file and the interpreter executes the new version.

Lets look at bsh file

This post displays the cocnept in a very simplistic manner. There can be many variations like implementing the interfaces completely in scripting. Scripting can used as an add-on to existing mocking frameworks taking the Unit Testing to next level.

References

• http://www.beanshell.org/

In this post we shall take an mpp and update the Google Calendar with the task details. The question is why would anyone do that? 

Lets cook the story. A small team is working and the poor Project Manager share the Schedule with the team. The team is so busy coding that they forget to look at the dates, and the tasks they must be starting on. The poor Project Manager himself loose track unless he keeps MS Project Open on his desktop :-( A pretty common situation  

Have faced this situation and decided to organize myself and this post is outcome of that. The driving force was "Out of sight, is out of mind". So my assumption was if the team has a visual of the task dates, the chances of task slippage shall reduce a lot. 

In this post, have built upon the my previous post to create an MPP parsing function. Have tried to keep the code in simplest form, so that it can customized later to specific needs. Have created a sample Schedule for this activity and used it for the implementation. Here is how it looks.

Google Calendar updater Project Schedule

The schdule is simple, some date, and tasks with dependencies. The concept shall work for any number of tasks.

Following are the steps that we need to perform:

1. Parse the mpp file
2. Create a Calendar Service (Lets assume that the Calendar is already there, though it is possible to programmatically create the same)
3. Get All tasks from MPP, and create Calendar Entries in Google Calendar

Google Calendar Entry Creation Code

The implementation is fairly simple. We have 3 functions:

• translateTaskToCalendarEventEntry() - Converts an Task into CalendarEventEntry, which can be added to Google Calendar
• updateCalendar() - Takes input of all the Tasks from MPP, translates into CalendarEventEntry (using function translateTaskToCalendarEventEntry) and update the Calendar
• updateGoogleCalendarWithMPP() - Parses the MPP using the util function from my previous post, and calls the API above to update the Calendar.

Google Calendar API usage is described in details on Google's pages, hence won't discuss here.
To use it for yourself, replace User Name and Password at line#70, with your User Credentials. Repace the URL at Line# 71 to point to your calendar.
After running the program, this is how my Calendar looked like 

Google Calendar View

A simple way to keep track of Tasks.