<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	>

<channel>
	<title>Ashish's Tech Blog</title>
	<atom:link href="http://www.ashishpaliwal.com/blog/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.ashishpaliwal.com/blog</link>
	<description>From Programmer, For Programmers</description>
	<pubDate>Wed, 10 Feb 2010 02:11:41 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.7</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>So you want Distributed, Scalable and Highly Available Cache?</title>
		<link>http://www.ashishpaliwal.com/blog/2010/02/so-you-want-distributed-scalable-and-highly-available-cache/</link>
		<comments>http://www.ashishpaliwal.com/blog/2010/02/so-you-want-distributed-scalable-and-highly-available-cache/#comments</comments>
		<pubDate>Tue, 09 Feb 2010 09:22:53 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Ehcache]]></category>

		<category><![CDATA[Terracotta]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=402</guid>
		<description><![CDATA[Abstract
So you want Distributed, Scalable and Highly Available Cache? If yes, then this is the right place for you. The Terracotta Ehcache release has these features inbuilt. The Express Installation mode has simplified Terracotta integration in your application. 
How this post is organized
First we shall start with simple standalone cache and then see how Terracotta [...]]]></description>
			<content:encoded><![CDATA[<p><strong>Abstract</strong></p>
<p><em><strong>So you want Distributed, Scalable and Highly Available Cache?</strong></em> If yes, then this is the right place for you. The Terracotta Ehcache release has these features inbuilt. The <a href="http://www.terracotta.org/documentation/ga/product-documentation-2.html#382351408_pgfId-11318">Express Installation</a> mode has simplified Terracotta integration in your application. </p>
<p><strong>How this post is organized</strong><br />
First we shall start with simple standalone cache and then see how Terracotta can easily help us create a Distributed, Scalable and Highly Available cache, with a few minor configurations</p>
<p><strong>What all do you need run the example?</strong></p>
<ul>
<li>Terracotta 3.2.0 - Download it from <a href="http://www.terracotta.org/dl/">here</a></li>
</ul>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x15LinkUnit */
google_ad_slot = "4400881690";
google_ad_width = 468;
google_ad_height = 15;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
<p><strong>Sample Application</strong><br />
Lets design a sample application (a real simple one), which we shall use to demonstrate the features. Device Monitoring is a common requirement in OSS System, and caching the Device information reduces the Database hits. The essential components of our application are<br />
- DevideInfo - A POJO that stores device information<br />
- CacheHandler - Class that handles cache initialization and other ops</p>
<p>Lets see the CacheHandler class, which is the heart of our implementation</p>
<pre class="brush: java">
public class DeviceInfo  implements Serializable {

    String deviceId;

    String name;

    // .... other device information

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getDeviceId() {
        return deviceId;
    }

    public void setDeviceId(String deviceId) {
        this.deviceId = deviceId;
    }
}
</pre>
<p>The class is simple and contains information related to the device.</p>
<p>Let see our CacheHandler class</p>
<pre class="brush: java">
public class CacheHandler {

    public Cache deviceCache;

    public void initCache() {
        // Initialize the CacheManager
        CacheManager cacheManager = CacheManager.create();
        // Get the Cache to store device information
        deviceCache = cacheManager.getCache(&quot;deviceCache&quot;);
    }

    public void addToCache(DeviceInfo device) {
        Element el = new Element(device.getDeviceId(), device);
        deviceCache.put(el);
    }

    protected int getDeviceCacheSize() {
        return deviceCache.getSize();
    }

    public static void main(String[] args) {
        CacheHandler handler = new CacheHandler();
        handler.initCache();

        // Lets add 10 devices
        // just to keep life simple
        for(int i = 0; i &lt; 10; i++) {
            DeviceInfo device = new DeviceInfo();
            device.setDeviceId(&quot;Device-&quot;+i);
            handler.addToCache(device);
        }

        // Not recommended in production
        System.out.println(&quot;Cache Size = &quot;+handler.getDeviceCacheSize());

    }
}
</pre>
<p>The class has two main functions<br />
- initCache - The API initializes the cache<br />
- addToCache - The API adds the device information to the device cache</p>
<p>Lets see our ehcache.xml</p>
<pre class="brush: xml">
&lt;ehcache xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
         xsi:noNamespaceSchemaLocation=&quot;ehcache.xsd&quot;
             updateCheck=&quot;true&quot; monitoring=&quot;autodetect&quot;&gt;

   &lt;defaultCache
            maxElementsInMemory=&quot;10000&quot;
            eternal=&quot;false&quot;
            timeToIdleSeconds=&quot;120&quot;
            timeToLiveSeconds=&quot;120&quot;
            overflowToDisk=&quot;true&quot;
            diskSpoolBufferSizeMB=&quot;30&quot;
            maxElementsOnDisk=&quot;10000000&quot;
            diskPersistent=&quot;false&quot;
            diskExpiryThreadIntervalSeconds=&quot;120&quot;
            memoryStoreEvictionPolicy=&quot;LRU&quot;
            /&gt;

    &lt;cache name=&quot;deviceCache&quot;
           maxElementsInMemory=&quot;1000000&quot;
           maxElementsOnDisk=&quot;1000&quot;
           eternal=&quot;false&quot;
           overflowToDisk=&quot;false&quot;
           diskSpoolBufferSizeMB=&quot;20&quot;
           timeToIdleSeconds=&quot;100&quot;
           timeToLiveSeconds=&quot;100&quot;
           memoryStoreEvictionPolicy=&quot;LFU&quot;&gt;
     &lt;/cache&gt;

&lt;/ehcache&gt;
</pre>
<p>In the configuration we have defined a deviceCache, that we shall use to store device information.</p>
<p>Following is the list of jars you would need to run this app </p>
<ul>
<li>ehcache-core-1.7.2.jar</li>
<li>slf4j-api-1.5.8.jar</li>
<li>slf4j-jdk14-1.5.8.jar</li>
</ul>
<p>All these jars are available as part of Terracotta installation under <TC Install Dir>/distributed-cache directory</p>
<p>You can run this example and see the same in action.</p>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x60ImageOnly */
google_ad_slot = "0221726572";
google_ad_width = 468;
google_ad_height = 60;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
<p><strong>Making it Distributed....</strong></p>
<p>Now lets assume that we want to make the Cache fault tolerant, survive JVM crashes, as well as have it available on different JVM's. It should be able to Scale as we add more JVM's to it and other stuff. With Terracotta, we can just do it in a few steps without changing the code. the magic lies with Express Installation mode, introduced in Terracotta version 3.2.0 and above</p>
<p>Lets see the 3 steps that we need to perform</p>
<p>1. Updates to the ehcache.xml<br />
Add following element to ehcache.xml</p>
<p>&lt;terracottaConfig url=&quot;localhost:9510&quot; /&gt;</p>
<p>and for the deviceCache, we add &lt;terracotta /&gt; element</p>
<p>here is the updated ehcache.xml</p>
<pre class="brush: xml">
&lt;ehcache xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
         xsi:noNamespaceSchemaLocation=&quot;ehcache.xsd&quot;
             updateCheck=&quot;true&quot; monitoring=&quot;autodetect&quot;&gt;

   &lt;terracottaConfig url=&quot;localhost:9510&quot; /&gt; 

   &lt;defaultCache
            maxElementsInMemory=&quot;10000&quot;
            eternal=&quot;false&quot;
            timeToIdleSeconds=&quot;120&quot;
            timeToLiveSeconds=&quot;120&quot;
            overflowToDisk=&quot;true&quot;
            diskSpoolBufferSizeMB=&quot;30&quot;
            maxElementsOnDisk=&quot;10000000&quot;
            diskPersistent=&quot;false&quot;
            diskExpiryThreadIntervalSeconds=&quot;120&quot;
            memoryStoreEvictionPolicy=&quot;LRU&quot;
            /&gt;

    &lt;cache name=&quot;deviceCache&quot;
           maxElementsInMemory=&quot;1000000&quot;
           maxElementsOnDisk=&quot;1000&quot;
           eternal=&quot;false&quot;
           timeToIdleSeconds=&quot;100&quot;
           timeToLiveSeconds=&quot;100&quot;
           memoryStoreEvictionPolicy=&quot;LFU&quot;&gt;

        &lt;terracotta /&gt;
     &lt;/cache&gt;

&lt;/ehcache&gt;
</pre>
<p>There is no need to change the code <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<p>2. Add ehcache-terracotta-1.8.0.jar to the classpath</p>
<p>Lets start the Terracotta Server</p>
<p>3. Goto Terracotta_Install_dir/bin and execute </p>
<blockquote><p>$ ./start-tc-server.sh</p></blockquote>
<p>This shall start the Terracotta server</p>
<p>Now run the application on multiple JVM's. The device information is available on all client JVM's <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<p>To monitor Cache, you can use <a href="http://www.terracotta.org/confluence/display/docs/Terracotta+Developer+Console">Terracotta Dev Console</a></p>
<p>If you have further question/queries, please visit <a href="http://forums.terracotta.org/forums/forums/list.page">Terracotta Forums</a>.</p>
<p><strong>What's coming up Next</strong>?<br />
The next post shall touch on write-behind feature in the latest <a href="http://www.terracotta.org/beta/darwin">Terracotta Darwin</a> release. Its essentially to get best of both worlds - Caching and Database Offloading <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> Stay tuned..</p>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 300x250, created 2/9/10 */
google_ad_slot = "1177095982";
google_ad_width = 300;
google_ad_height = 250;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2010/02/so-you-want-distributed-scalable-and-highly-available-cache/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Exploring Terracotta tim-async - Scalable way to off-load DB updates from App Transaction</title>
		<link>http://www.ashishpaliwal.com/blog/2010/01/demystifying-terracotta-tim-async-scalable-way-to-off-load-db-updates-from-app-transaction/</link>
		<comments>http://www.ashishpaliwal.com/blog/2010/01/demystifying-terracotta-tim-async-scalable-way-to-off-load-db-updates-from-app-transaction/#comments</comments>
		<pubDate>Mon, 25 Jan 2010 14:59:15 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Terracotta]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=373</guid>
		<description><![CDATA[
Lets start this chat with a small conversation
Manager: What are the options we have to increase the TPS of our SMSC Server?
Architect: Can you elaborate a bit on this?
Manager: As of now we process a request and update the Database in the same tranaction. Essentially, we are operating our Server at the TPS at which our DB [...]]]></description>
			<content:encoded><![CDATA[<div>
<p>Lets start this chat with a small conversation</p>
<p><strong>Manager:</strong> What are the options we have to increase the TPS of our <a href="http://en.wikipedia.org/wiki/SMSC">SMSC</a> Server?</p>
<p><strong>Architect:</strong> Can you elaborate a bit on this?</p>
<p><strong>Manager:</strong> As of now we process a request and update the Database in the same tranaction. Essentially, we are operating our Server at the TPS at which our DB can operate.</p>
<p><strong>Architect:</strong> Well we have to ensure that our Data is persisted</p>
<p><strong>Manager:</strong> Yes, but can't we update the DB asynchronously, without lossing data</p>
<p><strong>Architect:</strong> Yes it can be done, but we have to create a lot of infrastructure code around this, so that we don't loose data for failed transaction, process crashes etc.</p>
<p>If you have been in such a situation, the post is meant for you.<br />
<script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x15, created 1/25/10 */
google_ad_slot = "2118617107";
google_ad_width = 468;
google_ad_height = 15;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
<h2>Summary</h2>
<p><strong><em>For any request, don't update the Database in same transaction, but data need to be persisted</em></strong>. This has always been a typical need to increase the liveliness of the application. The application accessing data in memory work must faster, than fetching/updating data form databases. Assume you have a solution which helps you to work in memory and allows your application to update the data to Datastore in separate transaction, providing HA and Scalability out of the box. If this is what you are looking at, Terracotta async processor (tim-async) is the right fit for you.</p>
<p>The article is organized to detail what tim-async is, followed by a sample application.</p>
<p>We shall see tim async in brief and then later would walk through the a small POC that I build around the same.</p>
<h2>Pre-requisite</h2>
<p>You should have basic understanding of Terracotta 3.2 and its Configuration.<br />
You need to download tim-async using tim-get tool</p>
<h2>What is tim-async?</h2>
<p><strong><em>tim</em></strong> stands for <strong><em>Terracotta Integration Module</em></strong>, and <strong><em>tim-async</em></strong> is module for asynchronous processing. tim-async provides a scalable, high performance way to asynchronously write the business data to data source (typically a database), while the application works on in-memory data structures. Decoupling the main processing from the underlying database decreases the write latency of domain objects, while high availability of data is provided by terracotta.</p>
<h3>What features does it have</h3>
<ul>
<li>Multithreaded write behind to flush the processed data from the client VMs (L1s) to the data source</li>
<li>Data to be flushed remains highly available (HA) as it is shared with terracotta.</li>
<li>Every client takes the responsibility of writing its data set to the data source</li>
<li>Listeners to monitor work progress at bucket level.</li>
</ul>
<p>NOTE: Have pulled out this information from the attachment <a href="http://www.terracotta.org/confluence/display/wiki/Notes+on+Tim-Async">here</a></p>
<h3>Building Blocks of tim-async from User Perspective</h3>
<p>From a User's POV, these three are the important building blocks</p>
<ul>
<li>ItemProcessor</li>
<li>AsyncCoordinator</li>
<li>AsyncConfig</li>
</ul>
<p>These are all classes/interface that are part of tim-async. Lets look at them in brief</p>
<h3>ItemProcessor</h3>
<p>Its an interface and the implementation class of this interface is where we shall be writing the actual processing logic of the Domain Object, like persisting it to the DB.</p>
<pre class="brush: java">
public interface ItemProcessor&lt;I&gt; {
public void process(I item) throws ProcessingException;
}
</pre>
<h3>AsyncCoordinator</h3>
<p>Its a core class and need to be declared as Shared Root in tc-config.xml. It allows work to be added and processed asynchronously.<br />
The class has a method start() that needs to be called by every client node (Terracotta L1) to participate in processing.<br />
An ItemProcessor instance is local to the node, so its the right place to hold stuff DB Connection etc</p>
<h3>AsyncConfig</h3>
<p>Its an interface and can be implemented to custom tailor the configuration of Async processing. If we don't define one, the org.terracotta.modules.async.configs.DefaultAsyncConfig is used.</p>
<pre class="brush: java">
public interface AsyncConfig {
public long getWorkDelay();

public long getMaxAllowedFallBehind();

public boolean isStealingEnabled();
}
</pre>
<p>Now lets see how can we implement tim-async in our application (We shall see the sample application)</p>
<p>- We need to implement ItemProcessor for doing custom processing say for example putting the stuff in DB<br />
- We need to initialize AsyncCoordinator and call start() API, pass on the ItemProcessor from previous step as a param.<br />
- Create tc-config.xml (pick teh sample from tim-async example), add the AsyncCoordinator field instance as the root<br />
- Start TC Server and Client JVM's with tc-config.xml created above<br />
<script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x15, created 1/25/10 */
google_ad_slot = "2118617107";
google_ad_width = 468;
google_ad_height = 15;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script><br />
Lets see these steps in Action.</p>
<h2>Sample Application</h2>
<p>I had written a SMSC simulator based on jsmpp which used to receive the SubmitSM request from Client, used to persist in the DB and return<br />
the id to the client. There is a specific reason why I choose this application. I already had this running and since the SubmitSM just needed to persisted in DB for subsequent processing by other process. The business case was justified that the Server is very responsive to Client, as well as its ensured that the Data is persisted.</p>
<p>Lets walk through the a simple processing sequence</p>
<ul>
<li>Client send a SubmitSM request</li>
<li>Server translated it into an internal SMS POJO, assigns a unique ID to the SMS</li>
<li>Server persists the SMS in Database</li>
<li>Sever sends the ID generate as part of SubmitSM response</li>
</ul>
<p><em>The application uses Hibernate to save the POJO in MySQL Database</em>.</p>
<p>Since the Domain object is least significant so won't discuss the internals of SMS POJO</p>
<p>Lets see how we enhanced the application to asynchronously update the DB</p>
<h3>Implementing the ItemProcessor</h3>
<pre class="brush: java">
public class AsyncSMSProcessor implements ItemProcessor {
public void process(final SMS sms) throws ProcessingException {
Session hibernateSession = DAO.getSession();
DAO.getSession().getTransaction().begin();
DAO.getSession().save(sms);
DAO.getSession().getTransaction().commit();
}
}
</pre>
<p>Essentially what we have to write the logic of saving the SMS object to the database. This is the object that Terracotta is going to pass back to us.<br />
What we do here is save the SMS in a Hibernate transaction. This completes out ItemProcessor code</p>
<h3>Initialize AsyncCoordinator</h3>
<pre class="brush: java">
public class SMSProcessor {

public AsyncCoordinator asyncUpdator;

public SMSProcessor() {
asyncUpdator = new AsyncCoordinator();
// Use the default API

asyncUpdator.start(new AsyncSMSProcessor());

}

}
</pre>
<pre class="brush: java">
public long processIncomingSMS(SubmitSm submitSm,
SMPPServerSession source, ...) {

// some logic here
// give the date to AsyncCoordinator
asyncUpdator.add(sms);
// some logic there

}
</pre>
<p>This is the change that we have done here. Earlier we used to save the SMS in DB here. We have now given the data to AsyncCoordinator for subsequent processing.</p>
<p>Let's look at the tc-config.xml</p>
<p>this is what we need to add to the tc-config.xml</p>
<pre class="brush: xml">
&lt;application&gt;
&lt;dso&gt;
&lt;!--Declaring a field of a class a root will make it available for all instances
of our app that runs via DSO--&gt;
&lt;roots&gt;
&lt;!-- XXX: --&gt;
&lt;root&gt;
&lt;field-name&gt;com.tc.timasync.demo.smsAsyncCoordinator&lt;/field-name&gt;
&lt;/root&gt;
&lt;/roots&gt;
&lt;/dso&gt;
&lt;/application&gt;
</pre>
<p>We declared the AsyncCoordinator variable as DSO root here, which makes it shared across cluster.</p>
<p>That's it.</p>
<p>We are ready. Now we start Terracotta server with our tc-config file and start our SMSC Simulator as TC Client.</p>
<h3>How does it all fit together</h3>
<p>As we see the instead of updating the DB in the same transaction, we handed the data over to Terracotta. Essentially, this meant that our data has gone to Server, and won't be lost against crashes. The application operated at memory speed and responded to client. The Terracotta Server based on the configuration of tim-async shall call the ItemProcessor in different transaction. We can have multiple nodes updating the DB asynchronously or some other way, based upon the Use Case.</p>
<p>Let's run the both the implementations</p>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x15, created 1/25/10 */
google_ad_slot = "2118617107";
google_ad_width = 468;
google_ad_height = 15;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
<p><strong>Note on Comparison of both the implementations</strong></p>
<p>I shall run both the implementation on the same hardware/OS, which would be my Laptop with 2GB of RAM, runnning Windows and MYSQL 5.X. The comparison shall be relative for the two implementation approached, and not the absolute figures. The figures would be much better if the samples are run on higher end machined with Terracotta Server (L2) running on a dedicated machine and the Clients (L1) running on different machines. The idea here is just to see the TPS of the Server.</p>
<h3>Case I: DB updates in App transaction</h3>
<p>For 10 runs of sending 1000 SubmitSM request client took following time</p>
<p>Total Time for 1000 SMS = 28891<br />
Total Time for 1000 SMS = 34234<br />
Total Time for 1000 SMS = 27812<br />
Total Time for 1000 SMS = 34547<br />
Total Time for 1000 SMS = 29953<br />
Total Time for 1000 SMS = 34344<br />
Total Time for 1000 SMS = 33985<br />
Total Time for 1000 SMS = 35328<br />
Total Time for 1000 SMS = 36968</p>
<h3>Case II: DB updates based on tim-async</h3>
<p>Total Time for 1000 SMS = 1359<br />
Total Time for 1000 SMS = 1125<br />
Total Time for 1000 SMS = 781<br />
Total Time for 1000 SMS = 813<br />
Total Time for 1000 SMS = 750<br />
Total Time for 1000 SMS = 719<br />
Total Time for 1000 SMS = 703<br />
Total Time for 1000 SMS = 719<br />
Total Time for 1000 SMS = 718<br />
Total Time for 1000 SMS = 735</p>
<p>There was a significant decrease in the processing time of request <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<h2>What's next</h2>
<p>Download Terracotta from <a href="http://www.terracotta.org/dl/">here</a> and try yourself. You can also <a href="http://www.terracotta.org/beta/darwin">register for upcoming Darwin release</a>, which has many new interesting features</p>
<p><strong>Resources</strong></p>
<ul>
<li>http://blog.terracottatech.com/2009/02/offloading_a_db_even_when_upda.html - A wonderful post by Ari</li>
<li>http://www.slideshare.net/sbtourist/real-terracotta-presentation</li>
<li>http://forums.terracotta.org/forums/forums/list.page</li>
</ul>
</div>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 468x15, created 1/25/10 */
google_ad_slot = "2118617107";
google_ad_width = 468;
google_ad_height = 15;
//-->
</script><br />
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2010/01/demystifying-terracotta-tim-async-scalable-way-to-off-load-db-updates-from-app-transaction/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Joined Terracotta</title>
		<link>http://www.ashishpaliwal.com/blog/2010/01/joined-terracotta/</link>
		<comments>http://www.ashishpaliwal.com/blog/2010/01/joined-terracotta/#comments</comments>
		<pubDate>Thu, 07 Jan 2010 13:26:06 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Personal]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=368</guid>
		<description><![CDATA[Finally, after a sprint of 9+ years at Hughes family, I started a new career. Started my work life with Hughes Software Systems in 2000, then joined Hughes Systique in 2006.
Year 2009 was a year of happenings for me. As I got more involved with Open Source software, my thoughts towards Software Development changed drastically. [...]]]></description>
			<content:encoded><![CDATA[<p>Finally, after a sprint of 9+ years at Hughes family, I started a new career. Started my work life with Hughes Software Systems in 2000, then joined <a href="http://www.hsc.com">Hughes Systique</a> in 2006.</p>
<p>Year 2009 was a year of happenings for me. As I got more involved with Open Source software, my thoughts towards Software Development changed drastically. While involved in <a href="http://mina.apache.org">Apache MINA</a> mailing list, was introduced to <a href="http://www.terracotta.org">Terracotta</a>. Well this was the beginning and later on I got hired:-) (off-course I applied for it).</p>
<p>Well now the easy life is gone, have much more challenging tasks ahead and at Terracotta, I will be working with some of the finest talent in the industry. A long road ahead for me. The biggest reason for joining Terracotta, <strong>Passion</strong>. This little word was common between me and most of the folks I interacted at Terracotta. Its very close to my punch line "Great passions can defy Destiny".</p>
<p>With my joining Terracotta as Solution Architect, I am very hopeful that my contribution towards Open Source community will increase significantly.</p>
<p>Before I close this post, a lot of credit to this success goes to <a href="http://www.nextury.com">Emmanuel Lecharny</a>. He initiated me into Open Source and has been mentoring me all the way.</p>
<p>Wish you a very Happy New Year !</p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2010/01/joined-terracotta/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Essential Components of Terracotta based Application</title>
		<link>http://www.ashishpaliwal.com/blog/2009/11/essential-components-of-terracotta-based-application/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/11/essential-components-of-terracotta-based-application/#comments</comments>
		<pubDate>Thu, 26 Nov 2009 07:40:23 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Terracotta]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=353</guid>
		<description><![CDATA[Abstract
How does a Terracotta Application look like? What are the essential components that make up the application? These were the question I had after writing my first post “Getting Started with Terracotta”. Here I try to take a quick look at these questions from a Users point of view. Please note that this post doesn’t [...]]]></description>
			<content:encoded><![CDATA[<h2>Abstract</h2>
<p>How does a <a href="http://www.terracotta.org/" target="_blank">Terracotta</a> Application look like? What are the essential components that make up the application? These were the question I had after writing my first post “<a href="http://www.ashishpaliwal.com/blog/2009/11/getting-started-with-terracotta/" target="_blank">Getting Started with Terracotta</a>”. Here I try to take a quick look at these questions from a Users point of view. Please note that this post doesn’t try to dive deep into Terracotta inner working.</p>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 728x15, created 11/25/09 */
google_ad_slot = "8166984671";
google_ad_width = 728;
google_ad_height = 15;
// --></script><br />
<script src="http://pagead2.googlesyndication.com/pagead/show_ads.js" type="text/javascript"></script></p>
<div id="attachment_355" class="wp-caption aligncenter" style="width: 310px"><img class="size-medium wp-image-355" title="tc_app_comps" src="http://www.ashishpaliwal.com/blog/wp-content/uploads/2009/11/tc_app_comps-300x207.png" alt="Components of Terracotta based Application" width="300" height="207" /><p class="wp-caption-text">Components of Terracotta based Application</p></div>
<p>To keep life simple, have considered the general application architecture, which essentially can be any Terracotta app. From the figure we can see that we have four different parts that contribute to a complete Terracotta app.  </p>
<p><strong> User Application</strong> – These are applications that want to use Terracotta to Scale. They can either use Terracotta transparently or can use explicit Terracotta API’s  </p>
<p><strong> Terracotta Server</strong> – It’s the nerve center. Terracotta applications can’t run without it. For running any TC application, it’s a must to run atleast one TC Server.  </p>
<p><strong> Terracotta config file</strong> – Also known as tc-config.xml (default name). It provides the detailed configuration for Server as well as Client. It contains information like Server’s IP’s,  persistence mode, classes to instrument etc. </p>
<p><strong>Terracotta Libraries</strong> – These are essentially part of Client or User applications. They are the work horses of the running application and enables Terracotta integration with User application. An example would be dso-java.bat script adds the Terracotta bootjars to the classpath, enabling Terracotta features for the application.  </p>
<p>So how does it all fit together at runtime</p>
<ul>
<li>Start Terracotta Server, providing a configuration file with desired configuration</li>
<li>Start User Application using dso-java.(bat|sh) instead of normal java, along with a configuration file</li>
<li>That’s it <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </li>
</ul>
<p><script type="text/javascript"><!--
google_ad_client = "pub-6961884887741817";
/* 728x15, created 11/25/09 */
google_ad_slot = "8166984671";
google_ad_width = 728;
google_ad_height = 15;
// --></script><br />
<script src="http://pagead2.googlesyndication.com/pagead/show_ads.js" type="text/javascript"></script></p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/11/essential-components-of-terracotta-based-application/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Getting Started with Terracotta</title>
		<link>http://www.ashishpaliwal.com/blog/2009/11/getting-started-with-terracotta/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/11/getting-started-with-terracotta/#comments</comments>
		<pubDate>Mon, 23 Nov 2009 10:51:40 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Application Programming]]></category>

		<category><![CDATA[Java]]></category>

		<category><![CDATA[Add new tag]]></category>

		<category><![CDATA[Terracotta]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=340</guid>
		<description><![CDATA[Abstract
In this small post we shall explore Terracotta, a leading pure Java Scalability platform. The discussion is based on the AtomicInteger example from Terracotta site, which shows how to implement a Cluster wide id generator (Actually it’s the Sequencer example, but to keep my steps simple had used AtomicInteger). The reason why I choose this [...]]]></description>
			<content:encoded><![CDATA[<h2><strong>Abstract</strong></h2>
<p>In this small post we shall explore Terracotta, a leading pure Java Scalability platform. The discussion is based on the AtomicInteger example from Terracotta site, which shows how to implement a Cluster wide id generator (Actually it’s the Sequencer example, but to keep my steps simple had used AtomicInteger). The reason why I choose this example was coz of a very similar requirement that I had to implement in Clustered J2EE application.<br />
Terracotta is well known and needs no introduction <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> </p>
<h2><strong>About problem Statement</strong></h2>
<p>Well, I needed a simple solution to have<br />
•	Cluster wide unique id’s<br />
•	Less frequent access to these id’s<br />
•	Optional Persistence</p>
<p><strong>NOTE:</strong> Please note that the current example is slightly modified version of example from terracotta.org.</p>
<h2>Pre-requisite</h2>
<p>To run this example, you need to have following installed<br />
•	Terracotta<br />
•	And JDK offcourse <img src='http://www.ashishpaliwal.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /><br />
Let’s take a look at the sample code first</p>
<pre class="brush: java">
import java.util.concurrent.atomic.*;

public class IdGenerator
{

private AtomicInteger masterCounter = new AtomicInteger(0);

private AtomicInteger slaveCounter = new AtomicInteger(1024);

public int getNewMasterId()
{
int newId = masterCounter.getAndIncrement();
System.out.println(&quot;Master Id is: &quot; + newId);
return newId;
}

public int getNewSlaveId()
{
int slaveId = slaveCounter.getAndIncrement();
System.out.println(&quot;Slave Id is: &quot; + slaveId);
return slaveId;
}

public static void main(String[] args)
{
new IdGenerator().getNewMasterId();
}
}
</pre>
<p>The code has two fields ids master and slave, of which master id need to be unique cluster wide. The code is pretty straight forward, and there is nothing to explain. The id’s are generated by simple getAndIncrement() API.</p>
<p>Lets see the tc-config.xml</p>
<pre class="brush: xml">

&lt;tc:tc-config xmlns:tc=&quot;http://www.terracotta.org/config&quot;
xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
xsi:schemaLocation=&quot;http://www.terracotta.org/schema/terracotta-4.xsd&quot;&gt;

&lt;application&gt;
&lt;dso&gt;
&lt;roots&gt;
&lt;root&gt;
&lt;field-name&gt;IdGenerator.masterCounter&lt;/field-name&gt;
&lt;/root&gt;
&lt;/roots&gt;
&lt;/dso&gt;
&lt;/application&gt;
&lt;/tc:tc-config&gt;
</pre>
<p>Since only master Id needed to be unique in this case, hence the configuration</p>
<p>Let’s look at the deployment architecture</p>
<div id="attachment_343" class="wp-caption aligncenter" style="width: 310px"><img class="size-medium wp-image-343" title="nodes" src="http://www.ashishpaliwal.com/blog/wp-content/uploads/2009/11/nodees-300x134.png" alt="Deployment Architecture" width="300" height="134" /><p class="wp-caption-text">Deployment Architecture</p></div>
<p>The application was to run on 3 nodes (3 JVM’s on same machine) and one Terracotta server running.<br />
Before we get into running this application, lets take a look at some useful scripts provided by Terracotta<br />
•	<strong>dso-java (bat|sh)</strong> – startup script that bootstraps Terracotta libraries to your application<br />
•	<strong>start-tc-server (bat|sh)</strong> – Script to start Terracotta Server. It is mandatory to start Terracotta Server before running Clients</p>
<p>Let’s run the application<br />
1.	Compile the application. There is no dependency on TC libraries.<br />
2.	First Start the terracotta Server using script start-tc-server<br />
3.	Run 3 separate JVM with command “dso-java IdGenerator”<br />
That’s it. Our first application is running. If we observe the console output, you can see that we have unique id’s for each invocation.</p>
<h2>References:</h2>
<p>Source code for Terracotta examples - http://svn.terracotta.org/svn/forge/cookbook</p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/11/getting-started-with-terracotta/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Executors in Action - Part 1 - The Life cycle</title>
		<link>http://www.ashishpaliwal.com/blog/2009/09/executors-in-action-part-1-the-life-cycle/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/09/executors-in-action-part-1-the-life-cycle/#comments</comments>
		<pubDate>Tue, 29 Sep 2009 10:59:36 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Java]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=322</guid>
		<description><![CDATA[From the Javadoc
… interface provides a way of decoupling task submission from the mechanics of how each task will be run, including details of thread use, scheduling, etc. An Executor is normally used instead of explicitly creating threads.
Indeed, Executors provide a more robust and easy mechanism for decoupling of submitting a task, from its execution.
In [...]]]></description>
			<content:encoded><![CDATA[<p>From the Javadoc</p>
<p>… interface provides a way of decoupling task submission from the mechanics of how each task will be run, including details of thread use, scheduling, etc. An Executor is normally used instead of explicitly creating threads.</p>
<p>Indeed, Executors provide a more robust and easy mechanism for decoupling of submitting a task, from its execution.</p>
<p>In this series, we shall explore this concurrency construct in detail.</p>
<p>NOTE: Have tried to avoid explaining what is already available in javadoc.</p>
<p>Let’s first see the lifecycle of an Executor</p>
<p>ExecutorService extends the Executor interface, adding a bunch of life cycle methods.</p>
<p><div id="attachment_328" class="wp-caption alignnone" style="width: 475px"><img src="http://www.ashishpaliwal.com/blog/wp-content/uploads/2009/09/executorstatemachine1.png" alt="Executor Life Cycle" title="executorstatemachine1" width="465" height="220" class="size-full wp-image-328" /><p class="wp-caption-text">Executor Life Cycle</p></div><br />
Let’s create a simple program to demonstrate the same</p>
<pre class="brush: java">
public static void main(String[] args) {
// Executor doesn&#039;t exist before this step
// Executor created and moved to running state
ExecutorService executor = Executors.newSingleThreadExecutor();

// Executor into running state
executor.execute(new Runnable() {
public void run() {
System.out.println(&quot;Running...&quot;);
}
});

// Lets shutdown
executor.shutdown();

System.out.println(&quot; isShutdown = &quot;+executor.isShutdown());
System.out.println(&quot; isTermianted = &quot;+executor.isTerminated());
}
</pre>
<p>The program depicts the complete lifecycle. Please see the comments inline in the code</p>
<p>Initially the Executor doesn’t exist. When we create an Executor, using Executor’s we have it in Ready state (ready to accept tasks for execution).<br />
Once we submit a task for execution, the Executor moves to running state. Once it completes the task, it moves back to ready state.</p>
<p>Once we call <strong>shutdown()</strong> on Executor, it begins its shutdown cycle. Once it is completed it moves to Terminated state.</p>
<p>There are couple of things that need to be noted after shutdown is initiated. An Executor can be executing or may have couple of tasks pending to be executed.</p>
<p>Calling <strong>shutdown()</strong> initiates an orderly shutdown, means all the pending tasks shall be completed, but new tasks won’t be accepted</p>
<p>Calling <strong>shutdownNow()</strong> initiates a shutdown, by trying to halt the processing the tasks by interrupting the threads. The method returns a list of Runnable’s which were not executed. Programs can use them to reschedule them later or discard them.</p>
<p>What will happen, if we submit a task once shutdown has been initiated?<br />
This situation can be handled by using RejectExecutionHandler. The default implementation simply discards the tasks. We can have the custom logic to handle the tasks that couldn’t be executed.</p>
<p>Upcoming topic – Exploring a little on ScheduledExecutorService</p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/09/executors-in-action-part-1-the-life-cycle/feed/</wfw:commentRss>
		</item>
		<item>
		<title>MINA vs Netty Series - Closing down</title>
		<link>http://www.ashishpaliwal.com/blog/2009/09/mina-vs-netty-series-closing-down/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/09/mina-vs-netty-series-closing-down/#comments</comments>
		<pubDate>Thu, 17 Sep 2009 15:08:31 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=318</guid>
		<description><![CDATA[Folks after posting two articles in the series, I have decided to hold my work on this for a while. I am working of new MINA User Guide, and will be busy over there.
However, this is not an end to this. May be sometime in the future, when I am have bandwidth available shall start [...]]]></description>
			<content:encoded><![CDATA[<p>Folks after posting two articles in the series, I have decided to hold my work on this for a while. I am working of new MINA User Guide, and will be busy over there.</p>
<p>However, this is not an end to this. May be sometime in the future, when I am have bandwidth available shall start working on this again.</p>
<p>You can check the new MINA User Guide at the following link http://mina.apache.org/user-guide.html</p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/09/mina-vs-netty-series-closing-down/feed/</wfw:commentRss>
		</item>
		<item>
		<title>If-else vs switch – Which is better?</title>
		<link>http://www.ashishpaliwal.com/blog/2009/08/if-else-vs-switch-%e2%80%93-which-is-better/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/08/if-else-vs-switch-%e2%80%93-which-is-better/#comments</comments>
		<pubDate>Sun, 02 Aug 2009 02:16:35 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Java]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=303</guid>
		<description><![CDATA["Use switch instead of if-else, its more readable and has better performance." I have to admit that this was one of my favorite code review comment. Until one fine day, while hacking Apache Sanselan's image format decoding function, I tried optimizing the code based on the same comments and while benchmark there was hardly any [...]]]></description>
			<content:encoded><![CDATA[<p>"Use switch instead of if-else, its more readable and has better performance." I have to admit that this was one of my favorite code review comment. Until one fine day, while hacking Apache Sanselan's image format decoding function, I tried optimizing the code based on the same comments and while benchmark there was hardly any difference. I thought about investigating it further. Though found some interesting mail chains, though about posting my finding.</p>
<p>To begin with decided to run some samples on switch and if-else constructs and analyze further.</p>
<p>Wrote three function<br />
1. For if-else - a if-else ladder based on int comparisons<br />
2. For Switch - switch with 21 cases, from 1 to 20<br />
3. For Switch - switch with sparse random values</p>
<p>The reason for choosing two functions for switch was this statement from <a href="http://java.sun.com/docs/books/jvms/second_edition/html/Compiling.doc.html#14942">VM spec</a> <em>"Compilation of switch statements uses the tableswitch and lookupswitch instructions"</em></p>
<p>Lets see the function codes</p>
<h3>if-else</h3>
<pre class="brush: java">
public static void testIfElse(int jumpLabel) {
if(1 == jumpLabel) {
    System.out.println(&quot;1&quot;);
} else if(2 == jumpLabel) {
    System.out.println(&quot;2&quot;);
} else if(3 == jumpLabel) {
    System.out.println(&quot;3&quot;);
} else if(4 == jumpLabel) {
    System.out.println(&quot;4&quot;);
}
// Removed for simplicity
else {
    System.out.println(&quot;default&quot;);
}
}
</pre>
<p>Lets see the switch functions</p>
<h3>Finite switch version</h3>
<pre class="brush: java">
public static void testSwitchFinite(int jumpLable) {

switch (jumpLable) {
case 1:
    System.out.println(&quot;1&quot;);
    break;

case 2:
    System.out.println(&quot;2&quot;);
    break;

case 3:
    System.out.println(&quot;3&quot;);
    break;

case 4:
    System.out.println(&quot;4&quot;);
    break;

case 5:
    System.out.println(&quot;5&quot;);
    break;

// Removed other cases for simplicity

default:
    System.out.println(&quot;default&quot;);
    break;
}
}
</pre>
<h3>Sparse switch version</h3>
<pre class="brush: java">
public static void testSwitchSparse(int jumpLable) {

switch (jumpLable) {
case 100:
    System.out.println(&quot;1&quot;);
    break;

case -1:
    System.out.println(&quot;2&quot;);
    break;

case 5000:
    System.out.println(&quot;3&quot;);
    break;

case -8:
    System.out.println(&quot;4&quot;);
    break;

case 1600:
    System.out.println(&quot;5&quot;);
    break;

case 250:
    System.out.println(&quot;250&quot;);
    break;

// Removed other cases for simplicity

default:
    System.out.println(&quot;default&quot;);
    break;
}
}
</pre>
<p>With the groundwork done, its the benchmarking time. The benchmarking strategy was simple. Run these functions in loop and see the result, with iteration ranging from 100 to 1000.</p>
<p>Lets look at one of the functions</p>
<pre class="brush: java">
public static void testSwitchPerf(int iteration) {
    long t1 = System.nanoTime();
    for (int i = 0; i &lt; iteration; i++) {
        testSwitchFinite(i);
    }
    long t2 = System.nanoTime();
    System.out.println(&quot;Time Taken (switch) = &quot;+(t2 - t1)/1000000);
}
</pre>
<p>Well this was the ground work, after executing the conditions, here is the data</p>
<table border="1" width="100%">
<tbody>
<tr>
<td>Iteration -&gt;</td>
<td>100</td>
<td>1000</td>
</tr>
<tr>
<td>if-else</td>
<td>8 ms</td>
<td>69 ms</td>
</tr>
<tr>
<td>switch finite</td>
<td>3 ms</td>
<td>34 ms</td>
</tr>
<tr>
<td>switch sparse</td>
<td>7 ms</td>
<td>21 ms</td>
</tr>
</tbody>
</table>
<p>NOTE: There is some difference due to the way data is provided and the sample space doesn't provide precise results. However, since the sample space is same for both, it would serve its purpose</p>
<h3>Conclusion</h3>
<p>1. There is no significant execution difference between if-else and switch. The difference observed is due to the sample space choosen.<br />
2. If using if-else, its always recommended to put frequently used if condition at the top of if-else ladder<br />
3. The finite switch statement was converted to tableswitch and sparse switch was converted to lookupswitch</p>
<p>would be interested to hear from folks about their experience in relation to this. I would say, i still prefer switch for readability. Henceforth, my review comment shall be modified as "Use switch instead of if-else, its more readable <del datetime="2009-07-28T07:33:30+00:00">and has better performance</del>"</p>
<h3>Share your thoughts</h3>
<p>I would be very keen to hear from you all, about your opinions and experience about the topic</p>
]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/08/if-else-vs-switch-%e2%80%93-which-is-better/feed/</wfw:commentRss>
		</item>
		<item>
		<title>[Apache Sanselan] Demystifying how Sanselan determines image format</title>
		<link>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-demystifying-how-sanselan-determines-image-format/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-demystifying-how-sanselan-determines-image-format/#comments</comments>
		<pubDate>Sat, 18 Jul 2009 16:08:20 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Apache Sanselan]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=292</guid>
		<description><![CDATA[Apache Sanselan is a pure Java library for reading and Writing Image formats. It has recently graduated from Incubator and is now a proud member of Apache commons proper.
In previous articles, we saw how to retrieve Image metadata and information. In this post we shall see how Sanselan guesses the Image format. We shall take [...]]]></description>
			<content:encoded><![CDATA[<p>Apache Sanselan is a pure Java library for reading and Writing Image formats. It has recently graduated from Incubator and is now a proud member of Apache commons proper.</p>
<p>In previous articles, we saw how to retrieve Image metadata and information. In this post we shall see how Sanselan guesses the Image format. We shall take it in two steps</p>
<ul>
<li>First we shall look at the ImageFormat class</li>
<li>Then, we shall look into the implementation guessFormat() API</li>
</ul>
<h4><span style="text-decoration: underline;">ImageFormat class</span></h4>
<p><em><a href="http://svn.apache.org/viewvc/commons/proper/sanselan/trunk/src/main/java/org/apache/sanselan/ImageFormat.java?view=log" target="_blank">org.apache.sanselan.ImageFormat</a></em> Class has three members name, extension and actual. Name and extension are same, however I am not sure I understand the use of actual variable.</p>
<p>The class has list of formats (including Unknown) supported by the library. They are all instances of ImageFormat class. The list is</p>
<ol>
<li>IMAGE_FORMAT_UNKNOWN</li>
<li>IMAGE_FORMAT_PNG</li>
<li>IMAGE_FORMAT_GIF</li>
<li>IMAGE_FORMAT_ICO</li>
<li>IMAGE_FORMAT_TIFF</li>
<li>IMAGE_FORMAT_JPEG</li>
<li>IMAGE_FORMAT_BMP</li>
<li>IMAGE_FORMAT_PSD</li>
<li>IMAGE_FORMAT_PBM</li>
<li>IMAGE_FORMAT_PGM</li>
<li>IMAGE_FORMAT_PPM</li>
<li>IMAGE_FORMAT_PNM</li>
<li>IMAGE_FORMAT_TGA</li>
<li>IMAGE_FORMAT_JBIG2</li>
</ol>
<h4><span style="text-decoration: underline;">guessFormat API – Under the hood</span></h4>
<p>The guessFormat() API looks at the initial 2 to 4 bytes, also known as magic numbers, to determine the Image Format.</p>
<p>Algorithm is as follows:</p>
<ul>
<li>Read first two bytes</li>
<li>Match with existing set of magic numbers and determine format</li>
<li>It use byte 3 and 4 to determine format for JBig2</li>
</ul>
<p>The list of magic numbers for different formats is as follows</p>
<table border="1" width="300">
<tbody>
<tr>
<th width="171" scope="col">Format</th>
<th width="52" scope="col">Byte 1</th>
<th width="55" scope="col">Byte 2</th>
</tr>
<tr>
<td>IMAGE_FORMAT_GIF</td>
<td>0x47</td>
<td>0x49</td>
</tr>
<tr>
<td>IMAGE_FORMAT_PNG</td>
<td>0x89</td>
<td>0x50</td>
</tr>
<tr>
<td>IMAGE_FORMAT_JPEG</td>
<td>0xff</td>
<td>0xd8</td>
</tr>
<tr>
<td>IMAGE_FORMAT_BMP</td>
<td>0x42</td>
<td>0x4d</td>
</tr>
<tr>
<td>IMAGE_FORMAT_TIFF</p>
<p>(Motorola byte order)</td>
<td>0x4D</td>
<td>0x4D</td>
</tr>
<tr>
<td>IMAGE_FORMAT_TIFF</p>
<p>(Intel byte order)</td>
<td>0x49</td>
<td>0x49</td>
</tr>
<tr>
<td>IMAGE_FORMAT_PSD</td>
<td>0x38</td>
<td>0x42</td>
</tr>
<tr>
<td>IMAGE_FORMAT_PBM</td>
<td>0x50</td>
<td>0x31 or 0x34</td>
</tr>
<tr>
<td>IMAGE_FORMAT_PGM</td>
<td>0x50</td>
<td>0x32 or 0x35</td>
</tr>
<tr>
<td>IMAGE_FORMAT_PPM</td>
<td>0x50</td>
<td>0x33 or 0x36</td>
</tr>
<tr>
<td>IMAGE_FORMAT_JBIG2</td>
<td>0x97</td>
<td>0x4A</td>
</tr>
</tbody>
</table>
<p>In addition to this IMAGE_FORMAT_JBIG2 format, byte 3 must be equal to 0x42 and byte 4 must be equal to 0x32.</p>
<p>Based on this table, Sanselan recognizes the Image format. If the magic numbers don’t match from the one in the table, it returns IMAGE_FORMAT_UNKNOWN</p>
<p><strong><span style="text-decoration: underline;">References</span></strong></p>
<p><strong><a href="http://svn.apache.org/viewvc/commons/proper/sanselan/trunk/src/main/java/org/apache/sanselan/Sanselan.java?view=log" target="_blank">Sanselan.java</a></strong></p>
<a href='#' id="mfg_link_1" class="mfg"></a>
<script type="text/javascript">/* <![CDATA[ */
mfg_create_link( "mfg_link_1", "Apache Sanselan decode image format", "More from Google on <em>", "<\/em> &raquo;");
/* ]]&gt; */</script>]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-demystifying-how-sanselan-determines-image-format/feed/</wfw:commentRss>
		</item>
		<item>
		<title>[Apache Sanselan] Retrieving Image Metadata using Sanselan</title>
		<link>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-retrieving-image-metadata-using-sanselan/</link>
		<comments>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-retrieving-image-metadata-using-sanselan/#comments</comments>
		<pubDate>Fri, 17 Jul 2009 13:30:00 +0000</pubDate>
		<dc:creator>ashish</dc:creator>
		
		<category><![CDATA[Apache Sanselan]]></category>

		<guid isPermaLink="false">http://www.ashishpaliwal.com/blog/?p=287</guid>
		<description><![CDATA[Apache Sanselan is a pure Java library for reading and Writing Image formats. It has recently graduated from Incubator and is now a proud member of Apache commons proper.
In this article we shall see how to get Image metadata using Apache Sanselan.
Let’s see how the code looks
public static void main&#40;String&#91;&#93; args&#41; &#123;
    [...]]]></description>
			<content:encoded><![CDATA[<p>Apache Sanselan is a pure Java library for reading and Writing Image formats. It has recently graduated from Incubator and is now a proud member of Apache commons proper.</p>
<p>In this article we shall see how to get Image metadata using Apache Sanselan.</p>
<p>Let’s see how the code looks</p>
<pre class="java5"><span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000000; font-weight: bold;">static</span> <span style="color: #993333;">void</span> main<span style="color: #66cc66;">&#40;</span><a href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html"><span style="color: #aaaadd; font-weight: bold;">String</span></a><span style="color: #66cc66;">&#91;</span><span style="color: #66cc66;">&#93;</span> args<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span>
    <a href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html"><span style="color: #aaaadd; font-weight: bold;">String</span></a> imageFile = args<span style="color: #66cc66;">&#91;</span><span style="color: #cc66cc;">0</span><span style="color: #66cc66;">&#93;</span>;
    <span style="color: #000000; font-weight: bold;">try</span> <span style="color: #66cc66;">&#123;</span>
         <span style="color: #808080; font-style: italic;">// Metadata details</span>
         IImageMetadata meta = Sanselan.<span style="color: #006600;">getMetadata</span><span style="color: #66cc66;">&#40;</span><span style="color: #000000; font-weight: bold;">new</span> <a href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html"><span style="color: #aaaadd; font-weight: bold;">File</span></a><span style="color: #66cc66;">&#40;</span>imageFile<span style="color: #66cc66;">&#41;</span><span style="color: #66cc66;">&#41;</span>;
&nbsp;
         <a href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/System.html"><span style="color: #aaaadd; font-weight: bold;">System</span></a>.<span style="color: #006600;">out</span>.<span style="color: #006600;">println</span><span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">&quot;--- Image Metadata ----&quot;</span><span style="color: #66cc66;">&#41;</span>;
         <a href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/System.html"><span style="color: #aaaadd; font-weight: bold;">System</span></a>.<span style="color: #006600;">out</span>.<span style="color: #006600;">println</span><span style="color: #66cc66;">&#40;</span>meta<span style="color: #66cc66;">&#41;</span>;
     <span style="color: #66cc66;">&#125;</span> <span style="color: #000000; font-weight: bold;">catch</span> <span style="color: #66cc66;">&#40;</span>ImageReadException e<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span>
         e.<span style="color: #006600;">printStackTrace</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span>;
     <span style="color: #66cc66;">&#125;</span> <span style="color: #000000; font-weight: bold;">catch</span> <span style="color: #66cc66;">&#40;</span><a href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html"><span style="color: #aaaadd; font-weight: bold;">IOException</span></a> e<span style="color: #66cc66;">&#41;</span> <span style="color: #66cc66;">&#123;</span>
         e.<span style="color: #006600;">printStackTrace</span><span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#41;</span>;
     <span style="color: #66cc66;">&#125;</span>
<span style="color: #66cc66;">&#125;</span></pre>
<p>Most of the work is done in single line ?</p>
<p>getMetadata() API returns the IImageMetadata  details. The toString() method of actual instance class of IImageMetadata  prints the image information in a formatted fashion. The instance depends upon the format of the image.</p>
<p>Running the program on one the images shipped with Sanselan source produced this output</p>
<p>--- Image Metadata ----<br />
Exif metadata:<br />
Root:<br />
Image Description: '’'<br />
Make: 'Oregon Scientific'<br />
Model: 'DS6639'<br />
Orientation: 1<br />
XResolution: 72<br />
YResolution: 72<br />
Resolution Unit: 2<br />
Software: 'Adobe Photoshop 7.0'<br />
Modify Date: '2008:07:13 21:05:34'<br />
YCbCr Positioning: 2<br />
Exif Offset: 228</p>
<p>Exif:<br />
Exposure Time: 10/10266 (0.001)<br />
FNumber: 28/10 (2.8)<br />
Exposure Program: 2<br />
ISO: 55<br />
Exif Version: 48, 50, 50, 48<br />
Date Time Original: '2003:10:20 09:01:29'<br />
Create Date: '2003:10:20 09:01:29'<br />
Components Configuration: 1, 2, 3, 0<br />
Compressed Bits Per Pixel: 2<br />
Brightness Value: 9<br />
Exposure Compensation: 0<br />
Max Aperture Value: 3<br />
Metering Mode: 1<br />
Light Source: 0<br />
Flash: 0<br />
Focal Length: 864/100 (8.64)<br />
Subject Location: 27556, 57533, 60147, 20319<br />
Flashpix Version: 48, 49, 48, 48<br />
Color Space: 1<br />
Exif Image Width: 300<br />
Exif Image Length: 225<br />
File Source: 3<br />
Scene Type: 1<br />
Custom Rendered: 0<br />
Exposure Mode: 0<br />
White Balance: 1<br />
Digital Zoom Ratio: 1<br />
Focal Length In 3 5mm Format: 42<br />
Scene Capture Type: 0<br />
Gain Control: 0<br />
Contrast: 0<br />
Saturation: 0<br />
Sharpness: 0</p>
<p>Sub:  (jpegImageData)<br />
Compression: 6<br />
XResolution: 72<br />
YResolution: 72<br />
Resolution Unit: 2<br />
Jpg From Raw Start: 838<br />
Jpg From Raw Length: 6258</p>
<p>Photoshop (IPTC) metadata:<br />
Caption/Abstract: ?</p>
<a href='#' id="mfg_link_3" class="mfg"></a>
<script type="text/javascript">/* <![CDATA[ */
mfg_create_link( "mfg_link_3", "Apache Sanselan Image Metadata retrieval", "More from Google on <em>", "<\/em> &raquo;");
/* ]]&gt; */</script>]]></content:encoded>
			<wfw:commentRss>http://www.ashishpaliwal.com/blog/2009/07/apache-sanselan-retrieving-image-metadata-using-sanselan/feed/</wfw:commentRss>
		</item>
	</channel>
</rss>

<!-- Dynamic page generated in 1.768 seconds. -->
<!-- Cached page generated by WP-Super-Cache on 2010-03-10 23:43:27 -->
