Getting Started with Terracotta Toolkit – Part 3 (Clustered Map)

Did you ever wanted to have a Map which can be clustered across JVM, without doing much work for replicating data, ensuring coherence of data? Terracotta Toolkit allows you to fullfill your dream, Use the same Map API's you have been using for long and see your data being clustered transparently with Terracotta. The fun begins, when you add new clients and they have the data with them without any additional programming.

Lets start with some general information about Toolkit. Refer Toolkit javadoc for more details here

Pre-requisite

  • Download Terracotta 3.3.0 from Download page
  • Include following terracotta-toolkit-1.0-runtime-1.0.0.jar into your classpath for using Toolkit. The jar is present inside Terracotta_Install_dir/common folder.
  • You can spend some time reading the following posts on Toolkit, although they can be read in any order

    Lets explore a bit what do we achieve when we say Clustered Map

    Unclustered Map

    Unclustered Map

    As you can see in the figure above, we have different JVM's, each having a Map instance used by Application. Now if we were to share the same instance across app's running in different nodes, we would have to write a lot of infrastructure code to replicate data, keep data coherent etc.

    Clustered Map

    Clustered Map

    The figure above depicts a Clustered Map. The Map instance is visible to App's across JVM's as if they are accessing a local instance. You can achieve this in a few lines, without worrying about the infrastructure code. Lets see how we can achieve this in few simple steps.

    Steps to creare Clustered Map

  • Initialize Toolkit
  • Get a Map reference
  • Use the Map
  • Initializing the Toolkit

    Before any operations can be performed using Toolkit, it needs to be initialized first. The initialization code is very simple, just single line

    ClusteringToolkit clustering = new TerracottaClient("localhost:9510").getToolkit();
    

    This line initializes the Terracotta Client and gets an instance of Toolkit to work with. The argument passed to the TerracottaClient is the IP Address and Port of the Terracotta Server.

    Have created a simple function for initialization of the Toolkit.

    ClusteringToolkit clusterToolkit = null;
    
    public void initializeToolKit(String serverAdd) {
    clusterToolkit = new TerracottaClient(serverAdd).getToolkit();
    }
    

    Get/Create the Map

    public void initializeMap() {
        clusteredMap = clusterToolkit.getMap(MAP_NAME);    
    }
    

    The code above creates a Clustered Map. There is no more infrastructure code that you have to write. This gives us a Clustered Map reference. The best part is, there are no new API's to be learned, we continue to use the same old Map API's.

    Let's see the data class that we would share across the Map.

    public class SharedData implements Serializable {
    
        private String data;
    
        public SharedData(String data) {
            this.data = data;
        }
    
        public String getData() {
            return data;
        }
    
        public void setData(String data) {
            this.data = data;
        }
    }
    

    Its a no-brainer class. Just a simple class for demonstration purpose.

    Just to keep life simple, we are going to create a function which populates the Map, and the Nodes can then consume the populated data. Consume here would mean just call a get() and print. In real life scenario, you can mutate the data, and the updated data shall be visible across the cluster.

    Lets see the populate function

    protected void populateMap(int objectCount) {
            for(int i = 0; i < objectCount; i++) {
                SharedData data = new SharedData(""+i);
                clusteredMap.put(i, data);
            }
        }
    

    Its a simple function which just puts the data in the Map.

    Lets look at the consume function

     protected void consumeMapData(int maxObjectId) {
            Range range = new Range(1, maxObjectId);
            while(true) {
                long key = range.getRandomKey();
                byte[] dataBytes = clusteredMap.get((int)key);
                System.out.println("Key = "+key);
                if(dataBytes == null) {
                    System.out.println("OOPS!.. somethings wrong dude...");
                    return;
                }
                SharedData data = (SharedData)deserializeObject(dataBytes);
                if(data != null) {
                    System.out.println("Data = "+data.getData());
                }
                try {
                    // Sleep for 5 secs
                    Thread.sleep(5000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    

    The function just takes a random key and call a get on the map.

    To run this implementation, we would have 3 Nodes running, 1 producer and 2 consumer. The picture below gives an idea of the topology.

    Runtime Topology

    Runtime Topology

    To run the implementation

  • Start Terracotta Server
  • Start Producer
  • Start one or more Consumer
  • You can get the complete source of the sample here

    11 thoughts on “Getting Started with Terracotta Toolkit – Part 3 (Clustered Map)

    1. Hi Ashish,
      Found yourblog on loading reference data into Ehcache very interesting. I am also looking for some pointers on implementing the Write-behind features.

      Regards,
      Ravi

      • @javin, Just want to make a note that ClusteredMap works across JVM, against CHM which is good in single JVM only. However, Clustered Map is very efficient at what is does.

    2. Hi Ashish,

      I can get the code to work properly, except for the very big problem that when I have 2 clients ( 2 separate jvms ) changes to the clustered map on one client or NOT visible to the other. It appears that they each jvm has it’s own copy, though they are calling it by the same name.
      Interestingly enough, the type of map returned from the getMap(name) call is a ConcurrentDistributedMap rather than a ConcurrentDistributedServerMap, which is what I expected.
      I am using an out of box terracotta 3.4.0 server.

      • Use latest 3.5.0 release. If the Map name and the client configurations are same, this shouldn’t be the case.
        It would be worth to start dev-console (tcinstall/bin)and see whats going inside the Map 🙂

    3. Hi Ashish,

      Your blob is just great!, I find it really interesting and useful.

      Just a quick question in terms of clustering a map, what is actually the difference between cluster a map using Terracotta toolkit or using terracotta POJO (configuring tc-xml, locking and all those things)?

      Thanks a lot.

      Regards,

      Daniel

      • Using toolkit you don’t need instrumentation, which is the case with the later technique, so its easy to use and deploy, and performs better 🙂

        HTH !

        • Hi Ashish,

          Thank very much for your reply.

          I have dug a little bit more on ClusteredMap,I am trying to declare and use a ClusteredMap as follows:

          private ClusteredMap myMap.

          If I use Integer and Integer it works fine but if I use my own class, terracotta doesn’t understand it.

          I presume I need to use instrumentation, dont I?, do you know an easy way to do it?.

          I can not use DSO as I have not been told DSO is not compatible with quartz express and ehcache express.

          thank you very much.

          Regards

    4. Hi,

      I have a problem using the toolkit which I have detailed at http://forums.terracotta.org/forums/posts/list/5680.page#28104.

      in short:
      I have a map example running fine when server and client are on localhost

      when I tried to run the client on a different machine, toolkit connection gets established fine, i can see it in the dev console too.
      when i try to put any thing in the map, i get log line saying
      WARN – We couldn’t load configuration data from the server at ‘localhost:9510’; retrying. (Error: Connection refused.)
      and the operation fails.

    Leave a Reply to Daniel Cancel reply

    Your email address will not be published. Required fields are marked *

    This site uses Akismet to reduce spam. Learn how your comment data is processed.