Getting Started with Terracotta Toolkit – Part 3 (Clustered Map)
Did you ever wanted to have a Map which can be clustered across JVM, without doing much work for replicating data, ensuring coherence of data? Terracotta Toolkit allows you to fullfill your dream, Use the same Map API's you have been using for long and see your data being clustered transparently with Terracotta. The fun begins, when you add new clients and they have the data with them without any additional programming.
Lets start with some general information about Toolkit. Refer Toolkit javadoc for more details here
Pre-requisite
You can spend some time reading the following posts on Toolkit, although they can be read in any order
- Getting Started with Terracotta Toolkit – Part 1
- Getting Started with Terracotta Toolkit – Part 2 (Cluster Events)
Lets explore a bit what do we achieve when we say Clustered Map
Unclustered Map
As you can see in the figure above, we have different JVM's, each having a Map instance used by Application. Now if we were to share the same instance across app's running in different nodes, we would have to write a lot of infrastructure code to replicate data, keep data coherent etc.
Clustered Map
The figure above depicts a Clustered Map. The Map instance is visible to App's across JVM's as if they are accessing a local instance. You can achieve this in a few lines, without worrying about the infrastructure code. Lets see how we can achieve this in few simple steps.
Steps to creare Clustered Map
Initializing the Toolkit
Before any operations can be performed using Toolkit, it needs to be initialized first. The initialization code is very simple, just single line
ClusteringToolkit clustering = new TerracottaClient("localhost:9510").getToolkit();
This line initializes the Terracotta Client and gets an instance of Toolkit to work with. The argument passed to the TerracottaClient is the IP Address and Port of the Terracotta Server.
Have created a simple function for initialization of the Toolkit.
ClusteringToolkit clusterToolkit = null;
public void initializeToolKit(String serverAdd) {
clusterToolkit = new TerracottaClient(serverAdd).getToolkit();
}
Get/Create the Map
public void initializeMap() {
clusteredMap = clusterToolkit.getMap(MAP_NAME);
}
The code above creates a Clustered Map. There is no more infrastructure code that you have to write. This gives us a Clustered Map reference. The best part is, there are no new API's to be learned, we continue to use the same old Map API's.
Let's see the data class that we would share across the Map.
public class SharedData implements Serializable {
private String data;
public SharedData(String data) {
this.data = data;
}
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
}
Its a no-brainer class. Just a simple class for demonstration purpose.
Just to keep life simple, we are going to create a function which populates the Map, and the Nodes can then consume the populated data. Consume here would mean just call a get() and print. In real life scenario, you can mutate the data, and the updated data shall be visible across the cluster.
Lets see the populate function
protected void populateMap(int objectCount) {
for(int i = 0; i < objectCount; i++) {
SharedData data = new SharedData(""+i);
clusteredMap.put(i, data);
}
}
Its a simple function which just puts the data in the Map.
Lets look at the consume function
protected void consumeMapData(int maxObjectId) {
Range range = new Range(1, maxObjectId);
while(true) {
long key = range.getRandomKey();
byte[] dataBytes = clusteredMap.get((int)key);
System.out.println("Key = "+key);
if(dataBytes == null) {
System.out.println("OOPS!.. somethings wrong dude...");
return;
}
SharedData data = (SharedData)deserializeObject(dataBytes);
if(data != null) {
System.out.println("Data = "+data.getData());
}
try {
// Sleep for 5 secs
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
The function just takes a random key and call a get on the map.
To run this implementation, we would have 3 Nodes running, 1 producer and 2 consumer. The picture below gives an idea of the topology.
Runtime Topology
To run the implementation
You can get the complete source of the sample here
Hi Ashish,
Found yourblog on loading reference data into Ehcache very interesting. I am also looking for some pointers on implementing the Write-behind features.
Regards,
Ravi
@ravi, will definitely put some sample on write-behind feature soon.
Hi Ashish,
How about performance of this Map , is it comparable to ConcurrentHashMap ?
@javin, Just want to make a note that ClusteredMap works across JVM, against CHM which is good in single JVM only. However, Clustered Map is very efficient at what is does.
Hi Ashish,
I can get the code to work properly, except for the very big problem that when I have 2 clients ( 2 separate jvms ) changes to the clustered map on one client or NOT visible to the other. It appears that they each jvm has it’s own copy, though they are calling it by the same name.
Interestingly enough, the type of map returned from the getMap(name) call is a ConcurrentDistributedMap rather than a ConcurrentDistributedServerMap, which is what I expected.
I am using an out of box terracotta 3.4.0 server.
Use latest 3.5.0 release. If the Map name and the client configurations are same, this shouldn’t be the case.
It would be worth to start dev-console (tcinstall/bin)and see whats going inside the Map
Hi Ashish,
Your blob is just great!, I find it really interesting and useful.
Just a quick question in terms of clustering a map, what is actually the difference between cluster a map using Terracotta toolkit or using terracotta POJO (configuring tc-xml, locking and all those things)?
Thanks a lot.
Regards,
Daniel
Using toolkit you don’t need instrumentation, which is the case with the later technique, so its easy to use and deploy, and performs better
HTH !
Hi Ashish,
Thank very much for your reply.
I have dug a little bit more on ClusteredMap,I am trying to declare and use a ClusteredMap as follows:
private ClusteredMap myMap.
If I use Integer and Integer it works fine but if I use my own class, terracotta doesn’t understand it.
I presume I need to use instrumentation, dont I?, do you know an easy way to do it?.
I can not use DSO as I have not been told DSO is not compatible with quartz express and ehcache express.
thank you very much.
Regards
Hi,
I have a problem using the toolkit which I have detailed at http://forums.terracotta.org/forums/posts/list/5680.page#28104.
in short:
I have a map example running fine when server and client are on localhost
when I tried to run the client on a different machine, toolkit connection gets established fine, i can see it in the dev console too.
when i try to put any thing in the map, i get log line saying
WARN – We couldn’t load configuration data from the server at ‘localhost:9510′; retrying. (Error: Connection refused.)
and the operation fails.
The configuration parameter on the client needs to point to your server. Change “localhost” to the name of your server machine.