Playing with JClouds transient Blobstore

JClouds BlobStore API provides a portable way of managing key-value providers like Amazon S3. This post is a getting started guide with the API. We shall explore a bit about the API and create a simple program to use the same.

Before we begin, lets get hold of some concepts
Service - It's refers to the provider where we host our key-value data, like Amazon S3

Containers - They are namespace for the data, where we store our Blobs. For example, in Amazon S3, Containers == buckets

Blob - This is the unstructured data that we store inside Containers

There are some more details about the API, but they are not covered here to keep things simple.

Lets briefly see the steps that we need use the API

  • We create a BlobStoreContext. In simple words, it's our handle to Service provider
  • We get BlobStore handle from BlobStoreContext
  • We create a Container
  • We add our data/Blobs to the Container

Lets look at the code in these steps.

The best way to play with JClouds API is clone the examples git-hub repo (https://github.com/jclouds/jclouds-examples) and modify the code to fit the needs, this is what I have done

NOTE: We shall be using JClouds in-memory blobstore for our playing around so that we don't have to pay for the usage charges of BlobStore providers

Step 1: Creating the BlobStoreContext

 String provider = "transient";
 String identity = "Unused";
 String credential = Optional.absent().toString();

 // Init
 BlobStoreContext context = ContextBuilder.newBuilder(provider)
                .credentials(identity, credential)
                .build(BlobStoreContext.class);

The code is simple enough. The transient provider represents the in-memory BlobStore provider, and the other options are specific to this transient provider. If we need to use S3, we replace these options.

Step 2: Get the handle to BlobStore

BlobStore blobStore = context.getBlobStore();

From the BobStoreContext, we get the handle to BlobStore, which we shall use to perform subsequent operations.

Step 3: Creating a Container

String containerName = "dummybase";
blobStore.createContainerInLocation(null, containerName);

This shall create a Container with name "dummybase"

Step 4: Adding Blob to Container

// Add Blob
Blob blob = blobStore.blobBuilder("test").payload("testdata").build();
blobStore.putBlob(containerName, blob);

// Add Blob
Blob blob2 = blobStore.blobBuilder("test1").payload("testdata1").build();
blobStore.putBlob(containerName, blob2);

We create two blobs, with String payload and add to the Container. We can use different API's to add payload, like using a File or an InputStream

That's it, we have added it the Blob to transient BlobStore provider

Let's list the contents

for (StorageMetadata resourceMd : blobStore.list()) {
    if (resourceMd.getType() == StorageType.CONTAINER || resourceMd.getType() == StorageType.FOLDER) {
        // Use Map API
        Map<String, InputStream> containerMap = context.createInputStreamMap(resourceMd.getName());
        System.out.printf("  %s: %s entries%n", resourceMd.getName(), containerMap.size());
     }
}

The complete code together

public static void main(String[] args) throws IOException {
    String provider = "transient";
    String identity = "Unused";
    String credential = Optional.absent().toString();
    String containerName = "dummybase";

    // Init
    BlobStoreContext context = ContextBuilder.newBuilder(provider)
                .credentials(identity, credential)
                .build(BlobStoreContext.class);

    try {
        // Create Container
        BlobStore blobStore = context.getBlobStore();
        blobStore.createContainerInLocation(null, containerName);

        // Add Blob
        Blob blob = blobStore.blobBuilder("test").payload("testdata").build();
        blobStore.putBlob(containerName, blob);

        // Add Blob
        Blob blob2 = blobStore.blobBuilder("test1").payload("testdata1").build();
        blobStore.putBlob(containerName, blob2);

        // List Container
        for (StorageMetadata resourceMd : blobStore.list()) {
            if (resourceMd.getType() == StorageType.CONTAINER || resourceMd.getType() == StorageType.FOLDER) {
                // Use Map API
                Map<String, InputStream> containerMap = context.createInputStreamMap(resourceMd.getName());
                System.out.printf("  %s: %s entries%n", resourceMd.getName(), containerMap.size());
             }
         }
   } finally {
    // Close connecton
    context.close();
    System.exit(0);
  }
}

6 thoughts on “Playing with JClouds transient Blobstore

  1. Hello Ashish
    Just I nedd a small information regarding this jclouds transient feature here when we try to execute the example which you have given the following error is generated
    Bound mismatch: The generic method build(Class) of type ContextBuilder is not applicable for the arguments (Class). The inferred type BlobStoreContext is not a valid substitute for the bounded parameter

  2. Hi

    I have One doubt in the above post ,i,e
    BlobStoreContext context = ContextBuilder.newBuilder(provider)
    .credentials(identity, credential)
    .build(BlobStoreContext.class);
    Iam getting error in build method i,e bound mismatch error let me know what iam missing i have used the following dependencies

    JcloudsCore
    JcloudsAllBlobStore
    JcloudsCompedium
    JcloudsAllCompute

    Thanks in advance

  3. Hi,
    the solution for the above mentioned problem is simply to replace:
    BlobStoreContext context = ContextBuilder.newBuilder(provider).credentials(identity, credential).build(BlobStoreContext.class);

    by

    BlobStoreContext context = new BlobStoreContextFactory().createContext(provider, identity, credential);

    Cheers,
    M

  4. I got this error at the line
    mContext = ContextBuilder.newBuilder(“filesystem”).overrides(properties).credentials(“Unused”, Optional.absent().toString()).buildView(BlobStoreContext.class);

    14:11:23 ERROR main step.AbstractStep – Encountered an error executing the step
    java.lang.NoClassDefFoundError: com/google/gson/TypeAdapter
    at org.jclouds.rest.config.RestModule.configure(RestModule.java:62)
    at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
    at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
    at com.google.inject.spi.Elements.getElements(Elements.java:101)
    at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133)
    at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
    at com.google.inject.Guice.createInjector(Guice.java:95)
    at org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:403)
    at org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:327)
    at org.jclouds.ContextBuilder.buildView(ContextBuilder.java:618)
    at org.jclouds.ContextBuilder.buildView(ContextBuilder.java:598)
    at org.jclouds.ContextBuilder.build(ContextBuilder.java:591)

  5. Hi Ashish,

    I am working on integrating Jclouds with Amazon-SNS service for pushing the sample notifications to the endpoint URL.

    My question is it possible to integrate with Jclouds with Amazon-SNS service ?

    Please reply me on this and guide me how if the integration is affirmative.

    Thanks & Regards,
    amarender

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.