[Flume Cookbook] Anatomy of Flume Event

Event is basic unit of data processing or data transfer in Flume. All the user level data is carried as payload in Flume. Here is a top level view of a Flume Event.

Flume Event

Flume Event

A Flume Events is composed of two part

  • Optional Headers
  • Event Body or payload as byte[]

Lets look at the Event code

public interface Event {
  public Map<String, String> getHeaders();

  public void setHeaders(Map<String, String> headers);

  public byte[] getBody();

  public void setBody(byte[] body);
}

Apache Flume ships with two implementations of Event interface, both ships as part of flume-ng-sdk module

  1. SimpleEvent
  2. JSONEvent

Creating Events

We shall be working a lot with events, lets see various ways of creating Events.

Using Event Implementation classes

We can simply create Event instances using the Constructors, and can add headers and payload to it.

Event event1 = new SimpleEvent();
Map<String, String> headers = new HashMap<String, String>();
headers.put("eventType", "Log");
headers.put("eventCategory", "dummy");
event1.setHeaders(headers);
event1.setBody("Dummy Event Body".getBytes());

Using EventBuilder class

Flume ships with an EventBuilder class that helps in creating events. Following are the ways we can create Events using EventBuilder. Following are API's available for Event creation

  public static Event withBody(byte[] body, Map<String, String> headers)
  public static Event withBody(byte[] body) {}
  public static Event withBody(String body, Charset charset,
      Map<String, String> headers)
  public static Event withBody(String body, Charset charset)

NOTE: EventBuilder class creates instances of SimpleEvent only.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.