[Flume Cookbook] Flume Sources

Flume Sources are implementations through which Events enter Flume Agent.

Flow

Flume Flow

As we can see in the picture, a Source is an entry point in the Flume Agent. Agents can be chained to form Flume topologies.

All Sources implement Source interface


public interface Source extends LifecycleAware, NamedComponent {
  /**
   * Specifies which channel processor will handle this source's events.
   *
   * @param channelProcessor
   */
  public void setChannelProcessor(ChannelProcessor channelProcessor);

  /**
   * Returns the channel processor that will handle this source's events.
   */
  public ChannelProcessor getChannelProcessor();
}

All Flume Sources are Named components i.e. they can be uniquely identified by name.

All Sources are alos LifeCycleAware, meaning they have a defined Life Cycle and can report their current Life Cycle state.

Life Cycle of Source

Life Cycle of Source

When Sources are created, they are in Idle State. Once the start phase completes, they move to Started state. If an error occurs during initialization, they move to Error state.

Flume comes bundled with lot of Source implementation. They are covered in detail in Flume User Guide

Next we shall look deeper into Sources and implement a Custom Source.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.