Lets look at the Channels in this post. Channels are the link between Source and Sink. We shall briefly look at Channel Lifecycle and how the message is passed from Source to Channel.
Channel’s lifecycle is same as Source life cycle, as depicted in figure above.
Source to Channel message flow
Let’s look at how message is passed from a Source to Channel
Flume provide reliable message delivery and this brings transactions into picture. For simplicity of discussion would take Transaction later. As a simple rule, once the next stage Acks receipt of message, then only current stage would remove the message from it’s store. This ensures that messages are not lost.
Now let’s look at main flow.
- A message is received at Source
- A Source can be associated with one or more mandatory or optional channels
- Source has a ChannelProcessor associated with it, Event is passed to it
- Channel processor process the Interceptor’s configured
- Channel processor uses the ChannelSelector to determine the Channels the Event needs to be sent to
- The Event is pushed to all the Channels within a Transaction
That’s a brief about Channels. We shall look at implementing Custom Channel next.