In the last we saw the Line Count example, now lets add filtering to the example, to filter out empty lines. The code can be found here LineCountWithFiltering.java Lets look at the code public class LineCountWithFiltering { public static void main(String[] args) { SparkConf sparkConf = new SparkConf().setAppName("File Copy"); JavaSparkContext sparkContext = new JavaSparkContext(sparkConf); // […]
Apache Spark examples
[Learning Spark with Examples] Line Count
In the First post we looked at how to load/save an RDD. In this post we shall build upon the example and count number of lines present in RDD. The code can be found at LineCount.java For complete project refer https://github.com/paliwalashish/learning-spark Lets look at the code public static void main(String[] args) { SparkConf sparkConf = […]