[Apache Pig] Dealing with Unexpected character ‘$’ error

Recently while working on an ETL script, encountered following exception when using REPLACE() API to remove $ from column. 2014-04-08 21:24:41,454 [main] ERROR org.apache.pig.PigServer – exception during parsing: Error during parsing. <file test.pig, line 16, column 51> Unexpected character '$' Failed to parse: <file test.pig, line 16, column 51> Unexpected character '$' at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:243) at […]

Continue reading


[Apache Pig] Dealing with java.lang.OutOfMemoryError: Java heap space

Recently while working on an aggregation script, faced following exception java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:756) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) There was no custom UDF, and all the script was using built-in functions. This exception can be dealt by pushing up the Java heap (defaults to 1000m). We can either set […]

Continue reading