Strategies for Dealing with Out of Memory Errors

Jobs can sometimes fail due to their tasks running out of heap memory. Based on where these out of memory errors occur, we suggest few configuration tunings to deal with them.

Out of Memory Errors on Mapper Side

If out of memory errors are occurring on the mapper side, then reduce the value of following configuration:

#The maximum number of streams to merge at once when sorting files.
io.sort.factor (default: 10)

Out of Memory Errors on Reducer Side

If out of memory errors are occurring on the reducer side, then check if errors are thrown in:

• Copy/Shuffle phase

  In this case, reduce the value of following two configurations:

  #The number of threads used to copy map outputs to the reducer.
  mapred.reduce.parallel.copies (default: 5)
  

  #The proportion of total heap size to be allocated to the map outputs buffer
  #during the copy phase of the shuffle.
  mapred.job.shuffle.input.buffer.percent (default: 70% of the total heap size)
  

• Merge phase

  In this case, reduce the value of following configuration:

  #The maximum number of streams to merge at once when sorting files.
  io.sort.factor (default: 10)