Package org.apache.accumulo.hadoop.mapreduce

Class AccumuloFileOutputFormat

java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Key,Value>
org.apache.accumulo.hadoop.mapreduce.AccumuloFileOutputFormat
public class AccumuloFileOutputFormat extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Key,Value>
This class allows MapReduce jobs to write output in the Accumulo data file format.
Care should be taken to write only sorted data (sorted by Key), as this is an important requirement of Accumulo data files. The output path to be created must be specified via configure(), which uses a fluent API. For Example: 
 AccumuloFileOutputFormat.configure()
      .outputPath(path)
      .fileBlockSize(b)
      .compression(type)
      .summarizers(sc1, sc2).store(job));
For all available options see FileOutputFormatBuilder. Methods inherited from FileOutputFormat are not supported and may be ignored or cause failures. Using other Hadoop configuration options that affect the behavior of the underlying files directly in the Job's configuration may work, but are not directly supported at this time.
Since:
2.0

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

    org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.Counter

  • Field Summary

    Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

    BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART

  • Constructor Summary

    Constructors
    Constructor
    Description
    AccumuloFileOutputFormat()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static FileOutputFormatBuilder.PathParams<org.apache.hadoop.mapreduce.Job>
    configure()
    Sets all the information required for this map reduce job.
    org.apache.hadoop.mapreduce.RecordWriter<Key,Value>
    getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
     

    Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

    checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • AccumuloFileOutputFormat

      public AccumuloFileOutputFormat()

  • Method Details

    • getRecordWriter

      public org.apache.hadoop.mapreduce.RecordWriter<Key,Value> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException
      Specified by:
      getRecordWriter in class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Key,Value>
      Throws:
      IOException

    • configure

      public static FileOutputFormatBuilder.PathParams<org.apache.hadoop.mapreduce.Job> configure()
      Sets all the information required for this map reduce job.