Package org.apache.accumulo.core.client.rfile

Class RFileWriter

java.lang.Object
org.apache.accumulo.core.client.rfile.RFileWriter
All Implemented Interfaces:
AutoCloseable
public class RFileWriter extends Object implements AutoCloseable
This class provides an API for writing RFiles. It can be used to create file for bulk import into Accumulo using TableOperations.importDirectory(String, String, String, boolean)

A RFileWriter has the following constraints. Violating these constraints will result in runtime exceptions.

  • Keys must be appended in sorted order within a locality group.
  • Locality groups must have a mutually exclusive set of column families.
  • The default locality group must be started last.
  • If using RFile.newWriter().to("filename.rf"), the ".rf" extension is required.

Below is an example of using RFileWriter 

 
     Iterable<Entry<Key, Value>> localityGroup1Data = ...
     Iterable<Entry<Key, Value>> localityGroup2Data = ...
     Iterable<Entry<Key, Value>> defaultGroupData = ...

     try(RFileWriter writer = RFile.newWriter().to("filename.rf").build()) {

       // Start a locality group before appending data.
       writer.startNewLocalityGroup("groupA", "columnFam1", "columnFam2");
       // Append data to the locality group that was started above. Must append in sorted order.
       writer.append(localityGroup1Data);

       // Add another locality group.
       writer.startNewLocalityGroup("groupB", "columnFam3", "columnFam4");
       writer.append(localityGroup2Data);

       // The default locality group must be started last.
       // The column families for the default group do not need to be specified.
       writer.startDefaultLocalityGroup();
       // Data appended here can not contain any
       // column families specified in previous locality groups.
       writer.append(defaultGroupData);

       // This is a try-with-resources so the writer is closed here at the end of the code block.
     }

Create instances by calling RFile.newWriter()

Since:
1.8.0

  • Method Details

    • startNewLocalityGroup

      public void startNewLocalityGroup(String name, List<byte[]> families) throws IOException
      Before appending any data, a locality group must be started. The default locality group must be started last.
      Parameters:
      name - locality group name, used for informational purposes
      families - the column families the locality group can contain
      Throws:
      IllegalStateException - When default locality group already started.
      IOException

    • startNewLocalityGroup

      public void startNewLocalityGroup(String name, byte[]... families) throws IOException
      See javadoc for startNewLocalityGroup(String, List)
      Throws:
      IllegalStateException - When default locality group already started.
      IOException

    • startNewLocalityGroup

      public void startNewLocalityGroup(String name, Set<String> families) throws IOException
      See javadoc for startNewLocalityGroup(String, List).
      Parameters:
      families - will be encoded using UTF-8
      Throws:
      IllegalStateException - When default locality group already started.
      IOException

    • startNewLocalityGroup

      public void startNewLocalityGroup(String name, String... families) throws IOException
      See javadoc for startNewLocalityGroup(String, List).
      Parameters:
      families - will be encoded using UTF-8
      Throws:
      IllegalStateException - When default locality group already started.
      IOException

    • startDefaultLocalityGroup

      public void startDefaultLocalityGroup() throws IOException
      A locality group in which the column families do not need to specified. The locality group must be started after all other locality groups. Can not append column families that were in a previous locality group. If no locality groups were started, then the first append will start the default locality group.
      Throws:
      IllegalStateException - When default locality group already started.
      IOException

    • append

      public void append(Key key, Value val) throws IOException
      Append the key and value to the last locality group that was started. If no locality group was started, then the default group will automatically be started.
      Parameters:
      key - This key must be greater than or equal to the last key appended. For non-default locality groups, the keys column family must be one of the column families specified when calling startNewLocalityGroup(). Must be non-null.
      val - value to append, must be non-null.
      Throws:
      IllegalArgumentException - This is thrown when data is appended out of order OR when the key contains a invalid visibility OR when a column family is not valid for a locality group.
      IOException

    • append

      public void append(Iterable<Map.Entry<Key,Value>> keyValues) throws IOException
      Append the keys and values to the last locality group that was started.
      Parameters:
      keyValues - The keys must be in sorted order. The first key returned by the iterable must be greater than or equal to the last key appended. For non-default locality groups, the keys column family must be one of the column families specified when calling startNewLocalityGroup(). Must be non-null. If no locality group was started, then the default group will automatically be started.
      Throws:
      IllegalArgumentException - This is thrown when data is appended out of order OR when the key contains a invalid visibility OR when a column family is not valid for a locality group.
      IOException

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Throws:
      IOException