Package org.apache.accumulo.core.spi.compaction

package org.apache.accumulo.core.spi.compaction

This package provides a place for plugin interfaces related to executing compactions. The diagram below shows the functional components in Accumulo related to compactions. Not all of these components are pluggable, but understanding how everything fits together is important for writing a plugin.

The following is a description of each functional component.

• Compaction Manager A non pluggable component within the tablet server that brings all other components together. The manager will route compactables to compaction services. For each kind of compaction, an individual compactable will be routed to a single compaction service. For example its possible that compactable C1 is routed to service S1 for user compactions and service S2 for system compactions.
  • Compaction Service A non pluggable component that compacts tablets. One or more of these are created based on user configuration. Users can assign a table to a compaction service. Has a single compaction planner and one ore more compaction executors.
    • Compaction Executor A non pluggable component that executes compactions using multiple threads and has a priority queue.
    • Compaction Planner A pluggable component that can be configured by users when they configure a compaction service. It makes decisions about which files to compact on which executors. See CompactionPlanner, CompactionPlanner.makePlan(org.apache.accumulo.core.spi.compaction.CompactionPlanner.PlanningParameters), and DefaultCompactionPlanner
  • Compactable A non pluggable component that wraps a Tablet and per table pluggable compaction components. It tracks all information about one or more running compactions that is needed by a compaction service in a thread safe manor. There is a 1 to 1 relationship between compactables and tablets.
    • Compaction Configurer A pluggable component that can optionally be configured per table to dynamically configure file output settings. This supports use cases like using snappy for small files and gzip for large files. See CompactionConfigurer
    • Compaction Dispatcher A pluggable component component that decides which compaction service a table should use for different kinds of compactions. This is configurable by users per table. See CompactionDispatcher
    • Compaction Selector A pluggable component that can optionally be configured per table to periodically select files to compact. This supports use cases like periodically compacting all files because there are too many deletes. See CompactionSelector
    • Compaction Strategy A deprecated pluggable component replaced by the Selector and Configurer. See org.apache.accumulo.core.client.admin.CompactionStrategyConfig's own documentation for more information about why it was deprecated.

See Also:

• org.apache.accumulo.core.spi

Class	Description
CompactionDispatch	The dispatch of a CompactionDispatcher
CompactionDispatch.Builder
CompactionDispatch.ServiceBuilder	Required service parameter for CompactionDispatch
CompactionDispatcher	Can be configured per table to dispatch compactions to different compaction services.
CompactionDispatcher.DispatchParameters	The method parameters for CompactionDispatcher.dispatch(DispatchParameters).
CompactionDispatcher.InitParameters	The method parameters for CompactionDispatcher.init(InitParameters).
CompactionExecutorId	A unique identifier for a a compaction executor that a CompactionPlanner can schedule compactions on using a CompactionJob.
CompactionJob	An immutable object that describes what files to compact and where to compact them.
CompactionKind
CompactionPlan	The return value of CompactionPlanner.makePlan(PlanningParameters) that is created using CompactionPlanner.PlanningParameters.createPlanBuilder()
CompactionPlan.Builder
CompactionPlanner	Plans compaction work for a compaction service.
CompactionPlanner.InitParameters	This interface exists so the API can evolve and additional parameters can be passed to the method in the future.
CompactionPlanner.PlanningParameters	This interface exists so the API can evolve and additional parameters can be passed to the method in the future.
CompactionServiceId	A unique identifier for a compaction service
CompactionServices	Provider of information about configured compaction services.
DefaultCompactionPlanner	Finds the largest continuous set of small files that meet the compaction ratio and do not prevent future compactions.
ExecutorManager	Offered to a Compaction Planner at initialization time so it can create executors.
SimpleCompactionDispatcher	Dispatcher that supports simple configuration for making tables use compaction services.