Package org.apache.accumulo.core.spi.balancer

Class HostRegexTableLoadBalancer

java.lang.Object
org.apache.accumulo.core.spi.balancer.TableLoadBalancer
org.apache.accumulo.core.spi.balancer.HostRegexTableLoadBalancer
All Implemented Interfaces:
TabletBalancer
public class HostRegexTableLoadBalancer extends TableLoadBalancer
This balancer creates groups of tablet servers using user-provided regular expressions over the tablet server hostnames. Then it delegates to the table balancer to balance the tablets within the resulting group of tablet servers. All tablet servers that do not match a regex are grouped into a default group.
Regex properties for this balancer are specified as:
table.custom.balancer.host.regex.<tablename>=<regex>
Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration. If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the manager is restarted while the tablet servers are up. To change the out of bounds check time period, set the following property:
table.custom.balancer.host.regex.oob.period=5m
Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server IP address, then set the following property:
table.custom.balancer.host.regex.is.ip=true
It's possible that this balancer may create a lot of migrations. To limit the number of migrations that are created during a balance call, set the following property (default 250):
table.custom.balancer.host.regex.concurrent.migrations This balancer can continue balancing even if there are outstanding migrations. To limit the number of outstanding migrations in which this balancer will continue balancing, set the following property (default 0):
table.custom.balancer.host.regex.max.outstanding.migrations
Since:
2.1.0

  • Field Details

    • HOST_BALANCER_PREFIX

      public static final String HOST_BALANCER_PREFIX

    • HOST_BALANCER_OOB_CHECK_KEY

      public static final String HOST_BALANCER_OOB_CHECK_KEY

    • HOST_BALANCER_REGEX_USING_IPS_KEY

      public static final String HOST_BALANCER_REGEX_USING_IPS_KEY

    • HOST_BALANCER_REGEX_MAX_MIGRATIONS_KEY

      public static final String HOST_BALANCER_REGEX_MAX_MIGRATIONS_KEY

    • DEFAULT_POOL

      protected static final String DEFAULT_POOL
      See Also:

    • HOST_BALANCER_OUTSTANDING_MIGRATIONS_KEY

      public static final String HOST_BALANCER_OUTSTANDING_MIGRATIONS_KEY

    • lastOOBCheckTimes

      protected final Map<String,Long> lastOOBCheckTimes

  • Constructor Details

    • HostRegexTableLoadBalancer

      public HostRegexTableLoadBalancer()

  • Method Details

    • splitCurrentByRegex

      protected Map<String,SortedMap<TabletServerId,TServerStatus>> splitCurrentByRegex(SortedMap<TabletServerId,TServerStatus> current)
      Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool. This could be expensive in the terms of the amount of time to recompute the groups, so HOST_BALANCER_POOL_RECHECK_KEY should be specified in the terms of minutes, not seconds or less.
      Parameters:
      current - map of current tservers
      Returns:
      current servers grouped by pool name, if not a match it is put into a default pool.

    • getPoolNamesForHost

      protected List<String> getPoolNamesForHost(TabletServerId tabletServerId)
      Matches host against the regexes and returns the matching pool names
      Parameters:
      tabletServerId - tablet server host
      Returns:
      pool names, will return default pool if host matches more no regex

    • getNameFromIp

      protected String getNameFromIp(String hostIp) throws UnknownHostException
      Throws:
      UnknownHostException

    • getPoolNameForTable

      protected String getPoolNameForTable(String tableName)
      Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
      Parameters:
      tableName - name of table
      Returns:
      tablet server pool name (table name or DEFAULT_POOL)

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getPoolNameToRegexPattern

      public Map<String,Pattern> getPoolNameToRegexPattern()

    • getMaxMigrations

      public int getMaxMigrations()

    • getMaxOutstandingMigrations

      public int getMaxOutstandingMigrations()

    • getOobCheckMillis

      public long getOobCheckMillis()

    • isIpBasedRegex

      public boolean isIpBasedRegex()

    • init

      public void init(BalancerEnvironment balancerEnvironment)
      Description copied from interface: TabletBalancer
      Initialize the TabletBalancer. This gives the balancer the opportunity to read the configuration.
      Specified by:
      init in interface TabletBalancer
      Overrides:
      init in class TableLoadBalancer

    • getAssignments

      public void getAssignments(TabletBalancer.AssignmentParameters params)
      Description copied from interface: TabletBalancer
      Assign tablets to tablet servers. This method is called whenever the manager finds tablets that are unassigned.
      Specified by:
      getAssignments in interface TabletBalancer
      Overrides:
      getAssignments in class TableLoadBalancer

    • balance

      public long balance(TabletBalancer.BalanceParameters params)
      Description copied from interface: TabletBalancer
      Ask the balancer if any migrations are necessary. If the balancer is going to self-abort due to some environmental constraint (e.g. it requires some minimum number of tservers, or a maximum number of outstanding migrations), it should issue a log message to alert operators. The message should be at WARN normally and at ERROR if the balancer knows that the problem can not self correct. It should not issue these messages more than once a minute. This method will not be called when there are unassigned tablets.
      Specified by:
      balance in interface TabletBalancer
      Overrides:
      balance in class TableLoadBalancer
      Returns:
      the time, in milliseconds, to wait before re-balancing.

    • getOnlineTabletsForTable

      protected List<TabletStatistics> getOnlineTabletsForTable(TabletServerId tabletServerId, TableId tableId) throws AccumuloSecurityException, AccumuloException
      Throws:
      AccumuloSecurityException
      AccumuloException