Accumulo 4.x Documentation >> Configuration >> Metrics Documentation (4.x)

Metrics Documentation (4.x)

Edit this page

Below are the metrics used to monitor various components of Accumulo. Metrics emitted by Accumulo should contain the folowing tags: ‘instance.name’, ‘resource.group’, ‘process.name’, ‘host’ and ‘port’. Metrics emitted by Accumulo may contain additional tags where we think it makes sense to capture per-object metrics, for example on a table or tablet basis in the ScanServer and TabletServer, or on a per-queue basis in the CompactionCoordinator.

Table of Contents

General Server Metrics

Metrics that are generated across all server types.

accumulo.detected.low.memory

Type: GAUGE
Description: Reports 1 when process memory usage is above the threshold, reports 0 when memory is okay.

accumulo.recoveries.sorts.avg.progress

Type: GAUGE
Description: The average percentage (0.0 - 99.9) of the in progress log sorts.

accumulo.recoveries.sorts.in.progress

Type: GAUGE
Description: The number of log sorts in progress.

accumulo.recoveries.sorts.runtime.longest

Type: GAUGE
Description: The time (in milliseconds) of the longest running log sort.

accumulo.recoveries.tablets.completed

Type: GAUGE
Description: The number of tablet recoveries completed

accumulo.recoveries.tablets.failed

Type: GAUGE
Description: The number of tablet recoveries failed

accumulo.recoveries.tablets.in.progress

Type: GAUGE
Description: The number of tablet recoveries in progress

accumulo.recoveries.tablets.mutations.replayed

Type: GAUGE
Description: The number of mutations replayed for tablet recovery

accumulo.recoveries.tablets.started

Type: GAUGE
Description: The number of tablet recoveries started

accumulo.server.idle

Type: GAUGE
Description: Indicates if the server is idle or not. The value will be 1 when idle and 0 when not idle.

accumulo.thrift.execute

Type: DISTRIBUTION_SUMMARY
Description: Time to execute an RPC request.

accumulo.thrift.idle

Type: DISTRIBUTION_SUMMARY
Description: Time waiting to execute an RPC request.

executor.completed

Type: FUNCTION_COUNTER
Description: Task completed by a thread pool. Each thread pool emits this metric w/ a different tag.

executor.queued

Type: GAUGE
Description: Task queued for a thread pool. Each thread pool emits this metric w/ a different tag.

Compaction Metrics

Metrics specific to compactions, both minor and major. Metrics for major compactions will likely have a ‘queue.id’ tag. The CompactionCoordinator component in the Manager creates a queue for each CompactionService in the configuration. The ‘queue.id’ tag may map directly to the name of a Compactor resource group.

accumulo.compaction.entries.read

Type: FUNCTION_COUNTER
Description: Number of entries read by all compactions that have run on this compactor (majc) or tserver (minc).

accumulo.compaction.entries.written

Type: FUNCTION_COUNTER
Description: Number of entries written by all compactions that have run on this compactor (majc) or tserver (minc).

accumulo.compaction.majc.cancelled

Type: FUNCTION_COUNTER
Description: Number compactions that have been cancelled on this compactor

accumulo.compaction.majc.completed

Type: FUNCTION_COUNTER
Description: Number compactions that have succeeded on this compactor

accumulo.compaction.majc.failed

Type: FUNCTION_COUNTER
Description: Number compactions that have failed on this compactor

accumulo.compaction.majc.failures.consecutive

Type: GAUGE
Description: Number of consecutive compaction failures. Resets to zero on a successful compaction

accumulo.compaction.majc.in_progress

Type: GAUGE
Description: Indicator of whether a compaction is in-progress (value: 1) or not (value: 0). An in-progress compaction could also be stuck.

accumulo.compaction.majc.stuck

Type: LONG_TASK_TIMER
Description: Number and duration of stuck major compactions.

accumulo.compaction.minc.paused

Type: COUNTER
Description: Number of paused minor compactions.

accumulo.compaction.minc.queued

Type: TIMER
Description: Queued minor compactions time queued.

accumulo.compaction.minc.running

Type: TIMER
Description: Minor compactions time active.

accumulo.compaction.minc.stuck

Type: LONG_TASK_TIMER
Description: Number and duration of stuck minor compactions.

accumulo.compaction.process.terminated

Type: FUNCTION_COUNTER
Description: Will report 1 if the Compactor terminates due to consecutive failures, else 0. Emitting this metric is a best effort before the process terminates

accumulo.compaction.queue.count

Type: GAUGE
Description: Number of priority queues for compaction jobs.

accumulo.compaction.queue.jobs.avg.age

Type: GAUGE
Description: Average age of currently queued jobs in seconds.

accumulo.compaction.queue.jobs.dequeued

Type: GAUGE
Description: Count of dequeued jobs.

accumulo.compaction.queue.jobs.exit.time

Type: TIMER
Description: Tracks time a job spent in the queue before exiting the queue.

accumulo.compaction.queue.jobs.max.age

Type: GAUGE
Description: Maximum age of currently queued jobs in seconds.

accumulo.compaction.queue.jobs.min.age

Type: GAUGE
Description: Minimum age of currently queued jobs in seconds.

accumulo.compaction.queue.jobs.priority

Type: GAUGE
Description: Lowest priority queued job.

accumulo.compaction.queue.jobs.queued

Type: GAUGE
Description: Count of queued jobs.

accumulo.compaction.queue.jobs.rejected

Type: GAUGE
Description: Count of rejected jobs.

accumulo.compaction.queue.jobs.size

Type: GAUGE
Description: Size of queued jobs in bytes.

accumulo.compaction.svc.meta.misconfigured

Type: GAUGE
Description: A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid.

accumulo.compaction.svc.root.misconfigured

Type: GAUGE
Description: A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid.

accumulo.compaction.svc.user.misconfigured

Type: GAUGE
Description: A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid.

accumulo.minc.queued

Type: GAUGE
Description: Number of queued minor compactions.

accumulo.minc.running

Type: GAUGE
Description: Number of active minor compactions.

accumulo.minc.total

Type: GAUGE
Description: Total number of minor compactions performed.

Compactor Metrics

Metrics that are generated by the Compactor processes.

accumulo.compaction.majc.paused

Type: COUNTER
Description: Number of paused major compactions.

Fate Metrics

Metrics that are generated by the Fate component in the Manager process.

accumulo.fate.errors

Type: GAUGE
Description: Count of errors that occurred when attempting to gather fate metrics.

accumulo.fate.ops

Type: GAUGE
Description: Number of all the current FATE ops in any state.

accumulo.fate.ops.activity

Type: GAUGE
Description: Count of the total number of times fate operations are added, updated, and removed.

accumulo.fate.ops.in.progress.by.type

Type: GAUGE
Description: Number of FATE operations in progress. The op type is designated by the op.type tag.

accumulo.fate.ops.threads.inactive

Type: GAUGE
Description: Keeps track of the number of idle threads (not working on a fate operation) in the thread pool. The pool name can be found in the pool.name tag. The fate instance type can be found in the instanceType tag.

accumulo.fate.ops.threads.total

Type: GAUGE
Description: Keeps track of the total number of threads in the thread pool. The pool name can be found in the pool.name tag. The fate instance type can be found in the instanceType tag.

accumulo.fate.tx

Type: GAUGE
Description: Count of FATE operations in a certain state. The state is now in a tag (e.g., state=new, state=in.progress, state=failed, etc.).

Garbage Collection Metrics

Metrics that are generated by the Garbage Collector process.

accumulo.gc.candidates

Type: GAUGE
Description: Number of files that are candidates for deletion.

accumulo.gc.deleted

Type: GAUGE
Description: Number of candidate files deleted.

accumulo.gc.errors

Type: GAUGE
Description: Number of candidate deletion errors.

accumulo.gc.finished

Type: GAUGE
Description: Timestamp GC file collect cycle finished.

accumulo.gc.in.use

Type: GAUGE
Description: Number of candidate files still in use.

accumulo.gc.post.op.duration

Type: GAUGE
Description: GC metadata table post operation duration in milliseconds.

accumulo.gc.run.cycle

Type: GAUGE
Description: Count of gc cycle runs. Value is reset on process start.

accumulo.gc.started

Type: GAUGE
Description: Timestamp GC file collection cycle started.

accumulo.gc.wal.candidates

Type: GAUGE
Description: Number of files that are candidates for deletion.

accumulo.gc.wal.deleted

Type: GAUGE
Description: Number of candidate wal files deleted.

accumulo.gc.wal.errors

Type: GAUGE
Description: Number candidate wal file deletion errors.

accumulo.gc.wal.finished

Type: GAUGE
Description: Timestamp GC WAL collect cycle finished.

accumulo.gc.wal.in.use

Type: GAUGE
Description: Number of wal file candidates that are still in use.

accumulo.gc.wal.started

Type: GAUGE
Description: Timestamp GC WAL collection cycle started.

Tablet Server Metrics

Metrics that are generated by the TabletServer processes.

accumulo.ingest.bytes

Type: GAUGE
Description: Ingest byte count. The rate can be derived from this metric.

accumulo.ingest.entries

Type: GAUGE
Description: Ingest entry (a key/value) count. The rate can be derived from this metric.

accumulo.ingest.hold

Type: GAUGE
Description: Duration for which commits have been held in milliseconds.

accumulo.tablets.assignments.warning

Type: GAUGE
Description: Number of tablet assignments that are taking longer than the configured warning duration.

accumulo.tablets.files

Type: GAUGE
Description: Number of files per tablet.

accumulo.tablets.ondemand.online

Type: GAUGE
Description: Number of online on-demand tablets

accumulo.tablets.ondemand.unloaded.lowmem

Type: GAUGE
Description: Number of online on-demand tablets unloaded due to low memory

accumulo.tablets.online

Type: GAUGE
Description: Number of online tablets.

accumulo.tablets.opening

Type: GAUGE
Description: Number of opening tablets.

accumulo.tablets.unopened

Type: GAUGE
Description: Number of unopened tablets.

accumulo.tserver.entries

Type: GAUGE
Description: Number of entries assigned to a TabletServer.

accumulo.tserver.entries.mem

Type: GAUGE
Description: Number of entries in memory.

accumulo.updates.check

Type: TIMER
Description: Average time taken for conditional mutation to check conditions.

accumulo.updates.commit

Type: TIMER
Description: Average time taken to commit a mutation.

accumulo.updates.commit.prep

Type: TIMER
Description: Average time taken to prepare to commit a single mutation.

accumulo.updates.error

Type: GAUGE
Description: Count of errors during tablet updates. Type/reason for error is stored in the type tag (e.g., type=permission, type=unknown.tablet, type=constraint.violation).

accumulo.updates.lock

Type: TIMER
Description: Average time taken for conditional mutation to get a row lock.

accumulo.updates.mutation.arrays.size

Type: DISTRIBUTION_SUMMARY
Description: Batch size of mutations from client.

accumulo.updates.walog.write

Type: TIMER
Description: Time taken to write a batch of mutations to WAL.

Scan Metrics

Metrics specific to scans, which can be executed in the ScanServer or the TabletServer.

accumulo.scan.busy.timeout.count

Type: FUNCTION_COUNTER
Description: Count of the scans where a busy timeout happened.

accumulo.scan.close

Type: FUNCTION_COUNTER
Description: Number of calls to close a scan or multiscan.

accumulo.scan.continue

Type: FUNCTION_COUNTER
Description: Number of calls to continue a scan or multiscan.

accumulo.scan.errors

Type: FUNCTION_COUNTER
Description: Number of scan task that had an exception.

accumulo.scan.files.open

Type: GAUGE
Description: Number of files open for scans.

accumulo.scan.paused.for.memory

Type: FUNCTION_COUNTER
Description: Count of scans paused due to server being low on memory.

accumulo.scan.queries

Type: FUNCTION_COUNTER
Description: Number of queries made during scans.

accumulo.scan.query.results

Type: FUNCTION_COUNTER
Description: Query count. The rate can be derived from this metric.

accumulo.scan.query.results.bytes

Type: FUNCTION_COUNTER
Description: Query byte count. The rate can be derived from this metric.

accumulo.scan.query.scanned.entries

Type: FUNCTION_COUNTER
Description: Count of scanned entries. The rate can be derived from this metric.

accumulo.scan.result

Type: DISTRIBUTION_SUMMARY
Description: Results per scan.

accumulo.scan.return.early.for.memory

Type: FUNCTION_COUNTER
Description: Count of scans that returned results early due to server being low on memory.

accumulo.scan.start

Type: FUNCTION_COUNTER
Description: Number of calls to start a scan or multiscan.

accumulo.scan.times

Type: TIMER
Description: Scan session lifetime (creation to close).

accumulo.scan.yields

Type: DISTRIBUTION_SUMMARY
Description: Counts scans that have yielded.

accumulo.scan.zombie.threads

Type: GAUGE
Description: Number of scan threads that have no associated client session.

Scan Server Metrics

Metrics that are generated by the ScanServer processes.

accumulo.scan.reservation.conflict.count

Type: COUNTER
Description: Count of instances where file reservation attempts for scans encountered conflicts.

accumulo.scan.reservation.total.timer

Type: TIMER
Description: Time to reserve a tablet’s files for scan.

accumulo.scan.reservation.writeout.timer

Type: TIMER
Description: Time to write out a tablets file reservations for scan.

accumulo.scan.tablet.metadata.cache

Type: CACHE
Description: Scan server tablet cache metrics.

Block Cache Metrics

Metrics specific to RFile block cache usage in the ScanServer and TabletServer processes.

accumulo.blockcache.data.evictioncount

Type: FUNCTION_COUNTER
Description: Data block cache eviction count.

accumulo.blockcache.data.hitcount

Type: FUNCTION_COUNTER
Description: Data block cache hit count.

accumulo.blockcache.data.requestcount

Type: FUNCTION_COUNTER
Description: Data block cache request count.

accumulo.blockcache.index.evictioncount

Type: FUNCTION_COUNTER
Description: Index block cache eviction count.

accumulo.blockcache.index.hitcount

Type: FUNCTION_COUNTER
Description: Index block cache hit count.

accumulo.blockcache.index.requestcount

Type: FUNCTION_COUNTER
Description: Index block cache request count.

accumulo.blockcache.summary.evictioncount

Type: FUNCTION_COUNTER
Description: Summary block cache eviction count.

accumulo.blockcache.summary.hitcount

Type: FUNCTION_COUNTER
Description: Summary block cache hit count.

accumulo.blockcache.summary.requestcount

Type: FUNCTION_COUNTER
Description: Summary block cache request count.

Manager Metrics

Metrics that are generated by the Manager process.

accumulo.balancer.migrations.needed

Type: GAUGE
Description: The number of migrations that need to complete before the system is balanced.

accumulo.manager.goal.state

Type: GAUGE
Description: Manager goal state: -1=unknown, 0=CLEAN_STOP, 1=SAFE_MODE, 2=NORMAL.

accumulo.tabletmgmt.meta.errors

Type: GAUGE
Description: Error count encountered by the TabletGroupWatcher for the META data level.

accumulo.tabletmgmt.meta.recovery

Type: GAUGE
Description: Recovery count encountered by the TabletGroupWatcher for the META data level.

accumulo.tabletmgmt.root.errors

Type: GAUGE
Description: Error count encountered by the TabletGroupWatcher for the ROOT data level.

accumulo.tabletmgmt.root.recovery

Type: GAUGE
Description: Recovery count encountered by the TabletGroupWatcher for the ROOT data level.

accumulo.tabletmgmt.user.errors

Type: GAUGE
Description: Error count encountered by the TabletGroupWatcher for the USER data level.

accumulo.tabletmgmt.user.recovery

Type: GAUGE
Description: Recovery count encountered by the TabletGroupWatcher for the USER data level.

Find documentation for all releases in the archive Edit this page