com.neeve.rog.log

Class RogLogCheckpointReaderParams

java.lang.Object

com.neeve.rog.log.RogLogCheckpointReaderParams

Direct Known Subclasses:

RogLogCdcProcessor.Params, RogLogCompactor.Params

public class RogLogCheckpointReaderParams
extends Object

Configuration parameters for the checkpoint reader used by a RogLogCompactor or a RogLogCdcProcessor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy Enumerates the various loader strategies that can be used to load entries not retained by the checkpoint reader during its conflation and sorting pass.

Field Summary

Fields

Modifier and Type	Field and Description
static String	PROP_CONFLATE_CHECKPOINTS Specifies that the checkpoint reader should conflate checkpoints prior to dispatching to the handler.
static boolean	PROP_CONFLATE_CHECKPOINTS_DEFAULT
static String	PROP_ENTRY_RETENTION_THRESHOLD The maximum number of read entries that should be held in memory while preparing the compacted set of entries.
static int	PROP_ENTRY_RETENTION_THRESHOLD_DEFAULT
static String	PROP_MEM_MAPPED_READ_BUFFER_LOAD When PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY is RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy.MemoryMappedRead, controls whether or not to attempt to load the entire region into memory up front.
static boolean	PROP_MEM_MAPPED_READ_BUFFER_LOAD_DEFAULT
static String	PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE When PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY is RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy.MemoryMappedRead, controls the maximum size of the file to map into memory at a time.
static String	PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE_DEFAULT
static String	PROP_OFF_HEAP_BUFFERING Indicates whether or not non-retained entries should be buffered into an off heap buffer.
static boolean	PROP_OFF_HEAP_BUFFERING_DEFAULT
static String	PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY When off heap buffering is enabled, how to load sorted entries into off heap memory.
static RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy	PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY_DEFAULT
static String	PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE Indicates the maximum size (in bytes) of the buffers to use to hold off heap entries.
static String	PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE_DEFAULT
static String	PROP_SKIP_CDC_CHECKPOINTED Flag indicating the the compaction reader should skip over entries with a checkpoint version higher than the log's current CdcCheckpointVersion.
static boolean	PROP_SKIP_CDC_CHECKPOINTED_DEFAULT

Method Summary

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROP_ENTRY_RETENTION_THRESHOLD

public static final String PROP_ENTRY_RETENTION_THRESHOLD

The maximum number of read entries that should be held in memory while preparing the compacted set of entries.

The compaction reader will retain references to entries it reads from the log up to this threshold. When the threshold is exceeded the the reader will only retain a poin retain a pointer to the entry in the file which will be loaded on demand as needed by the handler.

Specifying a higher value results in faster access time for entries, but has higher memory overhead.

See Also:

Constant Field Values

PROP_ENTRY_RETENTION_THRESHOLD_DEFAULT

public static final int PROP_ENTRY_RETENTION_THRESHOLD_DEFAULT

See Also:

Constant Field Values

PROP_CONFLATE_CHECKPOINTS

public static final String PROP_CONFLATE_CHECKPOINTS

Specifies that the checkpoint reader should conflate checkpoints prior to dispatching to the handler.

When set, this flag will cause the compaction reader to conflate entries into a single checkpoint dispatched to the RogLogCompactionReader.ReadHandler. At most one entry will be dispatched to the ReadHandler per object, and Send and Remove entries won't be dispatched at all.

When used, all entries will be conflated up through the last stabilized checkpoint. A stabilized checkpoint is one in which all of the entries in the checkpoint are part of transactions that are that are less than or equal to the last recorded stableTransactionid in the log and at least one entry from a subsequent checkpoint has been logged to demarcate the start of a new checkpoint.

When set to false, this flag will cause the checkpoint reader to dispatch each checkpoint to the compactor allowing it to do incremental compaction in accordance with the configured maxCompactionWindowSize, which results in multiple compaction runs being done to fully compact the log, but can result in less memory usage by the compactor.

This property should not be enabled for readers (such as CDC) that are interested in seeing the intermediate changes to objects being conflated.

See Also:

Constant Field Values

PROP_CONFLATE_CHECKPOINTS_DEFAULT

public static final boolean PROP_CONFLATE_CHECKPOINTS_DEFAULT

See Also:

Constant Field Values

PROP_SKIP_CDC_CHECKPOINTED

public static final String PROP_SKIP_CDC_CHECKPOINTED

Flag indicating the the compaction reader should skip over entries with a checkpoint version higher than the log's current CdcCheckpointVersion.

When set this will cause the compaction reader to skip over entries below the CdcCheckpoint version which reduces the memory overhead and read time for readers only interested in newly changed entries.

For a CDC handler, setting this flag reduces the number of checkpoints that the checkpoint reader needs to pull into memory.

See Also:

Constant Field Values

PROP_SKIP_CDC_CHECKPOINTED_DEFAULT

public static final boolean PROP_SKIP_CDC_CHECKPOINTED_DEFAULT

See Also:

Constant Field Values

PROP_OFF_HEAP_BUFFERING

public static final String PROP_OFF_HEAP_BUFFERING

Indicates whether or not non-retained entries should be buffered into an off heap buffer.

When this flag is set, and the value of entries loaded by the reader exceeds that set by PROP_ENTRY_RETENTION_THRESHOLD, the compacted set of entries are sorted and loaded into an off heap memory buffer prior to dispatch to the handler.

This can result in a significant increase in performance as buffering and retrieval of the entries can be done in a sequential fashion. Enabling this will result in the allocation of buffer space equivalent to the size of the entries on disk to be dispatched to the handler in a checkpoint (or if PROP_CONFLATE_CHECKPOINTS is set, the size of the conflate entry set). It is important to note that this will not result in the actual entries being loaded, so this does not incur the expense of java object overhead.

See Also:

Constant Field Values

PROP_OFF_HEAP_BUFFERING_DEFAULT

public static final boolean PROP_OFF_HEAP_BUFFERING_DEFAULT

See Also:

Constant Field Values

PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY

public static final String PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY

When off heap buffering is enabled, how to load sorted entries into off heap memory.

When off heap buffering is enabled, entries that were not retained are loaded into memory after being conflated and sorted. This property controls the mechanism use to load the conflated entries into memory prior to disatch to the handler.

See Also:

Constant Field Values

PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY_DEFAULT

public static final RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY_DEFAULT

PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE

public static final String PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE

Indicates the maximum size (in bytes) of the buffers to use to hold off heap entries.

As entries are stored off heap new buffers are serialized in off heap pages with a maximum size given by the property.

The maximum allowable value of this property is Integer.MAX_VALUE, and the minimum value is 8kb.

When no unit is provided this value is interpreted in bytes.

See Also:

Constant Field Values

PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE_DEFAULT

public static final String PROP_OFF_HEAP_BUFFERING_MAX_PAGE_SIZE_DEFAULT

See Also:

Constant Field Values

PROP_MEM_MAPPED_READ_BUFFER_LOAD

public static final String PROP_MEM_MAPPED_READ_BUFFER_LOAD

When PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY is RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy.MemoryMappedRead, controls whether or not to attempt to load the entire region into memory up front.

See Also:

Constant Field Values

PROP_MEM_MAPPED_READ_BUFFER_LOAD_DEFAULT

public static final boolean PROP_MEM_MAPPED_READ_BUFFER_LOAD_DEFAULT

See Also:

Constant Field Values

PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE

public static final String PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE

When PROP_OFF_HEAP_BUFFERING_LOAD_STRATEGY is RogLogCheckpointReaderParams.OffheapBufferingLoadStrategy.MemoryMappedRead, controls the maximum size of the file to map into memory at a time.

The maximum supported value in Integer.MAX_VALUE.

When no unit is provided this value is interpreted in bytes.

See Also:

Constant Field Values

PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE_DEFAULT

public static final String PROP_MEM_MAPPED_READ_BUFFER_MAX_SIZE_DEFAULT

See Also:

Constant Field Values