Table of Contents

ConcurrentLruCache<TKey, TValue> Class

Definition

Namespace
Bodu.Collections.Generic.Concurrent
Assembly
Bodu.Collections.Concurrent.dll
Package
Bodu.Collections.Concurrent 1.0.0
Source
ConcurrentLruCache{T,T}.Enumerator.cs

Provides a thread-safe, fixed-capacity cache with lock-free reads and a segmented pseudo-LRU eviction policy.

public sealed class ConcurrentLruCache<TKey, TValue> : ICollection, IReadOnlyDictionary<TKey, TValue>, IReadOnlyCollection<KeyValuePair<TKey, TValue>>, IEnumerable<KeyValuePair<TKey, TValue>>, IEnumerable where TKey : notnull

Type Parameters

TKey

Specifies the type of keys in the cache (constraint: where TKey : notnull).

TValue

Specifies the type of values in the cache.

Inheritance
ConcurrentLruCache<TKey, TValue>
Implements
IReadOnlyDictionary<TKey, TValue>
IEnumerable<KeyValuePair<TKey, TValue>>
Inherited Members
Extension Methods

Examples

var cache = new ConcurrentLruCache<string, byte[]>(capacity: 1024);

Parallel.ForEach(requests, request =>
{
    byte[] payload = cache.GetOrAdd(request.Key, key => LoadPayload(key));
    Serve(request, payload);
});

Console.WriteLine($"Hit ratio: {cache.HitRatio:P1}");

Remarks

ConcurrentLruCache<TKey, TValue> is the read-optimized bounded cache of the package: entries live in a ConcurrentDictionary<TKey, TValue> and recency is tracked by three internal FIFO queues - hot (new arrivals), warm (entries that have proven reuse), and cold (entries one unaccessed pass from eviction). A successful lookup is entirely lock-free: it performs the dictionary probe and sets the entry's accessed flag with a single volatile write. Queue maintenance - promoting accessed entries, demoting idle ones, evicting from the cold end - is amortized onto writers: after each mutating operation at most one thread briefly cycles the queues while contending writers proceed without waiting.

The policy is a pseudo-LRU: it approximates least-recently-used ordering the way production caches (BitFaster.Caching, Caffeine) do, favoring read throughput over exact recency ordering. For exact, selectable eviction policies - at the cost of taking a segment lock on every operation, including reads - use ConcurrentEvictingDictionary<TKey, TValue>. The two types also differ in their capacity contract: this cache's Count may transiently exceed Capacity by at most the number of concurrently in-flight writers. That bound is enforced by write back-pressure: a writer that observes the cache over capacity converges maintenance before returning, so sustained insert pressure against a full cache serializes writes on the maintenance lock while reads stay lock-free. Explicitly removed entries keep their queue slot until the maintenance cycle drains them, so internal occupancy accounting is eventual rather than instantaneous.

GetOrAdd(TKey, Func<TKey, TValue>) follows GetOrAdd(TKey, Func<TKey, TValue>) semantics: racing callers may each invoke the factory, and exactly one produced value is stored and returned to everyone. This differs from GetOrAdd(TKey, Func<TKey, TValue>), whose factory is single-flight; choose that type when a duplicated factory invocation is expensive enough to matter more than lock-free reads.

Hit and miss telemetry is maintained on cache-line-padded striped counters so the lock-free read path never contends on a shared counter; HitCount, MissCount, and HitRatio aggregate on demand. Evictions raise the post-commit ItemEvicted event after all internal coordination has been released, with handler exceptions suppressed (except OutOfMemoryException) - the package's established concurrent eviction-event contract.

Enumeration and ToArray() observe a coherent point-in-time snapshot of the backing dictionary and never throw because of concurrent modification; the order of entries is unspecified. Time-based expiration is not supported in this version.

Constructors

ConcurrentLruCache()

Initializes a new instance of the ConcurrentLruCache<TKey, TValue> class with the default capacity and key comparer.

public ConcurrentLruCache()

ConcurrentLruCache(int)

Initializes a new instance of the ConcurrentLruCache<TKey, TValue> class with the specified capacity and the default key comparer.

public ConcurrentLruCache(int capacity)

Parameters

capacity int

The maximum number of entries the cache aims to retain. Must be positive.

Exceptions

ArgumentOutOfRangeException

capacity is less than or equal to zero.

ConcurrentLruCache(int, IEnumerable<KeyValuePair<TKey, TValue>>)

Initializes a new instance of the ConcurrentLruCache<TKey, TValue> class with the specified capacity, seeded with the entries of the specified sequence, using the default key comparer.

public ConcurrentLruCache(int capacity, IEnumerable<KeyValuePair<TKey, TValue>> source)

Parameters

capacity int

The maximum number of entries the cache aims to retain. Must be positive.

source IEnumerable<KeyValuePair<TKey, TValue>>

The sequence of key/value pairs to copy. Must not be null.

Exceptions

ArgumentNullException

source is null.

ArgumentOutOfRangeException

capacity is less than or equal to zero.

ConcurrentLruCache(int, IEnumerable<KeyValuePair<TKey, TValue>>, IEqualityComparer<TKey>?)

Initializes a new instance of the ConcurrentLruCache<TKey, TValue> class with the specified capacity and key comparer, seeded with the entries of the specified sequence.

public ConcurrentLruCache(int capacity, IEnumerable<KeyValuePair<TKey, TValue>> source, IEqualityComparer<TKey>? comparer)

Parameters

capacity int

The maximum number of entries the cache aims to retain. Must be positive.

source IEnumerable<KeyValuePair<TKey, TValue>>

The sequence of key/value pairs to copy. Must not be null.

comparer IEqualityComparer<TKey>

The equality comparer to use for keys, or null to use the default comparer.

Remarks

If more elements are provided than the capacity allows, earlier entries are evicted per the pseudo-LRU policy as the copy proceeds.

Exceptions

ArgumentNullException

source is null.

ArgumentOutOfRangeException

capacity is less than or equal to zero.

ConcurrentLruCache(int, IEqualityComparer<TKey>?)

Initializes a new instance of the ConcurrentLruCache<TKey, TValue> class with the specified capacity and key comparer.

public ConcurrentLruCache(int capacity, IEqualityComparer<TKey>? comparer)

Parameters

capacity int

The maximum number of entries the cache aims to retain. Must be positive.

comparer IEqualityComparer<TKey>

The equality comparer to use for keys, or null to use the default comparer.

Remarks

The capacity is partitioned into hot, warm, and cold queue slices of roughly one third each; every slice the partition can afford is at least one entry, and the slices sum to capacity exactly.

Exceptions

ArgumentOutOfRangeException

capacity is less than or equal to zero.

Properties

ApproximateCount

Gets an approximate entry count without touching the backing dictionary.

public int ApproximateCount { get; }

Property Value

int

A lock-free counter of live entries, updated on every successful add, removal, and eviction. Under concurrent mutation the value may momentarily differ from Count by the number of in-flight operations; the two agree once mutation quiesces.

Remarks

Use this property for fast size estimates on hot paths; prefer Count for the exact number of live entries at a coherent instant.

Capacity

Gets the number of entries the cache aims to retain.

public int Capacity { get; }

Property Value

int

The configured capacity. Count may transiently exceed this value by at most the number of concurrently in-flight writers; maintenance converges the count back under the bound.

Comparer

Gets the equality comparer used for key identity and hash-table lookup.

public IEqualityComparer<TKey> Comparer { get; }

Property Value

IEqualityComparer<TKey>

The active equality comparer.

Count

Gets the number of live entries currently stored in the cache.

public int Count { get; }

Property Value

int

The exact count of entries in the backing dictionary at the moment of the call.

Remarks

Reading this property takes the backing dictionary's internal locks briefly. The value may transiently exceed Capacity by at most the number of concurrently in-flight writers.

EvictionCount

Gets the cumulative number of entries evicted by capacity pressure since creation or the last Clear().

public long EvictionCount { get; }

Property Value

long

The eviction count. Explicit removals and replacements are not evictions.

HitCount

Gets the cumulative number of successful lookups since creation or the last Clear().

public long HitCount { get; }

Property Value

long

The hit count, aggregated from striped per-processor counters; exact once lookups have quiesced.

HitRatio

Gets the fraction of lookups that were hits.

public double HitRatio { get; }

Property Value

double

HitCount / (HitCount + MissCount), or 0.0 when no lookups have been recorded.

Remarks

Lookups are TryGetValue(TKey, out TValue), the indexer getter, and GetOrAdd(TKey, TValue) / GetOrAdd(TKey, Func<TKey, TValue>). ContainsKey(TKey) is a pure probe and is not counted.

IsEmpty

Gets a value indicating whether the cache currently stores no live entries.

public bool IsEmpty { get; }

Property Value

bool

true if the backing dictionary is empty; otherwise, false.

this[TKey]

Gets or sets the value associated with the specified key.

public TValue this[TKey key] { get; set; }

Parameters

key TKey

The key of the value to get or set.

Property Value

TValue

The value associated with key.

Remarks

The getter behaves as TryGetValue(TKey, out TValue) (lock-free; a hit sets the accessed flag and counts toward HitCount, a miss counts toward MissCount before throwing); the setter behaves as Add(TKey, TValue) (add-or-replace).

Exceptions

ArgumentNullException

key is null.

KeyNotFoundException

The property is retrieved and no entry exists for key.

Keys

Gets a point-in-time snapshot of the cache's keys.

public IReadOnlyCollection<TKey> Keys { get; }

Property Value

IReadOnlyCollection<TKey>

A new read-only collection holding the keys present when the property was read.

Remarks

Each read takes a fresh dictionary snapshot and allocates; later mutations are not reflected. Reading the property does not set accessed flags and does not contribute to the telemetry counters.

MissCount

Gets the cumulative number of failed lookups since creation or the last Clear().

public long MissCount { get; }

Property Value

long

The miss count, aggregated from striped per-processor counters; exact once lookups have quiesced.

Values

Gets a point-in-time snapshot of the cache's values.

public IReadOnlyCollection<TValue> Values { get; }

Property Value

IReadOnlyCollection<TValue>

A new read-only collection holding the values present when the property was read.

Remarks

Each read takes a fresh dictionary snapshot and allocates; later mutations are not reflected. Reading the property does not set accessed flags and does not contribute to the telemetry counters.

Methods

Add(TKey, TValue)

Adds the specified key and value to the cache, or replaces the value if the key already exists.

public void Add(TKey key, TValue value)

Parameters

key TKey

The key of the element to add or update.

value TValue

The value to associate with key.

Remarks

Replacing an existing key installs a fresh entry: the previous node is retired and the new entry starts its recency life in the hot queue. This differs from ConcurrentEvictingDictionary<TKey, TValue>, which preserves eviction metadata on in-place replacement, and is a consequence of values being immutable on a lock-free read path. A replacement is not an eviction and does not raise ItemEvicted.

Adding a new key may push a queue over its slice; the triggered maintenance cycle then evicts from the cold end, raising ItemEvicted for each evicted entry.

Exceptions

ArgumentNullException

key is null.

Clear()

Removes all entries present at the time of the call and resets the telemetry counters.

public void Clear()

Remarks

The sweep runs under the maintenance lock: every entry currently in the backing dictionary is removed with a node-conditional remove, and the recency queues are drained of the resulting dead nodes. Entries added concurrently with the sweep may survive it - unlike Clear(), this cache has no global write lock to make the reset atomic against writers.

A clear is a bulk reset, not an eviction - ItemEvicted is not raised for the removed entries. HitCount, MissCount, and EvictionCount are reset to zero.

ContainsKey(TKey)

Determines whether the cache contains an entry for the specified key.

public bool ContainsKey(TKey key)

Parameters

key TKey

The key to locate.

Returns

bool

true if an entry exists for key; otherwise, false.

Remarks

This is a pure, lock-free probe: it does not set the entry's accessed flag and does not contribute to HitCount or MissCount.

Exceptions

ArgumentNullException

key is null.

GetEnumerator()

Returns an enumerator that iterates over a point-in-time snapshot of the cache's entries.

public ConcurrentLruCache<TKey, TValue>.Enumerator GetEnumerator()

Returns

ConcurrentLruCache<TKey, TValue>.Enumerator

An ConcurrentLruCache<TKey, TValue>.Enumerator over the entries present when the enumerator was created.

Remarks

Enumeration operates on a snapshot captured via ToArray() at the moment this method is called. Entries added, removed, or evicted afterward are not reflected in the enumerated sequence.

Because the enumerator operates on a snapshot, it never throws InvalidOperationException due to concurrent modification - unlike enumerators on non-concurrent collections. The order of enumerated entries is unspecified.

GetOrAdd(TKey, Func<TKey, TValue>)

Adds a key/value pair produced by the specified value factory if the key does not already exist, and returns either the existing or the newly created value.

public TValue GetOrAdd(TKey key, Func<TKey, TValue> valueFactory)

Parameters

key TKey

The key of the element to get or add.

valueFactory Func<TKey, TValue>

The function used to generate a value for the key when it is absent.

Returns

TValue

The existing value when an entry for key is present; otherwise the value produced by valueFactory.

Remarks

The factory runs outside all internal coordination, so racing callers missing on the same key may each invoke it; exactly one produced value is stored, and every caller returns the stored value. Values produced by losing invocations are discarded. This matches GetOrAdd(TKey, Func<TKey, TValue>) and keeps the read path free of per-hit indirection; when the factory must run at most once per key, use GetOrAdd(TKey, Func<TKey, TValue>), whose factory is single-flight.

If the factory throws, nothing is added and the exception propagates.

Exceptions

ArgumentNullException

key or valueFactory is null.

GetOrAdd(TKey, TValue)

Adds a key/value pair to the cache if the key does not already exist, and returns either the existing or the newly added value.

public TValue GetOrAdd(TKey key, TValue value)

Parameters

key TKey

The key of the element to get or add.

value TValue

The value to add when the key is absent.

Returns

TValue

The existing value when an entry for key is present; otherwise value.

Remarks

A hit is lock-free, sets the entry's accessed flag, and counts toward HitCount; a successful add counts toward MissCount and may trigger evictions via the maintenance cycle.

Exceptions

ArgumentNullException

key is null.

ToArray()

Copies the cache's entries into a new array.

public KeyValuePair<TKey, TValue>[] ToArray()

Returns

KeyValuePair<TKey, TValue>[]

A new array containing a coherent point-in-time snapshot of the cache's entries, or an empty array if the cache is empty. The order of entries is unspecified.

Remarks

The snapshot is taken by the backing dictionary under its internal locks and is unaffected by concurrent modifications made afterward. Taking a snapshot does not set accessed flags and does not contribute to the telemetry counters.

TryAdd(TKey, TValue)

Attempts to add the specified key and value to the cache, without replacing an existing entry.

public bool TryAdd(TKey key, TValue value)

Parameters

key TKey

The key of the element to add.

value TValue

The value to associate with key.

Returns

bool

true if the entry was added; false if an entry for key already exists.

Remarks

A rejected add does not set the existing entry's accessed flag and does not contribute to the telemetry counters. A successful add may trigger evictions via the maintenance cycle.

Exceptions

ArgumentNullException

key is null.

TryGetValue(TKey, out TValue)

Attempts to retrieve the value associated with the specified key.

public bool TryGetValue(TKey key, out TValue value)

Parameters

key TKey

The key of the value to retrieve.

value TValue

When this method returns, contains the value associated with the specified key, if the key is found; otherwise, the default value for the type of the value parameter.

Returns

bool

true if the cache contains an entry for the specified key; otherwise, false.

Remarks

The lookup is entirely lock-free: a hit performs the dictionary probe and one volatile write of the entry's accessed flag (skipped when already set, so repeated hot-key reads do not write at all). A hit counts toward HitCount, a miss toward MissCount.

Exceptions

ArgumentNullException

key is null.

TryRemove(TKey, out TValue)

Attempts to remove the entry with the specified key, returning the removed value.

public bool TryRemove(TKey key, out TValue value)

Parameters

key TKey

The key of the entry to remove.

value TValue

When this method returns true, the value of the removed entry; otherwise, the default value for the type.

Returns

bool

true if an entry was found and removed; otherwise, false.

Remarks

The key is reusable immediately, but the removed entry's dead queue node keeps its occupancy slot until a later maintenance pass drains it. An explicit removal is not an eviction - it does not raise ItemEvicted and does not increment EvictionCount.

Exceptions

ArgumentNullException

key is null.

Events

ItemEvicted

Occurs immediately after an entry has been evicted by capacity pressure.

public event Action<TKey, TValue>? ItemEvicted

Event Type

Action<TKey, TValue>

Remarks

Handlers run on the thread whose write triggered the maintenance cycle, after the maintenance lock has been released, so a handler can safely call back into the cache. The key and value provided are no longer present by the time the handler observes them.

Each subscriber is invoked independently, and ordinary handler exceptions are caught and suppressed - only OutOfMemoryException propagates. There is no pre-removal event: a committed concurrent eviction cannot be unwound.

Explicit removals via TryRemove(TKey, out TValue), replacements via Add(TKey, TValue) or the indexer setter, and bulk resets via Clear() are not evictions and do not raise this event.

Explicit Interface Implementations

IEnumerable<KeyValuePair<TKey, TValue>>.GetEnumerator()

Returns an enumerator that iterates over a point-in-time snapshot of the cache's entries.

IEnumerator<KeyValuePair<TKey, TValue>> IEnumerable<KeyValuePair<TKey, TValue>>.GetEnumerator()

Returns

IEnumerator<KeyValuePair<TKey, TValue>>

An ConcurrentLruCache<TKey, TValue>.Enumerator over the entries present when the enumerator was created.

Remarks

Enumeration operates on a snapshot captured via ToArray() at the moment this method is called. Entries added, removed, or evicted afterward are not reflected in the enumerated sequence.

Because the enumerator operates on a snapshot, it never throws InvalidOperationException due to concurrent modification - unlike enumerators on non-concurrent collections. The order of enumerated entries is unspecified.

IReadOnlyDictionary<TKey, TValue>.Keys

Gets an enumerable collection that contains the keys in the read-only dictionary.

IEnumerable<TKey> IReadOnlyDictionary<TKey, TValue>.Keys { get; }

Returns

IEnumerable<TKey>

An enumerable collection that contains the keys in the read-only dictionary.

IReadOnlyDictionary<TKey, TValue>.Values

Gets an enumerable collection that contains the values in the read-only dictionary.

IEnumerable<TValue> IReadOnlyDictionary<TKey, TValue>.Values { get; }

Returns

IEnumerable<TValue>

An enumerable collection that contains the values in the read-only dictionary.

ICollection.CopyTo(Array, int)

Copies the entries of the ConcurrentLruCache<TKey, TValue> to a one-dimensional, zero-based Array, starting at the specified index.

void ICollection.CopyTo(Array array, int index)

Parameters

array Array

The destination array. Must not be null, must be single-dimensional, and must have zero-based indexing.

index int

The zero-based index in array at which copying begins.

Remarks

This method takes a point-in-time snapshot of the cache before copying. The destination array reflects the state of the cache at the moment the snapshot was taken and is not affected by concurrent modifications made afterward.

Exceptions

ArgumentNullException

array is null.

ArgumentException

array is multidimensional, does not have zero-based indexing, its element type is incompatible with KeyValuePair<TKey, TValue>, or the number of entries in the snapshot exceeds the available space from index to the end of array.

ArgumentOutOfRangeException

index is less than zero.

ICollection.IsSynchronized

Gets a value indicating whether access to the ConcurrentLruCache<TKey, TValue> is synchronized (thread safe).

bool ICollection.IsSynchronized { get; }

Returns

bool

Always false. ConcurrentLruCache<TKey, TValue> manages its own internal synchronization and does not expose a public lock object.

Remarks

Thread safety is achieved through lock-free reads over a concurrent dictionary plus a write-amortized maintenance cycle. Callers should not attempt to coordinate access externally via SyncRoot, as that property is not supported.

ICollection.SyncRoot

Gets an object that can be used to synchronize access to the collection. Not supported on this type - ConcurrentLruCache<TKey, TValue> manages its own internal synchronization.

object ICollection.SyncRoot { get; }

Returns

object

Remarks

Exposing a SyncRoot would allow callers to take the same coordination used internally, undermining the concurrency guarantees of the collection. This matches the behavior of ConcurrentDictionary<TKey, TValue> and other BCL concurrent collections.

Exceptions

NotSupportedException

Always thrown. Use the thread-safe members of this class directly.

IEnumerable.GetEnumerator()

Returns an enumerator that iterates over a point-in-time snapshot of the cache's entries.

IEnumerator IEnumerable.GetEnumerator()

Returns

IEnumerator

An ConcurrentLruCache<TKey, TValue>.Enumerator over the entries present when the enumerator was created.

Remarks

Enumeration operates on a snapshot captured via ToArray() at the moment this method is called. Entries added, removed, or evicted afterward are not reflected in the enumerated sequence.

Because the enumerator operates on a snapshot, it never throws InvalidOperationException due to concurrent modification - unlike enumerators on non-concurrent collections. The order of enumerated entries is unspecified.

Applies to

ProductVersions
.NET8, 10