Table of Contents

Caching and aggregating exchange rates

Bodu.Financial.ExchangeRates.Caching adds two pieces in front of the exchange-rate providers. The concrete providers (BoE, ECB, RBA, Yahoo, OFX, XE, OANDA, Fixer, exchangerate.host, FRED, IMF) stay pure fetchers that know nothing of caching; each piece implements the same IDatedRateProvider contract (and the timeless IRateProvider), so they drop in transparently:

The caller talks to an AggregatingRateProvider, which routes each FX pair to a CachingRateProvider; each caching provider reads through its own cache (SQLite, in-memory, …) and calls its concrete source only on a miss.

The two pieces are orthogonal: use the cache alone to add read-through caching to a single source, the aggregator alone to group already-cached (or uncached) providers, or compose them as above.

Concepts in one minute

  • Caching provider - CachingRateProvider wraps one inner source over one single-provider cache. It serves fresh cached rates and delegates to the source only on a miss, then caches what the source returns.
  • Cache - a cache is bound to one provider. IRateCache owns expiry: callers pass a duration, and the cache returns only fresh rows and prunes stale ones on write. Shipped stores are TomlFileRateCache and JsonFileRateCache (on disk, with a configurable layout and date partitioning), InMemoryRateCache, and the no-op NullRateCache.
  • Options - CachingRateOptions carries the cache location (CacheDirectory), the default expiry (DefaultExpiry), per-provider overrides (ProviderExpiry), the per-event log levels, and DefaultLookupOptions for the timeless surface.
  • Aggregator - AggregatingRateProvider groups named children behind one entry point, combining them with a pluggable IRateAggregationStrategy and optional per-FX-pair routing.
  • Entry - CachedRate is one cached row: the observation Date, the Rate, and the CachedAtUtc instant that drives expiry.

Quickstart

A durable SQLite cache in front of the RBA source, end to end. Both snippets are self-contained - nothing is assumed to exist beforehand.

Dependency injection (recommended - the container owns the HttpClient, logger, and disposal):

using Bodu.Financial;
using Bodu.Financial.ExchangeRates;          // AddRbaExchangeRates, RbaRateProvider
using Bodu.Financial.ExchangeRates.Caching;  // IRateCache
using Microsoft.Extensions.DependencyInjection;

var services = new ServiceCollection();

services.AddFinancialService()
        .AddRbaExchangeRates()                                   // the concrete RBA source
        .AddSqliteRateCache("RBA",                                 // a durable cache bound to it
            configure: o => o.DatabaseFilePath = "/var/cache/fx.db")
        .AddCachedRateProvider<RbaRateProvider>(   // read-through over that cache
            "RBA",
            cacheFactory: (sp, name) => sp.GetRequiredKeyedService<IRateCache>(name));

using var host = services.BuildServiceProvider();
var rates = host.GetRequiredService<IDatedRateProvider>();

RateLookupResult aud = rates.GetRate("AUD", "USD", new DateOnly(2024, 1, 3));
// aud.Rate is served from SQLite when fresh, otherwise fetched from the RBA source and cached.

Manual (new up every piece yourself - useful outside a DI container):

using Bodu.Financial;
using Bodu.Financial.ExchangeRates;          // RbaRateProvider, RbaRateProviderOptions
using Bodu.Financial.ExchangeRates.Caching;  // SqliteRateCache, CachingRateProvider

// The concrete source. This overload builds and owns an HttpClient, so dispose the provider.
using var source = new RbaRateProvider(new RbaRateProviderOptions());

// A durable cache for it, wrapped in a read-through provider.
using var cache = new SqliteRateCache("RBA", "/var/cache/fx.db");
var rates = new CachingRateProvider(source, cache, new CachingRateOptions());

RateLookupResult aud = rates.GetRate("AUD", "USD", new DateOnly(2024, 1, 3));

From here: add more providers behind one file, stack a faster tier, group several sources, or wire the degradation logging.

Caching one provider

CachingRateProvider caches exactly one source. It is storage-agnostic: it never chooses or constructs a cache, so you supply the IRateCache - and therefore the storage structure (TOML or JSON files, the on-disk layout and partitioning, in-memory, SQLite, or distributed) - at the composition root. The provider classes never learn they are being cached.

sequenceDiagram
    participant Caller
    participant Provider as CachingRateProvider
    participant Cache as IRateCache
    participant Source as Source provider
    Caller->>Provider: GetRate(AUD, USD, date)
    Provider->>Cache: fresh cached rate?
    alt cache hit
        Cache-->>Provider: fresh rate
    else cache miss
        Provider->>Source: fetch
        Source-->>Provider: rate
        Provider->>Cache: store (merge + prune)
    end
    Provider-->>Caller: RateLookupResult

var options = new CachingRateOptions
{
    DefaultExpiry = TimeSpan.FromHours(12),
};
options.ProviderExpiry["RBA"] = TimeSpan.FromDays(7);   // RBA publishes daily; cache longer

// Pick the cache explicitly. Cache files land under /var/cache/fx/RBA/.
var rbaCache = new TomlFileRateCache(
    new FileRateCacheOptions { Provider = "RBA", CacheDirectory = "/var/cache/fx" });
IDatedRateProvider cachedRba = new CachingRateProvider(rba, rbaCache, options);

// Or any other IRateCache - for example the in-memory store.
IDatedRateProvider cachedEcb =
    new CachingRateProvider(ecb, new InMemoryRateCache("ECB"), options);

The decorator is IDisposable. By default it does not dispose the inner provider - under dependency injection the container owns it, and a hand-composed inner is owned by whoever created it. Pass ownsInner: true to make disposing the decorator also dispose a disposable inner (for example a provider that built its own HttpClient):

using var cached = new CachingRateProvider(
    new RbaRateProvider(rbaOptions),
    new TomlFileRateCache(new FileRateCacheOptions { Provider = "RBA", CacheDirectory = "/var/cache/fx" }),
    options,
    ownsInner: true);

The decorator also implements the timeless surface, which resolves the current UTC date under CachingRateOptions.DefaultLookupOptions:

decimal todayRate = ((IRateProvider)cachedRba).GetRate("AUD", "USD");

Per-provider expiry and the global default

GetExpiry(name) returns a provider's specific override when present and DefaultExpiry otherwise:

options.GetExpiry("RBA");     // 7 days   (override)
options.GetExpiry("ECB");     // 12 hours (the default)

Single-date lookups

GetRate / TryGetRate flow through the cache. On a hit the cached rows are reconstructed into a FixedDatedRateProvider, so date-resolution policy, inverse pairs, and same-currency identity all behave exactly as the underlying stack would:

// Miss → fetched from the source, then cached.
RateLookupResult r1 = cachedRba.GetRate("AUD", "USD", new DateOnly(2024, 1, 3));

// Repeat within the expiry window → served from cache, no source call.
RateLookupResult r2 = cachedRba.GetRate("AUD", "USD", new DateOnly(2024, 1, 3));

// Resolution policies are honoured against the cached rows.
cachedRba.TryGetRate("AUD", "USD", new DateOnly(2024, 1, 5),
    RateLookupOptions.PreviousWithin(7), out RateLookupResult r3);

Range lookups

GetRatesAsync returns every rate whose date falls in the inclusive window. Whether the cache can serve a range is decided by coverage - the date ranges the source was actually fetched for - not by the span of the stored rows. A range is served from the cache only when the recorded coverage contains the whole requested window; otherwise the range is refetched and the rows plus the covered window are written back together (atomically) through StoreFetchedRange.

When the direct pair's coverage does not contain the window but the inverse pair's does - and inversion is permitted by the provider's DefaultLookupOptions (the default) - the range is served from the inverse pair by reciprocating each rate. So a USD/AUD range already fetched also satisfies an AUD/USD range request without a refetch, mirroring the single-date surface.

IReadOnlyList<ExchangeRate> january =
    await cachedRba.GetRatesAsync("AUD", "USD", new DateOnly(2024, 1, 1), new DateOnly(2024, 1, 31));
Note

Coverage is recorded for the whole fetched window even on days that returned no observation (a weekend, a holiday, a true gap), so a later lookup of the same window is served from the cache rather than refetched. A sparse set of rows is therefore never mistaken for proof that every interior day was fetched - the distinction a DateRangeCoverage makes explicit.

Respecting advertised history

Every shipped provider advertises how far back it can serve rates through HistoryAvailability, and the caching decorator consumes that declaration by default (RespectHistoryAvailability): when the inner provider implements IHistoricalRateProvider, misses for dates the source has declared unavailable are not forwarded.

  • Single-date lookups outside the advertised history surface as an ordinary miss (TryGetRate returns false; the throwing surfaces raise the usual KeyNotFoundException) without the inner provider being called. Forward-resolving rules are honoured: a request just before the advertised floor whose NextOnOrAfter/Nearest tolerance reaches back inside it is still delegated.
  • Range fetches that start before the advertised earliest date are issued from that date instead, and a window lying entirely before it is not fetched at all. In both cases coverage is recorded over the whole requested window, so the unavailable prefix becomes the covered-with-no-rows negative cache described above and is not re-asked until normal expiry.

The clamp only ever removes calls the source has declared doomed - a non-aware inner provider is treated as unbounded and never skipped, and a date inside the advertised window can still miss for ordinary reasons (weekends, holidays, unpublished series). Skips and clamps are logged at HistoryClampLogLevel. The decorator forwards the inner's declaration as its own HistoryAvailability, so stacked caches and the aggregator see through it.

The aggregator applies the same idea at routing time (via its own RespectHistoryAvailability): candidates that declared they cannot serve any part of the requested date or window are dropped before the strategy runs, so a priority fallback does not waste a call on a source that cannot answer - a shallow rolling-window source is skipped for deep historical dates while an unbounded central-bank source answers them. Its composed HistoryAvailability is the most generous declaration across the children.

The cache cascade

The cache is deliberately layered so you can plug in at whichever level fits:

Layer Type Responsibility
Contract IRateCache Single-provider store: a bound Provider; rate rows via GetRates/Store, fetch coverage via GetCoverage/RecordCoverage, and the atomic StoreFetchedRange that writes both together.
Core RateCacheBase<TOptions> Per-pair locking + row/coverage freshness filtering, merge, and prune. No physical layout.
File seam IFileRateCache / FileRateCacheBase<TOptions> Layout-driven directory + file-name resolution, date partitioning, best-effort IO.
Leaf TomlFileRateCache / JsonFileRateCache The TOML or JSON serialization format only.

The on-disk format

A cache bound to provider RBA stores AUD/USD as <directory>/RBA/AUDUSD.toml - a per-provider subdirectory with one file per pair (this default layout, and how to change it, is covered under File layouts and date partitioning). Each file opens with a self-describing header - the bound Provider and the pair's From/To currency codes - so a file carries its own identity rather than relying on its name and folder. Each dated rate is then a TOML table; the decimal rate is written as a quoted string so its full precision and scale round-trip exactly, and the dates use TOML's native RFC 3339 forms:

Provider = "RBA"
From = "AUD"
To = "USD"

[[Entries]]
Date = 2023-01-03
Rate = "0.5000"
CachedAtUtc = 2023-01-04T09:15:00+00:00

[[Entries]]
Date = 2023-01-06
Rate = "0.5100"
CachedAtUtc = 2023-01-04T09:15:00+00:00

The serializer is Bodu.Text.Toml with TomlDecimalHandling.String. A file written before the header existed simply has no Provider/From/To keys and still reads. The file is best-effort: any I/O or TOML error on read yields an empty result, and a failed write is swallowed, so a cache problem never breaks rate retrieval. You can use a cache directly - note there is no provider argument; the cache is bound to its provider at construction:

var cache = new TomlFileRateCache(
    new FileRateCacheOptions { Provider = "RBA", CacheDirectory = "/var/cache/fx" });

var now = DateTimeOffset.UtcNow;
cache.Store(new CurrencyPair(CurrencyCode.AUD, CurrencyCode.USD),
    new[] { new CachedRate(new DateOnly(2023, 1, 3), 0.5000m, now) },
    TimeSpan.FromHours(24), now);

IReadOnlyList<CachedRate> fresh =
    cache.GetRates(new CurrencyPair(CurrencyCode.AUD, CurrencyCode.USD), TimeSpan.FromHours(24), now);

The JSON format

JsonFileRateCache is the same cache with a JSON body instead of TOML - same .json files, same self-describing header, same layouts and partitioning, same best-effort IO. It is a drop-in swap when you want a format other tools read natively; decimals are written as JSON numbers, which System.Text.Json round-trips losslessly to decimal:

{
  "Provider": "RBA",
  "From": "AUD",
  "To": "USD",
  "Entries": [
    { "Date": "2023-01-03", "Rate": 0.5000, "CachedAtUtc": "2023-01-04T09:15:00+00:00" }
  ],
  "Coverage": []
}

var cache = new JsonFileRateCache(
    new FileRateCacheOptions { Provider = "RBA", CacheDirectory = "/var/cache/fx" });

File layouts and date partitioning

The Layout option decides where a pair's rows are stored: the folder hierarchy, the file name, and whether the rows are split across files by date. It defaults to RateCacheFileLayout.SingleFile - the <directory>/<provider>/<from><to>.toml layout shown above. The built-in partitioned layouts isolate each pair in its own folder and write one file per calendar period, keyed by the period:

Layout Files for AUD/USD under provider RBA
SingleFile (default) RBA/AUDUSD.toml
Yearly RBA/AUDUSD/2023.toml, RBA/AUDUSD/2024.toml, …
Monthly RBA/AUDUSD/2023-01.toml, RBA/AUDUSD/2023-02.toml, …
Daily RBA/AUDUSD/2023-01-03.toml, …

Each rate is routed to the file for its observation date, and a recorded coverage window that crosses a period boundary is split at the boundary so each file carries only its own period; a read concatenates every file in the pair's folder and the shared cache rules re-merge the halves, so the split is lossless.

// One file per month for each pair.
var monthly = new TomlFileRateCache(new FileRateCacheOptions
{
    Provider = "RBA",
    CacheDirectory = "/var/cache/fx",
    Layout = RateCacheFileLayout.Monthly,
});

For a layout the built-ins do not cover, build one with RateCacheFileLayout.Create: supply a partition strategy (one of Single/Yearly/Monthly/Daily, or RateCachePartitionStrategy.Custom for an arbitrary period such as fiscal quarters) and optional delegates that decide the directory and the file name:

var custom = new TomlFileRateCache(new FileRateCacheOptions
{
    Provider = "RBA",
    CacheDirectory = "/var/cache/fx",
    Layout = RateCacheFileLayout.Create(
        RateCachePartitionStrategy.Yearly,
        directory: ctx => System.IO.Path.Combine(ctx.Root, "fx", ctx.Provider, $"{ctx.Pair.From}{ctx.Pair.To}"),
        fileName: ctx => $"{ctx.PartitionKey}{ctx.FileExtension}"),
});

A partitioned layout has no single backing file, so ResolveFilePath throws for it; use ResolveDirectory(pair) for the pair's folder or ResolvePartitionPath(pair, date) for the file a given date lands in. CachingRateProvider takes whatever IRateCache you hand it, so a custom layout, the JSON format, or a SQLite or distributed cache is simply the cache you construct and pass to its (inner, cache, options) constructor. Under dependency injection, pass a cacheFactory to AddCachedRateProvider (or AddCachedChild) to choose the storage; when omitted, a default single-file TOML cache under the options' CacheDirectory is used.

Custom cache stores

A cache backend is any IRateCache implementation. To back the cache with a store of your own, implement that interface directly - the shipped SqliteRateCache and DistributedRateCache are exactly that and serve as worked references. Delegate the freshness, validity, merge, and coverage rules to the shared, public RateCacheRules so your backend stays behaviourally identical to the in-box caches - serving only fresh rows on read, merging and pruning on write, and reporting coverage only for windows it actually holds - and make StoreFetchedRange write the merged rows and the covered window as one atomic unit so a reader never observes coverage without its rows.

The in-box RateCacheBase<TOptions> and FileRateCacheBase<TOptions> are internal scaffolding for the in-memory, TOML, and JSON caches - they own the per-pair locking and the read-modify-write sequencing over a CachePairState - and their storage seam is not a public subclassing point. Implement IRateCache directly, as the SQLite and distributed backends do.

Persistent and shared backends

Two further IRateCache backends ship as separate packages and drop in the same way - construct one and hand it to a CachingRateProvider, or register it through the DI extension method that ships inside the backend's own package (in the Bodu.Financial.ExchangeRates namespace):

  • SqliteRateCache (Bodu.Financial.ExchangeRates.Caching.Sqlite) persists rates and coverage in a SQLite database - durable across restarts, with per-pair transactional writes. Register it with AddSqliteRateCache("RBA", …).
  • DistributedRateCache (Bodu.Financial.ExchangeRates.Caching.Distributed) stores each pair as a JSON blob in any IDistributedCache (Redis, SQL Server, in-memory), so several processes share one warm cache. Register it with AddDistributedRateCache("RBA") or AddRedisRateCache(redis => …, "RBA") - the Redis configurator is the first argument, the provider name the second.

Every backend shares the same freshness, merge, and coverage semantics, because each delegates to the same RateCacheRules.

Important

The distributed cache is a best-effort shared performance hint, not an authoritative multi-writer store. IDistributedCache offers no atomic read-modify-write, so same-process races are guarded by a per-pair in-process lock while cross-process writes to one pair are last-write-wins (a StoreFetchedRange blob is still all-or-nothing per write, so a reader never sees coverage without its rows). When correctness under concurrent writers matters, prefer the SQLite backend (one transaction per write) or a real database.

The choice is one of reach and durability:

Backend Best for Not for Correctness note
NullRateCache tests / disabling the cache any reuse stores nothing; every lookup is a miss
InMemoryRateCache a single, long-lived process restarts; multiple processes process-local; lost on restart
TomlFileRateCache / JsonFileRateCache simple durable local cache; inspectable files high multi-process write concurrency atomic temp-and-move per file; best-effort
SqliteRateCache durable single-host cache a cache shared across hosts strongest shipped local option; one transaction per write
DistributedRateCache a warm cache shared across processes/hosts an authoritative multi-writer store last-write-wins per pair across processes

Cache backends in depth

The earlier table picks a backend by reach and durability; this section goes one level down into the semantics each one commits to, so a choice survives the move from a single process to a fleet. Every backend implements the same IRateCache contract and delegates its freshness, validity, merge, and coverage rules to the shared RateCacheRules, so they differ only in where the bytes live and how a concurrent write is ordered - never in what counts as a hit.

Null InMemory TomlFile / JsonFile Sqlite Distributed
Package core core core .Caching.Sqlite .Caching.Distributed
Scope none one process one host one host many hosts
Survives restart n/a no yes yes yes (in the backing store)
Shared across processes n/a no through the file system through the file yes
StoreFetchedRange atomicity n/a per-pair lock temp-and-move per file one transaction one blob write
Cross-process write race n/a n/a last-write-wins per file serialized rows, OS-level last-write-wins per pair
Construct Create(provider) new(provider) new(options) AddSqliteRateCache AddDistributedRateCache / AddRedisRateCache

Expiry, invalidation, and refresh are uniform. None of the backends has a private eviction clock. A row is fresh while asOf - CachedAtUtc < duration (the resolved per-provider expiry), and freshness is evaluated on every read; stale rows are pruned on the next write. There is no explicit invalidate call - a value "refreshes" by being re-fetched on a miss and merged in, latest CachedAtUtc winning per date. This is why the same rules object is shared: an expired row in SQLite and an expired row in Redis disappear at the same instant relative to their own CachedAtUtc, with no backend-specific TTL drift. A distributed backend may additionally set an absolute expiration on its blob as a storage hint, but correctness never depends on it - the freshness filter still runs on read.

Atomicity is the axis that actually matters across processes. StoreFetchedRange must persist the merged rows and the covered window together or neither, so a reader never sees coverage without its rows and reports a false range hit. Each backend honours that differently:

  • InMemoryRateCache and the file caches (TomlFileRateCache and JsonFileRateCache) serialize per-pair writes under an in-process lock; the file caches additionally write each file through a temp-and-move so a half-written file is never observed. Two processes writing the same file (for example AUDUSD.toml) are last-write-wins; under a partitioned layout each per-period file is written that same atomic way.
  • SqliteRateCache wraps each StoreFetchedRange in a single transaction over its rates and coverage tables, so the all-or-nothing guarantee holds even when several processes on the host share the database file.
  • DistributedRateCache stores the whole per-pair state as one JSON blob, so a StoreFetchedRange is all-or-nothing across processes - but because IDistributedCache has no atomic read-modify-write, the read-merge-write cycle of Store / RecordCoverage is only same-process safe; cross-process writes to a pair are last-write-wins.

When to use which. Reach for NullRateCache to disable caching in a test without changing the composition. Use InMemoryRateCache for a single long-lived service that can afford a cold start after a restart. Pick TomlFileRateCache for a durable, inspectable local cache where write concurrency is low - the per-pair files are human-readable. Prefer SqliteRateCache when several processes on one host must share a warm, correct-under-concurrency cache. Choose DistributedRateCache only when the cache must span hosts and you accept its best-effort, last-write-wins nature as a performance hint rather than an authoritative store; when correctness under concurrent writers matters across a fleet, front a real database with your own IRateCache.

SQLite: concurrency, durability, and the shared file

SqliteRateCache is the strongest shipped local backend: durable across restarts, and safe when several caches - even several processes on one host - share a single database file.

One file, many providers. Construct one cache per provider, all pointed at the same DatabaseFilePath. Every row is keyed by (provider, from_code, to_code, obs_date), so each provider's series stays partitioned with no collisions - and one cache already covers all of its provider's currency pairs, so there is never a cache per pair:

using var rba = new SqliteRateCache("RBA", "/var/cache/fx.db");
using var ofx = new SqliteRateCache("OFX", "/var/cache/fx.db");

Why it is safe under concurrency.

  • Each StoreFetchedRange writes the merged rows and the covered window in one transaction, so a reader never sees coverage without its rows.
  • Write-ahead logging (UseWriteAheadLogging, on by default) lets readers run concurrently with a writer, lifting throughput when caches or processes share the file. It creates .db-wal and .db-shm sidecar files next to the database - normal, and managed by SQLite.
  • BusyTimeout (default 5 s) is how long a connection waits for a peer's write lock before failing. The best-effort cache would otherwise swallow a contended write as a silently dropped one, so a non-zero timeout is what makes multi-writer sharing reliable.
  • Each instance holds its own keep-alive connection and per-pair in-process locks; SQLite's own file locking serialises writes across processes.

When to tune or disable. Raise BusyTimeout under heavy multi-process write contention; lower it (or TimeSpan.Zero) where a fast failure beats waiting. Disable UseWriteAheadLogging on storage that cannot honour WAL - notably some network file systems (NFS/SMB) - where the cache falls back to the default rollback journal.

Stacking providers (tiered read-through)

A CachingRateProvider is an IDatedRateProvider, and its constructor takes one as its inner source - so caching providers stack. Wrap a source in a durable cache, then wrap that in a faster cache, to build a tiered read-through where each layer is consulted in turn and only a miss falls through to the next:

flowchart TD
    L([Lookup]) --> L1["L1 · CachingRateProvider<br/>InMemoryRateCache - short expiry"]
    L1 -- miss --> L2["L2 · CachingRateProvider<br/>SqliteRateCache - long expiry"]
    L2 -- miss --> O["Origin · RbaRateProvider<br/>network source of record"]
    O -. "writes back" .-> L2
    L2 -. "writes back" .-> L1

On the way back, the fetched rate is written into L2 and then L1, so both tiers warm up; a process restart loses L1 but L2 still serves without hitting the origin.

using Bodu.Financial.ExchangeRates.Caching;

// L2 short-circuits the network; L1 short-circuits even the SQLite read.
var l2Options = new CachingRateOptions { DefaultExpiry = TimeSpan.FromDays(7) };
var l1Options = new CachingRateOptions { DefaultExpiry = TimeSpan.FromMinutes(5) };

IDatedRateProvider durable = new CachingRateProvider(
    rbaSource, new SqliteRateCache("RBA", "/var/cache/fx.db"), l2Options);

IDatedRateProvider tiered = new CachingRateProvider(
    durable, new InMemoryRateCache("RBA"), l1Options);

Two rules make a stack behave:

  • Bind every cache in the stack to the same provider name. A served rate is tagged with the serving cache's Provider, so mismatched names would mislabel the source.
  • Give the outer (faster) tier a shorter expiry than the inner (durable) tier. L1 is a hot buffer; L2 is the longer-lived store of record. Each layer's CachingRateOptions.DefaultExpiry (or per-provider override) is evaluated independently.

The L1 tier can be process-local (InMemoryRateCache) or cross-process (DistributedRateCache, e.g. Redis), and an aggregator child (below) can itself be a stack - the patterns compose freely. Each RateLookupResult.Provenance reports which backend served the request, so you can see which tier answered.

Grouping providers with the aggregator

AggregatingRateProvider groups several named children behind one entry point and resolves each request through a strategy. Build the children (typically each wrapped in its own cache), then group them:

var rba = new CachingRateProvider(
    rbaSource, new TomlFileRateCache(new FileRateCacheOptions { Provider = "RBA", CacheDirectory = "/var/cache/fx" }), options);
var ecb = new CachingRateProvider(
    ecbSource, new TomlFileRateCache(new FileRateCacheOptions { Provider = "ECB", CacheDirectory = "/var/cache/fx" }), options);

IDatedRateProvider provider = new AggregatingRateProvider(
    new[]
    {
        new NamedDatedRateProvider("RBA", rba),
        new NamedDatedRateProvider("ECB", ecb),
    });

A child is just an IDatedRateProvider, so each can be a concrete source wrapped in any cache - including a SqliteRateCache, or a full stack from the previous section. Here the aggregator fronts two SQLite-cached sources sharing one database file, with AUD/USD preferring RBA and falling back to ECB:

var options = new CachingRateOptions { DefaultExpiry = TimeSpan.FromHours(24) };

IDatedRateProvider rba = new CachingRateProvider(
    rbaSource, new SqliteRateCache("RBA", "/var/cache/fx.db"), options);
IDatedRateProvider ecb = new CachingRateProvider(
    ecbSource, new SqliteRateCache("ECB", "/var/cache/fx.db"), options);

var aggregation = new RateAggregationOptions();
aggregation.Routes[new CurrencyPair(CurrencyCode.AUD, CurrencyCode.USD)] = new CurrencyPairRoute(new[] { "RBA", "ECB" });

IDatedRateProvider provider = new AggregatingRateProvider(
    new[]
    {
        new NamedDatedRateProvider("RBA", rba),
        new NamedDatedRateProvider("ECB", ecb),
    },
    aggregation);

Strategies

The combination is a pluggable IRateAggregationStrategy:

  • PriorityFallbackStrategy (the default) returns the first child that resolves - the successor to the former CompositeDatedExchangeRateProvider.
  • AverageStrategy returns the arithmetic mean of every child that resolves, tagged with a synthetic provider label (Average by default). The mean is an analytical, composite value - it can equal a rate no source actually published - so it suits smoothing or cross-source comparison rather than an authoritative observation; for tax, accounting, or audit use prefer a single source (priority or per-pair routing).
  • Implement the interface for anything else (weighted, median, first-non-stale).
var options = new RateAggregationOptions { DefaultStrategy = new AverageStrategy() };

Per-FX-pair routing

RateAggregationOptions.Routes maps a pair to an ordered child list and an optional pair-specific strategy, so each pair can prefer a different source - AUD/USD via [RBA, ECB] while USD/GBP prefers [ECB, RBA]:

var aggregation = new RateAggregationOptions();
aggregation.Routes[new CurrencyPair(CurrencyCode.AUD, CurrencyCode.USD)] = new CurrencyPairRoute(new[] { "RBA", "ECB" });
aggregation.Routes[new CurrencyPair(CurrencyCode.USD, CurrencyCode.GBP)] = new CurrencyPairRoute(new[] { "ECB", "RBA" });
aggregation.Routes[new CurrencyPair(CurrencyCode.EUR, CurrencyCode.USD)] = new CurrencyPairRoute(new[] { "ECB", "RBA" }, new AverageStrategy());

var provider = new AggregatingRateProvider(children, aggregation);

flowchart TD
    Q["GetRate(pair)"] --> AGG{"AggregatingRateProvider<br/>route by FX pair"}
    AGG -- "AUD/USD" --> P1["RBA, fallback ECB"]
    AGG -- "USD/GBP" --> P2["ECB, fallback RBA"]
    AGG -- "EUR/USD" --> P3["ECB + RBA, AverageStrategy"]
    AGG -- "unrouted" --> PD["DefaultProviderOrder + DefaultStrategy"]

A pair without a route uses DefaultProviderOrder (or the supplied child order) and DefaultStrategy. When inversion is allowed, an inverse-pair route is also consulted.

Reaching a specific source

The lookup methods always apply the configured strategy and routing. When you need one source's answer specifically, resolve it by name - without bypassing the contract:

if (((AggregatingRateProvider)provider).TryGetProvider("RBA", out IDatedRateProvider rbaOnly))
{
    RateLookupResult rbaRate = rbaOnly.GetRate("AUD", "USD", new DateOnly(2024, 1, 3));
}

Under dependency injection the same access is available through a keyed service (below).

When to use which: single cache, stacking, or aggregation

These three compositions answer different questions and combine freely - pick by what you are trying to improve:

Composition Shape Use it to Reach for when
Single cache one source → one cache avoid re-fetching one source a single provider and one store is enough
Stacking (tiered read-through) one source → cache over cache cut latency and survive restarts on one source a hot in-memory (or shared) tier in front of a durable SQLite tier, both over the same source
Aggregation many sources → one entry point get resilience and coverage across different sources fallback when a source is down, an averaged rate, or per-pair routing to the best source

The distinction that matters: stacking layers caches over a single source (the layers differ in speed and durability, not in where the rate comes from), while aggregation combines distinct sources (the children differ in who published the rate). They are orthogonal and nest: an aggregator child can be a stacked, SQLite-then-memory cached source, so a fleet can route AUD/USD to a fast-but-durable RBA stack and fall back to an ECB stack.

A quick decision path:

  • One source, one process, restarts acceptable → single cache with InMemoryRateCache.
  • One source, restarts must stay warm → single cache with SqliteRateCache, or stack memory over SQLite to also cut the per-lookup read cost.
  • Several sources, want fallback / averaging / per-pair preference → aggregation, each child cached (and optionally stacked) as above.

For where the bytes live within any one cache layer, see the backend decision table under Persistent and shared backends.

Dependency injection

The Bodu.Financial.ExchangeRates.Caching package ships its own DI registration (there is no separate *.DependencyInjection package); its extension methods register either shape on the IFinancialServiceBuilder and live in the Bodu.Financial.ExchangeRates namespace, so one using brings them into scope. Both resolve as the dated and timeless surfaces.

A single cached provider:

using Bodu.Financial;
using Bodu.Financial.ExchangeRates;
using Microsoft.Extensions.DependencyInjection;

services.AddFinancialService()
        .AddRbaExchangeRates()
        .AddCachedRateProvider<RbaRateProvider>("RBA",
            configure: o => o.DefaultExpiry = TimeSpan.FromHours(12));

A group of cached providers with per-pair routing. Each child is also registered as a keyed IDatedRateProvider, so a specific source is resolvable by name:

services.AddFinancialService()
        .AddRbaExchangeRates()
        .AddEcbExchangeRates()
        .AddAggregatedRateProvider(agg => agg
            .AddCachedChild<RbaRateProvider>("RBA")
            .AddCachedChild<EcbRateProvider>("ECB")
            .MapPair(new CurrencyPair(CurrencyCode.AUD, CurrencyCode.USD), "RBA", "ECB")
            .MapPair(new CurrencyPair(CurrencyCode.USD, CurrencyCode.GBP), "ECB", "RBA"));

// Later: the aggregate, or a specific source.
var aggregate = provider.GetRequiredService<IDatedRateProvider>();
var rbaOnly = provider.GetRequiredKeyedService<IDatedRateProvider>("RBA");

UseDefaultStrategy(...) overrides the default PriorityFallbackStrategy, and MapPair(pair, strategy, order) overrides the strategy for a single pair. Bind CachingRateOptions from configuration by passing an IConfiguration (default section Financial:RateCache).

How staleness works

  • A cached row is fresh while asOf - CachedAtUtc < duration, where duration is the provider's resolved expiry. Single-date serving filters per row.
  • A write merges new rows with existing ones (latest CachedAtUtc wins per date) and prunes rows that are no longer fresh, so the store self-cleans over time.
  • Range serving is decided by recorded coverage, not by the rows: a range is served only when the still-fresh coverage windows contain the whole request, and a range fetch writes its rows and covered window together through StoreFetchedRange. Coverage windows expire on the same duration and are pruned on write.

Stampede protection: jitter and refresh-ahead

A cache that expires is a cache that occasionally makes one caller pay the full origin fetch. Two opt-in options on CachingRateOptions smooth that cliff; both default to 0 (off), so existing behaviour is unchanged until you opt in:

  • ExpiryJitter deterministically shaves up to that fraction off each pair's effective expiry, keyed by a stable hash of the provider and pair - never randomness - so pairs warmed together do not all expire (and refetch) at the same instant. Jitter only ever shortens an expiry; no rate is served longer than the configured duration.
  • RefreshAheadFraction turns an aged hit into a stale-while-revalidate serve: when a hit's served data is older than this fraction of the pair's effective expiry, the caller is still served instantly from the cache and one background refresh of the pair (or range window) is scheduled. Concurrent aged hits join the pending refresh rather than duplicating it, and because the fraction is below 1, a continuously hot pair is renewed before it can ever miss. A failing refresh is logged (EventId 4517) and swallowed - the next aged hit simply retries.
var options = new CachingRateOptions
{
    DefaultExpiry = TimeSpan.FromHours(12),
    ExpiryJitter = 0.1,           // spread expiries by up to 10% per pair
    RefreshAheadFraction = 0.75,  // refresh in the background after 75% of the expiry
};
options.Validate();

Two more knobs matter at scale:

  • SkipInverseRangeProbeWhenDirectCovered (default false): a range miss normally probes the inverse pair as a second whole-state backend read. Workloads that only ever fetch one orientation can enable this to skip the guaranteed-empty probe whenever the direct pair holds any fresh coverage.
  • EntryExpirationMargin on DistributedRateCacheOptions (default one hour): every blob the distributed cache writes is stamped with a server-side lifetime of the caching duration plus this margin, so a pair that stops being queried self-evicts from Redis instead of accumulating forever. The lifetime is relative to the store's own clock, so clock skew cannot evict entries prematurely; set it to null to disable server-side expiry entirely.
var distributedOptions = new DistributedRateCacheOptions
{
    Provider = "RBA",
    EntryExpirationMargin = TimeSpan.FromHours(2),
};

Warming the cache at startup

A cold cache pays its origin fetches on the first user requests. The decorator's WarmAsync pre-pays them: each pair's window goes through the normal range read-through path - an already covered pair costs only a cache read, a miss populates rows and coverage - with up to four pairs fetched concurrently. A failing pair is logged (EventId 4518) and skipped; only the caller's cancellation aborts the run.

var provider = new CachingRateProvider(
    new FixedDatedRateProvider(Array.Empty<ExchangeRate>()),
    new InMemoryRateCache("RBA"),
    new CachingRateOptions());

int warmed = await provider.WarmAsync(
    new[] { ("AUD", "USD"), ("AUD", "EUR") },
    new DateOnly(2026, 5, 1),
    new DateOnly(2026, 5, 31));

Under dependency injection, register the warm-up as a hosted service instead and let it run when the application starts. It warms every provider registered through AddCachedRateProvider automatically; aggregation children are keyed services and are warmed only when named in Providers. The window defaults to a rolling LookbackDays look-back from the current date (so a daily restart always warms recent history), with optional fixed StartDate/EndDate overrides; the run never blocks or crashes the host:

services.AddFinancialService()
        .AddRbaExchangeRates(configuration)
        .AddCachedRateProvider<RbaRateProvider>("RBA", configuration)
        .AddRateCacheWarmup(configure: warmup =>
        {
            warmup.Pairs.Add("AUD/USD");
            warmup.Pairs.Add("AUD/EUR");
            warmup.LookbackDays = 90;
        });

Or bind the same options from configuration (section Financial:RateCacheWarmup):

{
  "Financial": {
    "RateCacheWarmup": {
      "Pairs": [ "AUD/USD", "AUD/EUR" ],
      "LookbackDays": 90
    }
  }
}

Observability: seeing hits, misses, and degradation

Two log channels tell you what the cache is doing.

Caching-decorator events. CachingRateProvider logs each hit, miss, refetch, and a served-rate provenance record, at levels you set on CachingRateOptions:

Event Default level Option
Single-date hit Trace CacheHitLogLevel
Single-date miss (resolved and cached) Trace CacheMissLogLevel
Range hit Debug CacheRangeHitLogLevel
Range refetch Debug CacheRangeRefetchLogLevel
Served-rate provenance Debug RateProvenanceLogLevel

SQLite degradation. The SQLite cache is best-effort: a storage failure degrades to an empty read or a skipped write rather than throwing. So the degradation is not silent, SqliteRateCache logs each swallowed failure at Warning under EventId 4520 - naming the provider and the failing operation and attaching the exception. The first failure is logged immediately; further failures are rate-limited to at most one warning per minute, each carrying the count suppressed since the previous warning, so a sustained outage is visible without flooding the log.

Under dependency injection the logger is wired automatically. Constructing by hand, pass one to the constructor:

var cache = new SqliteRateCache(
    new SqliteRateCacheOptions { Provider = "RBA", DatabaseFilePath = "/var/cache/fx.db" },
    timeProvider: null,
    logger: loggerFactory.CreateLogger<SqliteRateCache>());

Surface it with an ordinary logging filter - the category is the cache's full type name:

{
  "Logging": {
    "LogLevel": {
      "Bodu.Financial.ExchangeRates.Caching.SqliteRateCache": "Warning"
    }
  }
}

If you would rather a storage failure surface as an exception than degrade, set ThrowOnStorageFailure (or ValidateStorageOnStart to fail fast at startup) on the options instead.

Metrics

Alongside the logs, the caching layer publishes System.Diagnostics.Metrics counters through the process-wide meter Bodu.Financial.ExchangeRates.Caching (the SQLite and distributed add-ons publish their storage-failure counters through their own meters, …Caching.Sqlite and …Caching.Distributed). With no listener attached a counter add is a no-op branch, so the instrumentation costs nothing on the hot path:

Instrument Tags Counts
bodu.financial.rate_cache.hits provider, lookup (single/range) Lookups served from the cache
bodu.financial.rate_cache.misses provider, lookup Lookups refetched from the source
bodu.financial.rate_cache.refresh_ahead provider, outcome (success/failed) Background refresh-ahead attempts
bodu.financial.rate_cache.storage_failures provider, operation Swallowed best-effort storage failures

Storage-failure counts increment on every swallow - deliberately outside the rate-limited warning gate that throttles log volume - so sustained degradation is quantifiable even while its logging is suppressed. Subscribe with any OpenTelemetry metrics exporter, or with an in-process MeterListener:

using var listener = new System.Diagnostics.Metrics.MeterListener
{
    InstrumentPublished = (instrument, l) =>
    {
        if (instrument.Meter.Name == "Bodu.Financial.ExchangeRates.Caching")
            l.EnableMeasurementEvents(instrument);
    },
};
listener.SetMeasurementEventCallback<long>((instrument, value, tags, state) =>
    Console.WriteLine($"{instrument.Name} +{value}"));
listener.Start();

Log event reference

Every caching-layer log message carries a stable EventId, so filters and alerts can target a specific event:

EventId Event Level
4501 / 4502 Single-date hit / miss-and-stored option-set
4503 / 4504 Range hit / range refetched option-set
4505 Served-rate provenance option-set
4506-4508 Advertised-history skip / empty-window skip / clamped fetch option-set
4510-4512 Aggregation routing and outcomes option-set
4513-4515 File-cache degradation and corrupt-file warnings Warning
4516 / 4517 Refresh-ahead scheduled / refresh-ahead failed option-set / Warning
4518 Warm-up pair failed and skipped Warning
4520 SQLite storage failure swallowed Warning
4521-4524 Startup warm-up started / completed / failed / provider not found Information / Warning
4530 Distributed storage failure swallowed Warning

Troubleshooting

The cache is cold after every restart. Only in-memory caches lose state on restart. Use a durable backend (SqliteRateCache or a file cache) to survive restarts, and dispose the cache so its keep-alive connection is released cleanly. A tiered stack keeps a hot in-memory L1 and a durable L2 - see Stacking providers.

"database is locked". Several writers are contending for one file. Keep UseWriteAheadLogging on (it lets readers run during a writer) and raise BusyTimeout so a writer waits for the lock instead of failing; confirm every cache over the file uses WAL and that the storage supports it - see SQLite concurrency.

I can't tell whether the cache is degrading. Wire a logger and watch for EventId 4520 at Warning - see Observability. The message names the failing operation and carries the exception, and the suppressed count tells you the failure is sustained rather than a one-off. To make failures loud instead, set ThrowOnStorageFailure.

Stray .db-wal / .db-shm files. These are SQLite's write-ahead-log sidecars, created next to the database when WAL is on. They are normal; leave them for SQLite to manage, and copy them alongside the .db file if you move the database.

Which backend, and when to stack vs. aggregate? See the backend decision table and single cache vs. stacking vs. aggregation.

See also