Skip to main contentSkip to data table
Pharos

Data Pipeline

Stablecoin data pipeline guide for price enrichment, source priority, freshness checks, integrity guardrails, fallback behavior, and sync cadence.

Agent navigation — Grep the heading you need instead of reading wholesale: Supply Pipeline · Price Enrichment Pipeline · Data Integrity Guardrails · Gold & Silver Spot Prices · Stability Index (PSI) Computation · Pending Depeg Confirmation · Stale Data Monitoring (Frontend) · Blacklist Sync State Semantics · Coverage Discovery.

Supply Pipeline

Supply data uses a two-source model with automatic fallback:

  • DefiLlama — primary source for all stablecoins tracked by DefiLlama's stablecoin API
  • CoinGecko market cap — used for gold/silver/fiat tokens that DefiLlama doesn't track (e.g. XAUT, PAXG, KAU), and as a full supply fallback when the DefiLlama stablecoins API is down (circuit breaker triggers syncViaCoingeckoFallback())

Manual supply corrections, CMC supply patches, and open-ended on-chain overrides remain disallowed. Curated on-chain reads are code-reviewed fallback paths with explicit asset scope and fail-closed behavior.

V9-only bridge attribution runs on the dedicated +8 expression. The V9 fixed input is then prepared immediately after successful half-hourly DEX publication, carrying that exact DEX generation ID, and canonical compilation runs at +22 and +52. The compiler rejects an input when its DEX dependency is older than the latest accepted DEX generation. The isolated attribution and compilation lanes acquire a D1-backed memory-lane lease, bind to the immutable current Worker version ID and matching finished core slot, decline admission while an earlier scheduled slot is still active, and enforce absolute deadlines. An ok core slot admits directly. A degraded slot admits only when the durable publication ledger proves that the same Worker published the current stablecoins cache during that slot, and the compiler separately requires the fixed input's stablecoin timestamp to match the live cache generation. This keeps partial active-row coverage visible and asset-local without admitting stale or no-write stablecoin data. Delayed or competing deliveries skip neutrally; missing Worker-version metadata degrades visibly. The critical quarter-hour production lane is stablecoins → DDR; the atomic report-card publication that feeds V9 runs separately, on the half-hourly 16,46 chart slot (prepare-safety-score-v9-input, after DEX publication). That atomic publication includes a compact, publication-exact V9 peg-provenance seed; the later V9 compiler rejects missing, partial, or identity-mismatched seeds rather than reconstructing provenance from mutable event rows.

XAUT has a bounded, V9-only lock/mint attribution for its otherwise aggregate-only upstream row. Inside the isolated producer, the XAUT observer first reads the reviewed https://app.tether.to/transparency.json source. It requires exactly one XAUT Ethereum row, captures the raw response hash and issuer timestamp, converts totalAuthorized, notIssued, and quarantined to exact six-decimal raw units, rejects future or older-than-48-hour disclosures, and requires quarantined supply to be zero. It then selects one finalized Ethereum block at or before the scoring clock and reads canonical totalSupply(), the pinned Tether treasury and official XAUt0 OFT adapter balanceOf() values, and the adapter's token/LayerZero endpoint identity in one Multicall. The disclosure's authorized amount must equal finalized total supply and its not-issued amount must equal the finalized treasury balance. V9 therefore divides the locked adapter balance by circulating liabilities (totalAuthorized - notIssued), not by minted ERC-20 supply. It hash-binds and confirms the block, verifies canonical token and adapter proxy/implementation identities, and binds the result to the exact reviewed XAUt0 representationId route inventory. The non-group circulating liability is attributed to Ethereum and the adapter balance becomes one reviewed XAUt0 representation-group row. The group carries the exact aggregate share and common lockbox/protocol failure domains, but no destination-chain allocation; individual destination routes never receive inferred zero shares. Packet observation time is the later of the confirmed block and issuer disclosure timestamps, while validation independently ages both inputs. The partition conserves the upstream USD liability exactly and never sums destination representations on top of their locked backing. Missing, stale, skewed, identity-drifted, disclosure/on-chain-mismatched, inventory-drifted, non-conserving, or materially large pooled evidence fails closed to aggregate-only bridge materiality. Its admission journal truthfully identifies the source as issuer-disclosure-plus-on-chain, records an allowlisted exact leaf rejection code, and retains the rejected source timestamp for disclosure skew/staleness or stale finalized state without persisting response bodies, URLs, or free-form diagnostics.

The reviewed deployment-unit path also covers the explicitly allowlisted Centrifuge V3 JTRSY and ACRDX inventories. Those assets qualify because the protocol burns source-chain shares before hub-authorized destination minting, so complete deployment totalSupply() units form one non-duplicative partition; lock-and-mint and adapter inventories are not eligible for this path. Every official deployment must be present: JTRSY binds seven EVM contracts plus its Solana mint, while ACRDX binds four EVM contracts plus its Solana mint. Each EVM observation uses a safely lagged, hash-bound block at or before the scoring clock and atomically reads totalSupply(), decimals, and the pinned Centrifuge Spoke ward while also pinning runtime bytecode and requiring an empty EIP-1967 implementation slot. The Solana observation binds the Token-2022 mint and exact direct authority from one finalized account context and verifies the authority is a non-executable System-owned account. The complete packet must stay inside the 120-second cross-chain envelope, be no older than 30 minutes, and conserve the existing aggregate USD liability exactly. Any unavailable route, post-clock observation, code/controller/proxy drift, inventory mismatch, skew, or reconciliation failure rejects the whole packet and retains aggregate-only bridge materiality.

For tracked supplemental assets that are not in DefiLlama's stablecoin list, the worker still prefers DefiLlama's coins.llama.fi price proxy when it exists, including coingecko:{id} rows and exact chain:contract rows for single-deployment assets without a CoinGecko ID, then falls back to CoinGecko simple/price for the current token price when DefiLlama omits that geckoId, including protocol-backed commodity tokens that also carry a DefiLlama protocolSlug. Exact chain:contract supplemental rows are accepted only when DefiLlama returns a matching symbol, confidence of at least 0.8, a fresh upstream timestamp, and a price inside the shared peg-aware reasonableness bounds. Gold tokens also fall back to CoinGecko market cap when a configured DefiLlama protocolSlug returns TVL history but no usable mcap, preventing zero-supply rows or dropped rows for otherwise healthy commodity assets. For detailProvider === "coingecko" fiat assets, the preferred admission path is still CoinGecko market cap, but tracked assets can also enter the cached /api/stablecoins payload through a runtime-supported on-chain total-supply fallback: either exactly one supported deployment, a curated single-chain override, a curated aggregate where every configured chain can be read, or the Zephyr Scanner exception. Curated aggregate legs may explicitly allow reviewed zero-supply native deployments to contribute zero, but unreadable configured chains still fail the whole aggregate closed. The curated apyUSD aggregate sums its reviewed Ethereum and Base CCIP burn/mint deployments so a current per-chain materiality split accompanies the CoinGecko-priced NAV supply. The curated yUSD aggregate sums its reviewed Ethereum native deployment and nine LayerZero OFT burn/mint representations, because no leg escrows another. The curated savUSD aggregate reads its ten reviewed Chainlink CCIP deployments but reallocates the canonical Avalanche vault total, because the Avalanche CCIP LockRelease pool escrows every destination-chain mint, and its reviewed zero/dust legs on Katana, BSC, and MegaETH may contribute zero. Both aggregates pin reviewed public RPC endpoints for the chains outside the worker chain registry, and any unreadable leg still fails the whole aggregate closed onto the upstream CoinGecko market cap. Curated sUSDS and sDAI aggregates treat Ethereum totalSupply() as the conserved global total because it includes shares escrowed for their canonical lock/mint representations; Base, Optimism, and Arbitrum observations are reallocated out of the Ethereum chain bucket rather than added to that total, and an impossible representation sum fails closed. The curated sUSDe aggregate applies the same reallocation rule to its Ethereum LayerZero OFT adapter escrow across the twenty-four reviewed deployments a supported runtime can read; the TON jetton and Aptos fungible asset have no supply probe, so they are not configured legs and their balances stay inside the Ethereum bucket instead of failing the aggregate closed. The curated wsrUSD aggregate reallocates the same way out of its Ethereum OFT Adapter lockbox across seventeen reviewed deployments, and the curated srUSD, syrupUSDT, KRWQ, thBILL, and wiTRY aggregates each reallocate out of their own reviewed Ethereum lockbox — a LayerZero OFT Adapter for srUSD, KRWQ, and thBILL, a Chainlink CCIP LockRelease pool for syrupUSDT, and a protocol escrow contract for wiTRY. thBILL follows the sUSDe rule for its untracked Stable-chain representation, whose balance stays inside the Ethereum bucket rather than failing the aggregate closed. The curated cUSDO, sUSDai, sYUSD, IAUon, SLVon, mHYPER, and sDOLA aggregates sum instead, because each remote leg is either a locally backed vault or a burn/mint representation that no canonical leg escrows; their reviewed zero-supply and dust legs may contribute zero. The curated USDK and XO entries are single Solana deployments configured only so the aggregate lane publishes a per-chain row: their reviewed lock/mint routes escrow the underlying M0 $M, never the tracked token, so the published aggregate is unchanged. For active DefiLlama-listed rows that collapse to zero supply, the worker can repair only the curated CADD and Mento JPY/PHP/ZAR/XOF deployments from verified on-chain total-supply reads, and only when every configured chain read succeeds and a fresh/static FX reference exists for USD normalization. A narrow protocol-inventory variant can subtract configured non-circulating holder balances from the same live on-chain total supply read and tags the row supplySource = "onchain-circulating-supply"; it is currently used for Tangent USG PegKeeper balances and fails closed if the balance reads are unavailable. The same configured exclusion can be replayed by POST /api/backfill-supply-history from historical EVM totalSupply() and holder balanceOf() reads, so daily supply_history rows and chart overlays use the same circulating-supply rule as the live cache. Zephyr assets are a narrow protocol-native exception: zsd-zephyr-protocol and zys-zephyr-protocol use Zephyr Scanner live-stats for native-chain circulation, and ZYS uses the same payload's protocol-published share price because neither CoinGecko nor DefiLlama exposes that wrapper.

If the supplemental CoinGecko market-cap fetch is temporarily unavailable, syncStablecoins() now reuses the last known good cached supply snapshot for those supplemental assets instead of emitting zero-supply rows or dropping them from the payload. That preservation rule now covers all tracked detailProvider === "coingecko" assets, including ones that currently rely on on-chain supply fallback without a geckoId. When a fresh DefiLlama coins.llama.fi price is still available, that fresher price is merged onto the restored supply snapshot. Carry-forward is bounded: restores preserve the original supplyObservedAt, and once that observation is older than 7 days (SUPPLEMENTAL_RESTORE_MAX_AGE_SEC) the restore expires — the asset publishes with its real (empty) supply and the run logs the expired IDs instead of indefinitely re-publishing stale totals. Restored rows are flagged supplyRestored with supplyObservedAt provenance, and the coin detail hero renders a "Stale supply · as of {date}" note from those fields.

The same restore-or-degrade rule guards tracked-id coverage of the main DefiLlama list itself: when the list omits an active tracked coin that was published last cycle, restoreMissingTrackedAssets() re-publishes the previous row (marked supplyRestored, same 7-day ceiling) instead of silently dropping the coin from the payload for a cycle, and the run records restored/dropped IDs in cron metadata (trackedCoverage). Past the ceiling — or with no usable previous supply — the coin stays out and is reported as dropped.

The 2026-07-10 Night Watch supply audit restored rusd-royal-dollar to that primary path by mapping the live DefiLlama Royal Dollar row (llamaId = 415). Eight other assets remained absent from the DefiLlama stablecoins list while CoinGecko reported no positive market cap: benji-franklin-templeton, wtgxx-wisdomtree, busd0-usual, tbill-openeden, cetes-etherfuse, jusd-jusd-stable-token, vndc-jade-labs, and sofid-sofi. The 2026-07-11 follow-up added gramg-token-teknoloji and grams-token-teknoloji, whose permitted sources likewise report no positive circulating market cap. Those ten reviewed no-supply records are quarantined and therefore outside the active publication contract; they have no default publication waivers. The 2026-07-12 review moved AUDm, CADm, CHFm, COPm, GBPm, and ZARm to CoinGecko detail admission because their mapped DefiLlama rows reported explicit zero supply and the permitted fallback path had positive coverage. ZARm can retain its existing curated on-chain supply source inside that admission path. XOFm remains DefiLlama-backed and uses its existing curated Celo total-supply repair; zero-supply collapse candidates are processed before non-blocking chain gaps so that repair cannot be starved by the 15-candidate cap.

The aggregate stablecoin-charts cache now reconciles structural supplemental tracked assets back into the published total-market-cap history instead of relying on DefiLlama's aggregate chart feed alone. syncStablecoinCharts() still starts from stablecoincharts/all, but after the FX repair pass it overlays only the tracked non-DefiLlama cohort with no llamaId from D1 supply_history before downsampling and cache publication. BRZ is the narrow audited exception because its retained legacy DefiLlama ID has no chart rows. A CoinGecko-admitted live row with a populated DefiLlama chart identity is therefore not double-counted. GET /api/stablecoin-charts then appends or replaces the trailing point with a live aggregate built from the current stablecoins cache so the chart endpoint's latest value matches the homepage KPI card.

Circuit Breakers

Most high-risk external integrations are protected by per-source circuit breakers (worker/src/lib/circuit-breaker.ts). State is persisted in the D1 cache table under keys like circuit:defillama-stablecoins. Bounded low-volume fallbacks such as gold-api.com metal spot quotes, the secondary FX mirror, and ExchangeRate-API daily reference snapshots use explicit retry/timeout/cooldown behavior but are not currently circuit-gated.

  • Open threshold: 3 consecutive failures
  • Probe interval: 30 minutes (one request allowed to test recovery)
  • Alerts: Open/close transition alerts are sent when the caller provides a webhook URL to recordOutcome(...)
  • Health impact: 3 or more open circuits degrade /api/health; smaller circuit failures still surface in the circuit list without degrading public health on their own

The source-name registry is maintained in worker/src/lib/constants.ts under CIRCUIT_SOURCE, while emitted pricing provenance and circuit semantics are tied together by shared/lib/pricing-source-registry*.ts and worker/src/lib/pricing-circuit-map.ts. Current circuit keys span the main data and delivery lanes, including DefiLlama (defillama-*; retained defillama-confirm state is legacy because pending confirmation no longer queries the CoinGecko mirror), CoinGecko (coingecko-prices, coingecko-detail-platforms, coingecko-mcap, coingecko-ticker, coingecko-confirm), CoinMarketCap, DexScreener (dexscreener-prices for exact token-address stablecoin price fallback, dexscreener-liquidity for optional DEX liquidity/discovery pool lookups, dexscreener-address-prices for optional targeted exact-address primary augmentation, and legacy dexscreener-search state for the retired symbol-search path), DexPaprika, Alchemy Prices, Moralis token prices, Birdeye token prices, GeckoTerminal pool probes, Jupiter, Pyth, Binance, Kraken, Bitstamp, Coinbase, RedStone, external live protocol-redeem RPC overrides, Curve (curve-onchain, curve-oracle, curve-liquidity-api), the protocol-native DEX lanes (Fluid, Balancer, Curve, Raydium, Orca, Meteora, PancakeSwap, Aerodrome Slipstream, Velodrome Slipstream), FX (fx-frankfurter, fx-realtime, chainlink-feeds), treasury rates, Etherscan, Alchemy, Bluechip, Anthropic, Twitter, Telegram, TronGrid, and the Kinesis Horizon sources. dRPC is an upstream RPC provider for some blacklist balance reads, but it is not a CIRCUIT_SOURCE key today. Synthesized or scoped emitted sources such as coingecko-native-implied, zephyr-scanner, dex-promoted, uniswap-v3-dex, uniswap-v3-exact, pool-tvl-weighted, and cached are explicitly marked as not directly circuit-gated. When sources depend on producer jobs, scoped supplemental fetches, local par/inherited override logic, or cached rows, that enforcement path is recorded separately from direct provider breakers.

npm run check:provider-resilience backs this posture with a registry in scripts/lib/provider-resilience-registry.mjs. It records the expected timeout, response-body handling, circuit source where applicable, and regression tests for external provider/fetcher surfaces, and it fails when a new production Worker file adds raw fetch(...) without a registry entry.

DefiLlama list vs detail API

The list endpoint (stablecoins.llama.fi/stablecoins) returns circulating values already in USD for all peg types — peggedRUB, peggedEUR, peggedJPY, etc. are all denominated in USD despite their key names.

The detail endpoint (stablecoins.llama.fi/stablecoin/{id}) returns values in native currency (e.g. RUB for A7A5, EUR for EURC). The worker's stablecoin-detail.ts handler multiplies by parsed.price to convert these to USD before caching.

Do not multiply list endpoint values by price — that would double-convert and produce wildly wrong numbers (e.g. A7A5: $508M × 0.013 = $6.6M instead of $508M).

Price Enrichment Pipeline

Primary Price Fetch

Before the enrichment pipeline runs, fetchPrimaryPrices() collects prices from multiple sources and runs N-source weighted consensus to determine the best price for each asset:

Sources (each behind its own circuit breaker):

SourceWeightModuleNotes
CoinGecko /simple/price2built-inPrimary market data
CoinGecko ticker2worker/src/lib/cg-ticker.tsCurated ticker corroboration surface for tracked exchange pairs
DefiLlama stablecoins list1built-inIndependent typed DL-list quote with explicit freshness provenance
Pyth Network Hermes2worker/src/lib/pyth.tsOracle prices with confidence intervals; coverage is driven by curated pythFeedId entries in the per-coin stablecoin metadata assets (shared/data/stablecoins/coins/*.json, loaded through shared/lib/stablecoins/registry.ts)
Binance spot tickers2worker/src/lib/cex-tickers.tsDirect CEX prices (single batch call)
Kraken spot tickers2worker/src/lib/cex-tickers.tsAlias-safe explicit pair mapping
Bitstamp spot tickers1worker/src/lib/cex-tickers.tsLower-weight all-tickers corroboration venue
Coinbase spot tickers2worker/src/lib/cex-tickers.tsDirect CEX prices (per-symbol)
RedStone oracle1worker/src/lib/redstone.tsPer-venue breakdown + agreement % for exact-case tracked symbols in REDSTONE_TRACKED_SYMBOL_ALLOWLIST, attributed only to configured canonical stablecoin IDs
Curve on-chain get_dy()3worker/src/lib/curve-onchain.tsStableSwap implied prices from explicit direct, one-hop, and opt-in chained-hop configs
Curve oracle (crvusd-curve)3worker/src/cron/sync-stablecoins/enrich-prices-primary.tsAdditional primary-consensus voice for crvUSD
DEX promoted prices1worker/src/lib/depeg-helpers.tsAggregate DEX voice when no overlapping promoted protocol-level DEX source is admitted
Promoted protocol-level DEX prices2-3worker/src/lib/depeg-helpers.tsOne aggregated source per registered protocol, including Uniswap V3/V4; freshness now preserved per source from price_sources_json
Exact-address augmentation providers1worker/src/lib/address-price-providers/index.tsTargeted DexScreener, DexPaprika, CoinGecko Onchain, Alchemy Prices, Moralis, and Solana Birdeye quotes for assets with missing prices, low confidence, or previous source depth below 3

Dead or explicitly blocked DEX ids are removed upstream from DEX crawl intake, pool scoring, challenger publication, and dex_prices publication. The current runtime blocklist includes Retro variants and Bunni variants, so those venues cannot leak into primary consensus through the DEX bridge or pool challenge.

Consensus algorithm (worker/src/lib/price-consensus.ts):

  • Collects all available source prices for each asset
  • Groups sources into agreement clusters within a configurable threshold (default 50 bps for pegged tokens, 500 bps for NAV tokens)
  • Picks the largest agreeing cluster; for 2+ source winners, publishes the cluster median and keeps the best member internally for provenance
  • If no 2+ cluster forms, picks the best trusted fallback source for publication
  • ≥2 sources agreepriceConfidence: "high"
  • Single source onlypriceConfidence: "single-source"
  • Sources disagreepriceConfidence: "low", best trusted fallback source used
  • All sources down → skip, falls through to enrichment pipeline

Cluster selection breaks ties by size, then total cluster weight, then strongest source trust tier, then tighter spread, then peg proximity, and finally a stable source label. The internal selected source inside the winning cluster is chosen by weight, trust tier, reference proximity, and finally source key, but that selected source is no longer forced to be the published high-confidence price.

Each asset gets tagged with priceConfidence (high/single-source/low/fallback) and supplySource (defillama, coingecko-fallback, onchain-total-supply, or onchain-circulating-supply). The onchain-total-supply path is used for supplemental assets whose circulating supply is derived from an on-chain total-supply probe instead of an upstream market-cap field, and for the curated DefiLlama zero-supply repairs that would otherwise publish an active asset with no market cap; onchain-circulating-supply uses the same live probe but subtracts configured protocol inventory balances before USD normalization. Preview-only fiat CoinGecko assets can use those paths with the existing FX reference for USD normalization while still keeping price = null. Solana total-supply fallback now reuses the same shared multi-endpoint probe used by the reserve-adapter path, so supplemental Solana assets do not depend on a narrower RPC list than the rest of the worker.

Consensus source provenance

After N-source consensus, each asset receives a consensusSources: string[] field listing all source names that returned a valid price for that coin during the sync cycle. For enrichment-pass fallbacks, this is a single-element array. Direct protocol-redeem overrides and high-confidence inherited overrides replace it with ["protocol-redeem"]; scoped inheritance from a fresh replay-safe single-source parent instead keeps the parent's single source so publication guardrails retain the soft upstream provenance.

Provider-Specific Normalization

Primary pricing also includes a few source-specific normalization rules that are easy to miss when reading the high-level algorithm:

  • Pyth Hermes feed IDs are normalized to lowercase with any leading 0x stripped before matching back to tracked assets. Hermes can return feed IDs in either form.
  • Pyth confidence weighting now degrades smoothly as confidence intervals widen instead of dropping medium-confidence quotes abruptly.
  • Coinbase uses uppercased product symbols.
  • RedStone uses exact-case tracked symbols only. The worker filters requests through REDSTONE_TRACKED_SYMBOL_ALLOWLIST, sends them in sequential batches of 10, retries any batch-dropped symbol individually once, and keys usable quotes by the configured canonical stablecoin id rather than by bare symbol.
  • RedStone admission now requires at least 2 venues and at least 60% venue agreement before the quote can enter primary consensus.
  • Breaker accounting for sparse responses is data-aware: Pyth and RedStone only count as successful breaker outcomes when they return at least one usable price, while Jupiter treats documented V3 sparse no-quote rows as healthy empty coverage because they indicate provider reachability but no usable quote for that mint.
  • CEX freshness semantics are explicit: Binance and Kraken use local-fetch observation times; Bitstamp and Coinbase publish upstream-observed timestamps when the upstream response provides them. Registry metadata records whether each feed is last-trade-only or exposes bid/ask-style spot data.
  • Exact-address augmentation only queries canonical chain+address deployments from asset.address, contracts, or tradedContracts for assets with missing prices, low confidence, previous active-price coverage misses, previous source depth below 3, or an accepted observation that expires before the next generation. Its recovery scheduler treats a current price as already publishable only when the price is positive and carries a concrete source plus observation-time provenance; bare numeric prices and missing / unknown source markers stay in the missing cohort. Alert-eligible persistent active gaps are pinned first, followed by current missing rows, recently missing rows, expiring observations, priced rows with at most two previous sources, and remaining eligible priced rows. Durable fairness cursors rotate only inside each cohort, so a cursor cannot move breadth-oriented priced work ahead of unresolved active assets. Skipped DexPaprika and DexScreener targets are reported when request caps are hit. Birdeye stops its Solana target tail on the first 429 or provider-wide quota/compute-unit exhaustion response. These sources are soft, non-replay-safe, and non-depeg-authoritative; symbol search is retired.
  • Pancake V3 orientation and DEX persistence: DexApiPool.price is token0 per token1, which Pancake's subgraph exposes as token1Price. After duplicate collapse, every DEX observation must pass the peg-aware plausibility validator before it can participate in aggregate selection or keep a dex_prices row alive. This rejects inverse or decimal-broken records before D1 persistence rather than relying only on final publication validation.

These rules live in the named provider modules plus worker/src/cron/sync-stablecoins/enrich-prices-primary.ts, worker/src/lib/address-price-providers/index.ts, worker/src/cron/dex-liquidity/fetch-pancakeswap.ts, and worker/src/cron/dex-liquidity/scoring.ts.

Authoritative Price Source Registry

After the CG/DL primary pass is applied, syncStablecoins() can still replace market-derived prices for specific redeemable assets when a shared authoritative-price provider exposes a better executable mark than secondary-market liquidity.

The registry lives under worker/src/lib/authoritative-price-sources/ and supports two capabilities:

  • Live override — used by syncStablecoins() to replace the current cached price

  • Historical replay — used by backfill-depegs.ts so historical rebuilds can consult the same authoritative provider instead of drifting back to CoinGecko/DefiLlama for those assets

  • Current scope: see Pricing Pipeline for the asset-by-asset registry. The current code covers direct redeem quotes, scoped redemption-par references, tracked-base inheritance, fee-adjusted tracked-base inheritance, ERC-4626 NAV wrappers, Aave previewRedeem, Idle CDO virtual-price tranches, Kava USDX oracle state, the exact thin AZND Curve route, and the funded public Citrea JUSD bridge path. crvusd-curve was migrated out of the authoritative override registry and into primary consensus as a curve-oracle source at weight 3.

  • Source: direct eth_call redemption/NAV quotes or tracked-base inheritance when a redeemable wrapper should shadow another tracked asset:

    • Cap getBurnAmount(address,uint256) for cUSD -> USDC
    • infiniFi RedeemController.receiptToAsset(uint256) for iUSD -> USDC
    • USDAI inherits the tracked PYUSD live price and historical market replay because the base token is treated as an instantly redeemable PYUSD wrapper rather than a free-floating market-priced asset
    • Initia iUSD and Movement USDCx inherit their tracked parent prices; M, USDK, XO, USDN, and USDnr inherit tracked M0-unit pricing because Pharos models them as M0 units or extension units rather than independently discovered secondary-market price surfaces
    • WEUSD inherits the tracked USDC live price and historical replay with the documented 1% redemption-fee haircut
    • Direct-redeem rows such as SOFID, USBD, USDQ, CHFAU, CADD, JPYm, ZARm, and XOFm can publish protocol-redeem parity when active supply is observable; non-USD live parity requires a fresh/static FX reference and falls back to normal market/native-peg history until historical FX replay exists
    • ERC-4626, Aave savings, and Idle CDO wrappers read the contract's asset-per-share value and multiply it by a trusted tracked parent price; ERC-4626 NAV wrappers are prioritized ahead of lower-priority RPC-backed override families inside the live override budget
  • Deterministic route admission: each asset-specific adapter pins its market, bridge, vault, exchange, tokens, decimals, and dependencies. It then requires fresh protocol or block state plus route-specific capacity, executable-notional, impact, agreement, or public-redemption checks. Missing dependencies or any identity, freshness, depth, code-hash, capacity, transport, or quote failure returns no override. Thin fallback routes such as AZND remain non-replay-safe, non-depeg-authoritative, and subject to soft-source corroboration guardrails by themselves.

  • Scheduling: the 10-second live-override stage processes all current missing-price candidates before already-priced candidates and interleaves provider families within each partition. The scheduler uses the same publishable-price definition as exact-address augmentation, so numeric values that would not survive publication metadata checks remain eligible for missing-only recovery routes such as AZND. Each started candidate also has a bounded fairness cap inside the shared stage budget, with longer caps only on heavier audited routes. This keeps one slow provider call from skipping the remaining active recovery candidates.

  • Circuit isolation: Kava USDX, Citrea JUSD, and thin AZND Curve use kava-pricefeed, jusd-citrea-bridge, and aznd-curve-pool circuits rather than sharing one grouped protocol-redeem circuit. A failure burst in one specialty route therefore does not suppress unrelated authoritative providers. These single-asset breakers stay visible in admin provider diagnostics but are excluded from the source-wide public circuit count; exact active-price coverage reports any missing asset output. The retired mento-broker and usx-stable-pools cache keys are filtered from active circuit diagnostics.

  • Reason: CG/DL can overweight thin secondary-market liquidity for wrapper-style assets whose real executable value is set by direct protocol redemption or by an instantly redeemable base asset

  • Result: the final cached asset keeps priceSource = "protocol-redeem" and priceConfidence = "high" when a direct protocol/NAV quote or high-confidence inherited parent validates against peg bounds. Scoped inherited prices from fresh replay-safe single-source parents keep the parent source and single-source confidence so they do not bypass weak-source publication guardrails.

Enrichment Pipeline

enrichMissingPrices() in worker/src/cron/sync-stablecoins/enrich-prices.ts now delegates to the ordered fallback-pass manifest in worker/src/cron/sync-stablecoins/enrich-prices-fallback.ts for assets still missing prices after primary fetch. The orchestration is centralized in one pass list instead of one ad hoc block per provider:

  1. Pass 1: Canonical tracked contract identity -> DefiLlama coins API, but only quotes that pass peg-aware validation can claim the asset. Coin IDs are encoded as one URL path segment so slash-bearing identifiers such as Osmosis IBC denoms cannot invalidate the whole batch.
  2. Pass 1b: Tracked alternate deployment fallback (tries exact tracked deployment ids via DefiLlama coins API under the same validation gate)
  3. Pass 2: CoinMarketCap category batch (cryptocurrency/category?id=604f2753ebccdd50cd175fc1&limit=300&convert=USD), followed when needed by one v3/cryptocurrency/quotes/latest request for at most 25 rotated unresolved configured slugs. A truncated category response records the unseen tail but ignores category rows, so unresolved configured slugs must pass the exact targeted quote lane instead of publishing from a partial category page. The targeted path requires exact slug/symbol identity, active status, a supplied configured-contract match for assets with known contracts, a quote no older than one hour, positive 24-hour volume, and peg-aware validation. Accepted targeted rows may replay from the provider-local verified cache during the next three 15-minute generations, preserving the original upstream timestamp and expiring at one hour. Neither request retries; success or 429 writes the shared D1 cooldown.
  4. Pass 3: Jupiter Price API for tracked Solana mints (sends JUPITER_API_KEY as x-api-key when configured, checks quoted block freshness against one cached current slot from a bounded three-endpoint sequential RPC fallback, remains liquidity-gated and peg-aware when a quote exists; sparse V3 no-quote rows are treated as healthy empty coverage rather than provider failure; agreeing low-depth Solana primary prices can receive jupiter as a bounded soft candidate without replacing the selected price)
  5. Pass 4: DexScreener exact token-address pool lookups when a resolvable chain+address exists. Successful exact-address enrichments publish priceSource = "dexscreener-exact". The older symbol-search fallback is retired after production Worker probes repeatedly failed without resolving prices. The pass makes at most one same-chain request containing up to 30 exact addresses, with no retries, a 5s request timeout, and a 45s total pass budget. Candidate chains rotate each quarter-hour, and later visits rotate the bounded address window within a large chain cohort. HTTP 429 responses and provider WAF code 1015 are hard refusals.
  6. Pass 5: Allowlisted low-volume CoinGecko recovery for selected tracked assets that still have no price. The reviewed cohort includes BTCUSD, DLLR, ebUSD, AUDm, CHFm, COPm, and GBPm after current CoinGecko rows passed the shared freshness and peg-aware validation gates. Successful rows publish priceSource = "coingecko-low-volume" with priceConfidence = "fallback"; the pass leaves each row's admitted supply source unchanged and does not become replay-safe cached continuity.

Note: the isolated sync-dex-discovery job uses DexScreener's complete single-token pool endpoint (/token-pairs/v1/{chainId}/{tokenAddress}) for staged pool/price observations. The pricing fallback keeps the separate batch token endpoint (/tokens/v1/{chainId}/{addresses}) so it can cover up to 30 exact addresses in one request. sync-dex-liquidity-stage later consumes the fresh discovery rows without repeating the contract fanout, and price enrichment no longer falls back to symbol search.

Price validation ordering: sync-time price validation runs before replay-cache staging so that unreasonable enriched prices never enter price_cache. This prevents a single bad API response from poisoning replay continuity across multiple sync cycles. The worker now distinguishes between authoritative primary validation, fallback enrichment validation, DEX observation validation, and historical-backfill validation instead of using one identical rule for every context. The DefiLlama-down CoinGecko full-supply fallback path now follows the same price guardrails: authoritative live overrides run before enrichment, invalid CoinGecko spot prices are pre-rejected, valid fallback-run prices can refresh price_cache only after canonical publication succeeds, cached-price fallback can heal newly missing prices, and pending-depeg confirmation still runs after fallback detection. Single-source fallback/search-family address-provider quotes also need stronger corroboration before publishing fixed-peg depeg-sized prices, so weak address prints can fall through to later exact-contract enrichment.

Coverage Health And Replay Continuity

activePriceCoverage is computed after the final price-selection and validation stages for both the main and CoinGecko-supply-fallback sync paths. It compares the final rows to the exact active registry and treats every absent row or non-finite/non-positive price as missing. Cron metadata records counts, exact IDs, affected positive circulating USD, current per-gap provenance, consecutive missing generations, rejection class, and last accepted source/time. Incomplete coverage degrades /api/health, but does not downgrade a successfully completed sync-stablecoins execution or suppress an otherwise valid cache publication; consumers can still inspect supply and lifecycle data while the pricing incident remains explicit. After two consecutive published generations, the Worker emits a structured event and sends the shared webhook an asset-specific alert, using a successful-delivery-only 24-hour cooldown. This price contract is separate from exact active-row publication coverage and has no waivers.

For sources that emit asset-level lifecycle telemetry, priceSourceAttemptLedger retains the adapter, source, exact target or slug, attempted/skipped result, rejection class, timestamps, and replay eligibility for assets still missing at publication. The ledger is bounded to 100 records and has a compact tuple form that survives the 64 KiB cron metadata guard. Aggregate sources without asset-attributable telemetry remain represented by source-level diagnostics rather than synthetic per-asset claims.

Replay continuity is a recovery path, not a source of fresh prices. Only validated replay-safe prices above low/fallback confidence are staged, and only after canonical publication succeeds. A missing row may reuse a cache entry for at most six hours, further reduced to the smallest maxTrustedAgeSec among all component sources; a composite containing any non-replay-safe source is ineligible. Replayed rows publish as cached with fallback confidence and still pass current peg, temporal-jump, and previous-trusted-price validation. Extending the global ceiling to hide an unresolved provider outage is not supported.

Data Integrity Guardrails

The sync pipeline includes multiple layers of validation to prevent bad data from reaching users:

  1. Structural validation: DefiLlama response must contain MIN_VALID_ASSET_COUNT (50) assets with valid id, name, symbol, and circulating fields. Malformed objects are dropped before caching
  2. Price validation ordering: sync-time validation rejects prices before savePriceCache(), not after. Fixed pegs use canonical tracked metadata (pegType, navToken, commodityOunces) during validation, NAV tokens still use broad positive-price checks, fiat FX pegs share the USD upside tolerance when a usable reference exists, fractional commodity tokens are always scaled by commodityOunces and keep their broader reference band, and weak fallback/search-family address-provider quotes cannot publish uncorroborated fixed-peg depeg-sized prices
  3. Concurrent cron guard: setCacheIfNewer() uses a compare-and-swap pattern — a slow sync run can't overwrite a newer run's data. Uses syncStartSec as CAS guard. Applied to cache-writing crons such as stablecoins, stablecoin-charts, FX rates, bluechip ratings, and USDS status.
  4. Detail JSON validation: stablecoin-detail.ts parses response JSON before caching; skips cache on parse failure
  5. Detail history freshness guard: /api/stablecoin/:id rejects CoinGecko-derived history whose latest point is more than 72 hours old and falls back to D1 supply_history instead of caching stale chart data
  6. fetchWithRetry: Default 15s timeout prevents hanging Workers. 404 is not passed through by default; callers must opt in via { passthrough404: true }. Timeout and passthrough behavior are configurable per call ({ timeoutMs: N }, { passthroughStatuses: [...] })
  7. Depeg dedup: UNIQUE INDEX (stablecoin_id, started_at, source) prevents duplicate depeg events. Partial index on ended_at IS NULL speeds up open-event queries
  8. Depeg interval merge: computePegScore() and computePegStability() merge overlapping depeg intervals before summing duration
  9. Depeg direction handling: If a coin flips from below-peg to above-peg (or vice versa) without recovering, the old event is closed and a new one opened with the correct direction
  10. Peg score consistency: Both the detail page and peg-summary API use the same tracking-window start helper: coinTrackingStart(...), which applies max(firstSeen, fourYearsAgo) when first-seen data exists. First-seen data is anchored by curated launch date first, then earliest supply_history, then a durable first valid-price observation for priced assets that have not yet written supply history.
  11. Backfill batch safety: backfill-depegs.ts bundles the per-coin DELETE together with the first chunk of up to 99 inserts in one atomic db.batch() (reserving one slot for the delete so it never exceeds D1's 100-statement batch limit), so a partial crash cannot leave the coin event-less; any remaining inserts then ship in sequential groups of 100
  12. OFFSET/LIMIT safety: SQL queries use LIMIT -1 when offset > 0 but no limit is set (bare OFFSET is invalid SQLite). Values are parameterized, not interpolated
  13. Freshness header: /api/stablecoins returns X-Data-Age (seconds since last cache write)
  14. Cloudflare Access admin auth: Admin endpoints are gated by the ops-api.pharos.watch origin lane. When CF_ACCESS_OPS_API_AUD is configured, the worker cryptographically verifies the Cloudflare Access JWT (worker/src/lib/auth.ts). Timing-safe HMAC comparison (timingSafeCompare) is used for the Telegram webhook secret, not for admin endpoints.
  15. Pagination defaults: /api/depeg-events defaults limit to 100 and caps at 1000; /api/blacklist defaults limit to 1000, caps at 1000, and treats limit=0 as "use default". The blacklist frontend fetches a single events page via src/lib/blacklist-api.ts (fetchBlacklistEvents, limit/offset params), and the chart/summary stats are served by the dedicated /api/blacklist-summary endpoint (fetchBlacklistSummary) rather than by client-side multi-page hydration.
  16. Unbounded query guard: /api/peg-summary bounds via the 4-year started_at > filter on the depeg_events query
  17. Cache-empty 503: /api/peg-summary returns HTTP 503 (not 200) when cache is empty, signaling data unavailability
  18. Orphan depeg cleanup: detectDepegEvents() closes open depeg events whose stablecoin was not processed during the current run (removed from tracked list, failed validation, etc.)
  19. Cron history pruning: logCronRun() no longer prunes old rows inline. The daily prune-cron-history job on 0 3 * * * deletes cron_runs rows older than 7 days and cron_slot_executions rows older than 14 days.
  20. Security headers: Worker adds X-Content-Type-Options: nosniff to all responses
  21. Admin cache bypass: cache bypass is declared by each endpoint's cacheBypass flag in shared/lib/api-endpoints/definitions.ts and exposed through isCacheBypassPath(). This covers admin status/API-key/action-log reads plus mutating repair, backfill, and control endpoints; use the shared endpoint registry instead of maintaining a hand-copied route list.
  22. Fail-closed schema guard (stablecoins): syncStablecoins() validates both main and fallback payloads against StablecoinListResponseSchema before setCacheIfNewer(). On schema failure, it does not overwrite the canonical stablecoins cache; instead it writes the rejected payload to stablecoins:invalid-last, returns cron status: "degraded", and alerts with validation context (main/fallback) plus last-known-good cache age
  23. Strict cache payload validation (yield rankings): syncYieldData() validates the yield-rankings cache payload against YieldRankingsResponseSchema before setCache(). On schema failure, cache write is skipped, validationFailures is incremented in cron metadata, and the run returns status: "degraded" so status surfaces do not mark it healthy
  24. Fail-closed transformed cache reads: cache-backed endpoints that must parse and reshape stored JSON now return HTTP 503 when the cached payload is malformed instead of serving a 200 with raw cached bytes. This currently applies to /api/yield-rankings and the cached fallback path in /api/mint-burn-flows.
  25. Canonical V9 safety guard (yield): syncYieldData() reads the accepted report-cards:v9 publication through the shared current-safety loader. The hourly publisher requires a complete, identity-valid V9 active set; missing, held, stale, or incompatible safety data produces an empty degraded safety snapshot while the yield cache remains available. provenance.safetySnapshot records the accepted V9 generation, methodology, policy, and publish time. API-time hydration may use a newer complete current V9 publication with compatible identity; failure emits yield-safety-hydration-degraded without rewriting the hourly evidence. No yield path reads the retired V8 compact cache or recomputes report cards.
  26. Shared stablecoins cache loader: Consumers that read stablecoins (/api/status, /api/peg-summary, /api/mint-burn-flows, daily-digest, compute-dews, stability-index, backfill-depegs) use worker/src/lib/stablecoins-cache.ts instead of ad-hoc JSON.parse logic. The loader separates cache read tolerance (mode: "strict" | "lenient") from cache contract validation (contract: "published" | "critical-fields"). Strict mode fails closed on missing cache, malformed JSON, invalid top-level shape, schema-invalid published-contract objects, legacy array compatibility, and filtered malformed entries. Lenient mode still fails closed on invalid JSON and unusable shapes, but may return kind: "degraded" with a usable payload for opt-in legacy arrays or whole-entry critical-field filtering, including reason and filteredCount. Legacy array-shape compatibility remains opt-in through allowLegacyArray.
  27. DEWS source-failure accounting: computeAndStoreDEWS() records upstream read failures as structured sourceFailures metadata and emits status: "degraded" when non-bootstrap-critical inputs fail. Metadata now includes source coverage and validation-failure counts.
  28. Stage-structured stablecoins sync: syncStablecoins() keeps the same output contract but now delegates intake/fallback gating to worker/src/cron/sync-stablecoins/intake.ts, shared post-enrichment/cache/depeg steps to worker/src/cron/sync-stablecoins/post-enrichment.ts, final run metadata shaping to worker/src/cron/sync-stablecoins/metadata.ts, helper contracts to worker/src/cron/sync-stablecoins/shared.ts, and normalization/filtering/staleness/supply-history fill to worker/src/cron/sync-stablecoins/stages.ts, while supplemental-assets.ts owns commodity and CG-only overlay fetches.
  29. DefiLlama ID remap before enrichment/cache writes: in syncStablecoins(), assets are remapped via REGISTRY_BY_LLAMA_ID immediately after normalizeChainCirculating() and before supplemental merges/applyTrackedAssetOverrides(). This ensures downstream maps and keys (primaryPriceResults.get(asset.id), savePriceCache, cached-price fallback lookups, supply-history fill inputs, and final stablecoins cache payload) consistently use canonical IDs.
  30. Post-remap canonical dedupe: if DefiLlama emits duplicate rows that collapse onto the same canonical Pharos ID, syncStablecoins() now keeps a single preferred row before caching or enrichment. This prevents duplicate canonical assets from double-counting supply in the final stablecoins payload.
  31. Stage-structured yield sync: syncYieldData() now delegates source evaluation and previous-best normalization to worker/src/cron/yield-sync/evaluation.ts, rankings/cache publication and persistence helpers to worker/src/cron/yield-sync/publication.ts, and batched history preload plus stale/orphan cleanup to worker/src/cron/yield-sync/history.ts, keeping resolution logic separate from D1 housekeeping and payload assembly.
  32. Stage-structured mint/burn run-state: syncMintBurn() now delegates disabled-config normalization, lane rotation, and run-state persistence to worker/src/cron/mint-burn/run-state.ts; the two 30-minute scheduled handlers already share worker/src/handlers/scheduled/mint-burn-slot.ts for slot-specific dispatch.
  33. Stage-structured blacklist EVM ingestion: syncBlacklist() now delegates EVM event fetch/parsing, RPC fallback target selection, and shared explorer URL helpers to worker/src/cron/blacklist/evm-source.ts and worker/src/cron/blacklist/shared.ts, isolating the Tron path and downstream balance enrichment from the source-ingest stage.
  34. Stablecoins stale-publication guard: syncStablecoins() now emits stage-level progress and compares fresh prices against the previous published cache before writing. If at least 50 comparable prices exist and >=98% are identical, the run returns status: "degraded", records staleWriteBlocked=true, and skips the canonical stablecoins cache write instead of republishing a stale snapshot as fresh.
  35. Fail-closed PSI dependency handling: computeAndStoreStabilityIndex() no longer treats an unavailable depeg_events query as "no active depegs". The run now also requires a non-empty latest DEWS row set with usable computed_at no older than two compute-dews intervals before deriving stress breadth. These dependency failures degrade and skip publication so PSI remains anchored to the last valid sample.
  36. DEWS bootstrap + freshness guard: computeAndStoreDEWS() now uses a dedicated dews:bootstrap-complete sentinel to end bootstrap grace after the first successful publication, and stale dex_liquidity inputs (>2 hours old) now count as a hard degraded source failure.
  37. Yield publication guardrails: syncYieldData() now degrades on invalid/empty direct DeFiLlama payloads, on total deterministic on-chain failure, and blocks yield-rankings cache writes when the new rankings payload shrinks severely versus the last published cache.
  38. DEWS blacklist coverage parity: computeAndStoreDEWS() now derives blacklist-signal coverage from the shared BLACKLIST_STABLECOINS set instead of a local hardcoded subset, so PYUSD and USD1 receive the same blacklist_events-driven stress input as the other live blacklist-tracked coins.
  39. DEWS thin-peg FX parity: computeAndStoreDEWS() now passes cached fxFallbackRates into derivePegRates(), matching live depeg detection and peg-summary behavior for thin non-USD peg groups.
  40. Recent-only chart FX repair: syncStablecoinCharts() still corrects obvious recent totalCirculatingUSD corruption with the live FX cache, but it no longer rewrites deep historical points with today's FX reference.
  41. Completed-day supply history reads: snapshotSupply() requires exact active-registry coverage rather than a percentage threshold. Every active ID must have usable supply or an owned, reasoned, unexpired waiver; incomplete writes do not advance snapshot-supply:last-write, remain hidden from completed-day reads, and can be retried on the same UTC day. Public supply-history and non-usd-share reads cap rows to the completion marker when present and emit X-Data-Age from the latest completed supply snapshot run.
  42. Single-deployment on-chain supply fallback: CoinGecko-detail supplemental assets may use on-chain totalSupply * price only when exactly one supported deployment can represent global supply. Multi-deployment assets skip that fallback instead of treating one chain's supply as the total market cap unless they are in the curated aggregate on-chain supply list, where every configured chain is read and the fallback fails closed if any configured chain cannot be read. Reviewed zero-supply native deployments may contribute zero only when their aggregate config explicitly allows it. Configured protocol-inventory exclusions may subtract live holder balances from that single deployment before publishing onchain-circulating-supply; if any configured balance read fails, the on-chain fallback is skipped for that run.
  43. CAS outcome visibility and cadence buckets: setCacheIfNewer() returns whether a cache row was actually published or skipped because a newer canonical row already exists. FX and stablecoin-chart producers claim generation-fenced scheduled cadence buckets, complete them only after successful canonical publication (or readback-confirmed newer publication), and leave failures retryable. This avoids 45/90-minute drift caused by elapsed write-age checks.
  44. Freshness sentinel validation: /api/health and /api/status trust freshness:dex-liquidity, freshness:yield-data, and freshness:dews only when the cache row contains a valid JSON producer assertion: updatedAt, expected source, and publishStatus: "ok" with optional rowsWritten / coverageRatio. Malformed, stale, future-dated, wrong-source, or non-ok sentinels fall back to table freshness and then latest successful producer cron freshness, with freshnessSource, sentinelValidationReason, and a warning surfaced in the cache status. compute-dews publishes the DEWS freshness sentinel only after a non-degraded run persists at least one current row, so zero-result or degraded runs cannot mark stale stress-signal tables fresh.
  45. Supplemental last-known-good supply provenance: when a tracked supplemental asset cannot refresh supply in the current run, mergeSupplementalLastKnownGood() may retain the previous positive supply, but the published row is marked with supplyRestored: true and supplyObservedAt from the older cache snapshot. This keeps restored supplemental market caps readable without presenting the retained supply as freshly observed.
  46. Exact stablecoin publication capability: every published stablecoins run compares cache IDs to the active registry. Named unwaived omissions degrade the run and clear the downstream-safe cache capability. The default waiver roster is empty; any future exception must be explicit, owned, reasoned, and time-bounded. Persistent no-supply impossibility is handled through a reviewed quarantine rather than a silent active-coverage waiver. Price gaps and genuine depegs remain active monitoring failures. The data-invariant canary uses the same exact evaluator.
  47. Bounded cron diagnostics: stablecoins cron rows retain counts and bounded diagnostics but never the deduplicated asset objects. Their producer-level guard compacts below 60 KiB before the scheduled wrapper appends lease and slot identity, preserving top-level coverage evidence beneath the global 64 KiB persistence ceiling. Mint/burn rows retain totals, top laggers, and at most 12 config samples; normalized full-run drilldown and per-config attempt age live in latest-only cache records.
  48. Independent active-price coverage: every stablecoins run records exact active price coverage separately from row coverage. A published active row with a missing or invalid price degrades public health without downgrading a successfully completed cron execution or blocking the rest of the cache, and the health parser revalidates the complete priced-ID set against the current active registry before reporting complete.
  49. Publishable-price recovery targeting: authoritative live overrides and exact-address augmentation no longer treat every positive numeric incumbent as resolved. Recovery targeting requires publishable price provenance before a row leaves the missing cohort, and live override candidates have per-candidate fairness caps inside the unchanged shared budget so one stalled protocol route cannot consume the full active-price repair window.

Gold & Silver Spot Prices (gold-api.com)

syncFxRates() in worker/src/cron/sync-fx-rates.ts fetches gold and silver spot prices from the gold-api.com API for commodity-pegged stablecoin peg validation (XAUT, PAXG, KAU, KAG, etc.).

Why gold-api.com?

The previous source (DefiLlama's coingecko:gold / coingecko:silver coins API) silently returns empty data, producing garbage peg references and phantom trillion-BPS depegs in backfilled events. gold-api.com requires no API key, and the worker only performs two live spot requests in each claimed 30-minute sync-fx-rates cadence bucket.

Live Sync (sync-fx-rates.ts)

  • Endpoint: GET https://api.gold-api.com/price/XAU (gold), GET https://api.gold-api.com/price/XAG (silver)
  • Request volume: 2 requests per claimed sync-fx-rates cadence bucket (gold + silver), with no repo-level rate limiter; the quarter-hourly trigger maps to one upstream-work bucket every 30 minutes.
  • Validation: Same isValidRate() bounds + delta checks as FX rates (gold: $500-$10,000/oz, silver: $5-$500/oz, max 20% change from previous value). When a fresh commodity peer median exists, gold-api.com metal spots must also stay within 5% of that peer reference; divergent spots are rejected before they can refresh the shared FX cache.
  • Fallback: If the gold-api.com live fetch fails or fails the peer-median cross-check, sync-fx-rates.ts now derives a fresh commodity reference from the just-written stablecoins cache (peer median across tracked gold tokens; single tracked silver token for silver) before inheriting the previous cached metal rate. This keeps /api/health anchored to an actually fresh commodity reference when the anonymous metals endpoint is blocked from Workers.
  • Chainlink overlay: XAU/XAG Chainlink quotes still have to be fresh under their feed-level staleness window and within 5% of the current metal reference. Because the metals lane can stamp gold-api.com or peer-median references at the current sync start, older but still feed-fresh Chainlink metal quotes are used as validation-only cross-checks instead of being skipped before divergence detection or replacing fresher spot metadata.

For fiat FX, Frankfurter remains the preferred ECB-backed source for the business-day set, including the tracked HKD and INR peg groups. The worker targets the maintained hosted endpoint at https://api.frankfurter.dev/v1, which replaced the retired frankfurter.app host. The existing fawazahmed0/currency-api mirror still owns CNH/RUB/UAH/ARS/KGS/NGN/XOF/VND, and it can also backstop the wider fiat set when Frankfurter is temporarily unavailable so the cron can keep publishing live dated FX references instead of immediately dropping to a cached-only run. If both Frankfurter and the existing secondary mirrors are unavailable, sync-fx-rates.ts falls through to ExchangeRate-API's daily USD reference snapshot as a tertiary full-set fallback before reusing cached rates. If none of those live fetches respond but the previously persisted daily references are still within their expected publish cadence, the cron carries them forward as a live success instead of incrementing the cached-fallback streak. Even after a cached-fallback run begins, the independent OXR, Chainlink, and metals probes still execute; if they restore fresh full-set fiat coverage, the run exits cached fallback immediately instead of waiting for the primary Frankfurter path to recover first.

Backfill (backfill-depegs.ts)

  • backfill-depegs.ts now asks the same authoritative-price registry used by live sync for historical series first. If a coin has an authoritative historical provider and that provider cannot return enough coverage, the backfill preserves existing source='backfill' rows instead of rebuilding from a known-weaker fallback source.
  • Supported non-USD fiat backfills prefer direct CoinGecko native-fiat history and compare it to the native 1.0 peg before they fall back to USD history plus historical FX.
  • Commodity backfill does not call a gold-api.com timeseries endpoint.
  • Instead, it builds daily GOLD/SILVER peg references from CoinGecko historical prices across tracked commodity tokens (buildCommodityMedianSeriesFromCg()), normalized to per-troy-ounce and median-aggregated per day.
  • The resulting { GOLD: FxTimeSeries[], SILVER: FxTimeSeries[] } series feeds buildFxLookup() for time-varying commodity peg references.
  • Fiat backfill uses Frankfurter historical ranges from api.frankfurter.dev/v1 for ECB-covered currencies and date-addressed fawazahmed0/currency-api snapshots for non-ECB currencies such as CNH, RUB, UAH, ARS, KGS, NGN, XOF, and VND.
  • Secondary historical FX snapshots are cached in D1 by year (fx-history-secondary:<year>) so repeated admin backfills do not re-fetch the same daily files.
  • Fallback behavior: if series data is sparse/missing for a timestamp, buildFxLookup() falls back to the current peg reference derived from live rates.
  • Historical depeg extraction validates each price point against the direct peg reference for that timestamp (historical_backfill mode). That preserves confirmed catastrophic downside moves without weakening the tighter fallback/DEX filters used for noisy live sources.
  • Dry-run backfill audits now accept startDay / endDay plus optional contextDays, replay only that UTC window with the requested context pad, and keep long-history non-USD repairs below ops-api timeout limits without changing the full-coin mutation path.

Budget

The live /price/ endpoint requires no API key and is called only in claimed sync-fx-rates cadence buckets (2 requests: gold + silver), roughly 2,880 requests/month. Backfills source commodity history from CoinGecko market-chart data (via existing CoinGecko integration), so there is no separate gold-api.com historical-request budget.

Stability Index (PSI) Computation

computeAndStoreStabilityIndex() in worker/src/cron/stability-index.ts runs every 30 minutes on the DB-only DEWS/PSI lane (26,56 * * * *) and computes a composite ecosystem health score (0–100). Formula: Score = 100 − severity − breadth − stressBreadth + trend. If the DEWS dependency query is unavailable, empty, missing usable computed_at, or stale beyond two compute-dews intervals, the run returns status: "degraded" with fallbackMode: "dews-unavailable" and preservedCurrentSample: true, then skips fresh PSI sample publication instead of treating missing stress breadth as zero. If the active-depeg query is unavailable, the run also fails closed and skips publication instead of treating that outage as an empty depeg set. See Pharos Stability Index for the full algorithm, calibration examples, and band definitions.

Band classification: BEDROCK (90–100), STEADY (75–89), TREMOR (60–74), FRACTURE (40–59), CRISIS (20–39), MELTDOWN (0–19)

Storage: 30-minute samples go into stability_index_samples; daily averages are aggregated by snapshotPsiDaily() into stability_index. Both tables store score, band, components (JSON), input_snapshot (JSON). Schema definitions are in worker/migrations/0000_baseline.sql.

Pending Depeg Confirmation

For stablecoins at or above the large-cap confirmation floor, plus tiered near-large-cap cases, depeg detection uses a two-phase confirmation system:

  1. Phase 1 (detect-depegs.ts): When a coin requires confirmation instead of direct mutation, a record is inserted into depeg_pending (schema in worker/migrations/0000_baseline.sql). This now covers large-cap supply (>= $1B), tiered near-large-cap checks (>= $750M with weak source depth or >= 2x severity; >= $500M only when both weak-source and severe), low-confidence/cached/stale primary prices, and extreme moves (abs(bps) >= 5000)
  2. Phase 2 (confirm-pending-depegs.ts): On the next cron cycle, pending records are re-checked. If the depeg persists and a secondary source agrees, a real depeg event is opened. If an authoritative primary price recovered, the pending record is deleted

This prevents false positive depeg events for systemically important stablecoins during brief price feed glitches.

Stale Data Monitoring (Frontend)

The StaleDataBanner component (src/components/stale-data-banner.tsx) warns users when data from selected critical queries is degraded or stale. Frontend freshness uses the shared FRESHNESS_RATIOS thresholds from shared/lib/status-thresholds.ts: fresh through 8x staleTime, degraded through 12x staleTime, then stale. When a hook uses apiFetchWithMeta(), backend freshness metadata (_meta.status, X-Data-Age, stale Warning) takes precedence over browser fetch time so a fresh client refetch cannot mask stale server data. The table below lists the page-level banner coverage; some routes also render additional detail queries that are handled locally rather than by the page-level banner:

PageQueries monitoredstaleTime constants
HomepagePrices, Peg Data, Liquidity, Report CardsCRON_15MIN, CRON_15MIN, CRON_30MIN, CRON_15MIN
Stablecoin detailPrices, Peg Data, Liquidity, Report Cards, Redemption BackstopsCRON_15MIN, CRON_15MIN, CRON_30MIN, CRON_15MIN, CRON_RESERVE_SYNC
DepegPeg Data, DEWS, Depeg EventsCRON_15MIN, CRON_30MIN, CRON_15MIN
ComparePrices, Peg Data, Liquidity, Report Cards, BluechipCRON_15MIN, CRON_15MIN, CRON_30MIN, CRON_15MIN, CRON_24H
Safety scoresGrades, PricesCRON_15MIN, CRON_15MIN
LiquidityLiquidityCRON_30MIN
YieldYield Rankingssync-yield-data descriptor interval (1 hour)
FlowsMint/Burn FlowsCRON_MINT_BURN
BlacklistBlacklistCRON_BLACKLIST
CoverageCoverage matrix inputsroute model stale-query set
PortfolioGradesCRON_15MIN
ChainsChain DataCRON_30MIN freshness budget from /api/chains
Chain detailChain Data, PricesCRON_30MIN, CRON_15MIN
Stability IndexStability IndexCRON_30MIN
Digest archiveDigestsCRON_24H

Homepage KPI cards also consume PSI, mint/burn, and DEWS data, while Compare can fetch supply-history and per-coin mint/burn detail queries. Those additional queries are not part of the current page-level stale banner contract.

Constants defined in src/lib/cron-intervals.ts: CRON_1MIN (1 min), CRON_15MIN (15 min, stablecoins list), CRON_30MIN (30 min, DEX liquidity), CRON_MINT_BURN (30 min, mint/burn), CRON_1H (1 hour, generic budget), CRON_RESERVE_SYNC (4 hours, live reserves + redemption backstops), CRON_BLACKLIST (6 hours), CRON_24H (24 hours). Yield queries use the canonical sync-yield-data interval from FRONTEND_API_QUERY_DESCRIPTORS.

The staleTime value for each query matches the cron interval of the backend job that produces the data. TanStack Query's refetchInterval is always 2x the staleTime. Local browser age becomes degraded after 8x staleTime and stale after 12x staleTime, while hook-level freshness metadata can mark data degraded/stale sooner when the worker explicitly reports old cache age or stale-table warnings.

Blacklist Sync State Semantics

The blacklist_sync_state.last_block column has different semantics per chain type:

  • EVM chains: stores actual block numbers
  • Tron: stores millisecond timestamps (Tron events are ordered by timestamp, not block number)

This is intentional — do not mix these values across chain types.