Reference for limits we can verify from repo code and checked-in config.
This document intentionally focuses on:
- limits enforced in code
- budgets encoded in config
- runtime assumptions the scheduler is explicitly designed around
It intentionally does not treat vendor pricing-plan quotas as source of truth. Cloudflare, CoinGecko, Alchemy, Dwellir, Etherscan, Anthropic, X, and similar providers can change those independently of this repo. Re-check official vendor docs or your live account dashboard before making spend-sensitive or capacity-sensitive changes. Where the repo enforces its own ceiling instead of trusting the vendor, the named repo constant — not the plan's published quota — is the authority (the Dwellir credit cap below is the worked example).
Agent navigation — Runtime limits · Connection assumptions · Discovery/reserve budgets · Telegram planning · Safety Score budgets · Delivery/supporting producers · Blacklist · Mint/burn · Cron timeouts · Price providers · Metered providers · Market/DEX fetches · Body caps · Request timeouts · Digest.
Primary Sources
worker/wrangler.tomlshared/lib/cron-jobs.tsworker/src/lib/cron-timeouts.tsworker/src/lib/rate-limit.tsworker/src/lib/evm-rpc.tsworker/src/lib/rpc-provider-budget.tsshared/lib/ops-limits.tsworker/src/lib/api-keys.tsworker/src/lib/circuit-breaker.tsworker/src/handlers/http/gates.tsworker/src/cron/sync-blacklist.tsworker/src/cron/sync-mint-burn.tsworker/src/cron/sync-live-reserves.tsworker/src/cron/sync-live-reserves-config.tsworker/src/lib/address-price-providers/index.tsworker/src/lib/authoritative-price-sources/index.tsworker/src/cron/sync-stablecoins/enrich-prices-cmc-pass.tsworker/src/cron/sync-stablecoins/post-enrichment.tsworker/src/cron/dex-discovery/orchestrator.tsworker/src/cron/dex-discovery/refresh-stale-pools.tsworker/src/cron/measured-execution/sync.tsworker/src/cron/measured-execution/profiles.tsworker/src/cron/sync-stablecoins/enrich-prices.tsworker/src/cron/sync-fx-rates.tsworker/src/cron/prepare-safety-score-v9-input.tsworker/src/cron/compute-safety-score-v9.tsworker/src/cron/daily-digest.tsworker/src/cron/sync-yield-data.tsworker/src/cron/sync-yield-supplemental.tsworker/src/cron/fetch-tbill-rate.ts
Worker Runtime
| Constraint | Current repo value | Source | Notes |
|---|---|---|---|
| Worker CPU budget per invocation | 300000 ms | worker/wrangler.toml | Hard repo-configured CPU cap via [limits].cpu_ms. Cloudflare's Workers limits apply a 30-second CPU class to Cron expressions with intervals below one hour and a 15-minute class to hourly-or-longer expressions. The existing DEX lanes (halfHourlyOffset, halfHourlyChartsOffset) and the converted quarterHourly, v9SupplyAttributionOffset, depegResolverOffset, v9PublicationOffset, and statusSelfCheckOffset lanes, plus both paired mint/burn lanes, use hourly physical aliases to qualify for the 15-minute class while preserving logical sub-hourly cadence; the repo cap remains five minutes. Telegram dispatch (C102) continues to apply its own fresh-send budget before formatting. |
| Cron expressions / trigger slots | Source-owned; reviewed ceilings in CRON_GROWTH_HEADROOM_POLICY; run the cron checks | worker/wrangler.toml, shared/lib/cron-jobs.ts, shared/lib/scheduled-runner-registry.ts | CRON_SCHEDULES owns logical status/slot cadence, while CRON_TRIGGER_SCHEDULES owns the deployed physical expressions. The reviewed physical-trigger, fetch-capable-entry, and headroom-full ceilings live in CRON_GROWTH_HEADROOM_POLICY (shared/lib/cron-jobs.ts); run npm run check:cron-sync / npm run check:cron-connections for the live topology rather than reading a count here. The reviewed shape is the hourly DEX source lane (halfHourlyOffset), the paired-hourly DEX consumer lane (halfHourlyChartsOffset), plus hourly aliases for quarterHourly, v9SupplyAttributionOffset, depegResolverOffset, statusSelfCheckOffset, and v9PublicationOffset, and paired hourly aliases for both mint/burn lanes; ADR-21 converted V9 publication, ADR-22 isolated DDR memory and restored the critical mint/burn lane's hourly CPU class, and ADR-23 corrected the extended lane after production abandonments. The runner registry maps every physical expression back to one logical slot identity, and npm run check:cron-sync enforces that mapping. Splitting aliases rebalances existing logical work rather than adding scheduled work. check:cron-connections also models budget-only scheduled surfaces with connectionGroup metadata. Failure-isolated reserve recovery remains independent of the interrupted invocation. |
| Status-tracked and budget-only jobs | Source-owned; exposed by status and budget registries | shared/lib/cron-jobs.ts, shared/lib/scheduled-runner-registry.ts | Runtime scheduling matches the shared status metadata expected by /api/status; budget-only work is modeled separately by the connection-budget registry. |
| API key default limiter | 120 requests / 60 seconds per key | shared/lib/ops-limits.ts, worker/src/lib/api-keys.ts | Non-exempt /api/* requests require a valid X-API-Key. shared/lib/api-endpoints/ owns the no-key exceptions; read the current set there rather than from a list here. The D1-backed api_key_rate_limit table enforces per-key quotas in the normal path, and per-key overrides live in api_keys.rate_limit_per_minute. Protected cacheable GET edge-cache hits can use a bounded isolate-local fast path only for recently verified non-self-serve keys; cold, unknown, self-serve, expired auth-cache, cache-miss, cache-bypass, and non-GET requests stay on the D1-backed path or fail closed. After repeated D1 limiter failures, protected cacheable GET routes open a 60-second isolate-local fallback circuit capped at the self-serve quota (30/min); cold or unknown keys still fail closed with 503 and Retry-After: 60. Last-used metadata writes are best-effort. |
| Legacy self-serve API key policy | 30 requests / 60 seconds, 60 days expiry | shared/lib/ops-limits.ts, worker/src/lib/api-key-auth.ts | Retired lane (removed 2026-09-29): existing tier="self-serve" keys keep authenticating at 30 requests per minute on the D1-only path until they drain through their 60-day expiry (about 2026-11-06); keyed access today is the donor supporter key via POST /api/donor-key-claims or an operator-issued partner key. |
| API key revocation latency | ≤5 seconds for protected cacheable edge-cache hits; immediate on the D1-backed path | worker/src/lib/api-key-core.ts (API_KEY_AUTH_CACHE_TTL_MS) | Normal API-key authentication consults D1 and fails closed if lookup storage is unavailable. A recently verified non-self-serve key can use the bounded fresh auth cache only on the narrow protected cacheable GET edge-cache-hit fast path, and the cache is not accepted as stale authentication material after lookup failures. |
| Query-free GET cache identity | URL query strings are stripped only for endpoint descriptors whose handlers declare no query inputs; parameterized routes retain their query-bearing keys | shared/lib/api-endpoints/definitions.ts, worker/src/handlers/http/edge-cache.ts | Cache-key normalization does not bypass per-request API-key auth, attribution, Vary, CORS, or Cache-Control handling. |
| Feedback limiter | <!-- GENERATED-START: worker-feedback-rate-limit -->3 submissions / 10 minutes per salted IP hash<!-- GENERATED-END: worker-feedback-rate-limit --> | worker/src/api/feedback.ts, worker/src/lib/rate-limit.ts | Separate from the per-key limiter |
| Supporter key claim limiter | 10 requests / 60 seconds per client IP | shared/lib/ops-limits.ts, worker/wrangler.toml (DONOR_KEY_CLAIM_RATE_LIMIT), worker/src/api/donor-key-claims.ts | Cloudflare [[ratelimits]] binding spent before the POST /api/donor-key-claims body is read, so signature verification and D1 are never reached by a denied caller. Denial returns 429 with Retry-After: 60; a missing binding returns 503. |
| Supporter (donor) key policy | 10 requests / 60 seconds, no expiry | shared/lib/ops-limits.ts, worker/src/api/donor-key-claims.ts | One key per eligible donor wallet. tier="donor" keys are excluded from the isolate fast-cache auth path and always use the D1-backed limiter, so the ceiling is global rather than per isolate and a D1 auth outage fails them closed. |
| Request attribution telemetry retention | 35 days | worker/src/lib/request-source-attribution.ts, functions/lib/request-attribution.ts | Total site-vs-external demand, worker-lane load, and per-key public-API load buckets in api_request_consumer_stats / site_data_request_stats / api_key_request_stats are pruned opportunistically. Worker route/source, Worker per-key, and Pages site-data counters use short isolate-local batching before D1 upsert. Set REQUEST_SOURCE_ATTRIBUTION_DISABLED=true on Worker and/or Pages to pause low-value route/source writes without disabling API-key auth, D1-backed rate limiting, or per-key public API load telemetry. Set API_KEY_REQUEST_ATTRIBUTION_DISABLED=true on the Worker only for keyed public-API spikes where per-key observability writes also need a D1 pressure relief valve; auth, rate limiting, and last-used metadata stay enabled. |
Connection-budget operating assumption
Cloudflare currently limits each invocation to six simultaneous outbound requests that are still waiting for response headers. A request stops occupying that header-wait slot once headers arrive. Pharos deliberately uses a stricter trigger-wide six-connection model in its static scheduler budget so nested and chained fetch phases cannot accidentally overlap at the platform ceiling. See the Workers limits and the April 2026 connection-limit change.
The scheduler is structured around that conservative repo constraint:
- heavy lanes get isolated trigger slots (
sync-blacklist,sync-mint-burn,sync-mint-burn-extended,sync-dex-discovery,sync-dex-liquidity-stage) - shared slots bundle only related work
- the quarter-hourly handler sequences jobs instead of fanning them out blindly, with D1-only DDR work moved to the later
+8follow-up lane npm run check:cron-connectionsfails any trigger at or above6/6and reports5/6triggers as headroom full- the connection check includes budget-only scheduled surfaces that do not create separate
cron_runsrows:telegram-registration-reconciliation,telegram-digest-outbox-drain,safety-map-producer-kick, anddigest-trigger-poll
Treat any new fetch-heavy work added to an existing trigger slot as competing for the same trigger-wide outbound connection budget. A trigger at 5/6 must be treated as full for new fetch-heavy work unless the change also reduces existing peak usage or moves work to a different slot.
Current state: halfHourlyOffset and halfHourlyChartsOffset are the two modeled 5/6 slots and are full for new fetch-heavy work. halfHourlyChartsOffset reaches that peak only on bounded same-hour stage recovery inside the DEX consumer at :16, or at :46 when the hour still needs publication; its normal serial chain peaks at 3/6. The separate halfHourlyOffset contains the hourly :10 sync-dex-liquidity-stage, whose nested direct-API phase can reach 5/6 while staying below both the platform header-wait ceiling and the repo budget. Active measured execution uses three EVM lanes (3/6), while the shadow EVM lane runs daily. The sync-dex-liquidity consumer, exit-route turnover watchdog, V9 input preparation, and chart writer remain one serial chain: prices, liquidity scores, and the watchdog publish hourly at :16; :46 reuses the current generation without rewriting DEX surfaces unless that hour's stage was never consumed, in which case it publishes it, recovering a terminal or never-started stage first.
Yield supplemental families are serialized, with Beefy, Royco, and Aave each capped at three concurrent outbound operations. The standalone sync-yield-supplemental peak is 3/6; the hourly catch-up plus parity lane is declared at 4/6. Aave's six-target inventory runs in two batches of three within its 28-second family deadline; nested detail/RPC fan-out must stay inside the family cap.
For sync-stablecoins, failed upstream responses must still be consumed or canceled before later passes start. That rule bounds unread bytes, completes transport cleanup, and keeps fetch phases deterministic; it is not a claim that an unread body still occupies Cloudflare's header-wait slot. The late fallback phase remains CoinMarketCap -> Jupiter -> DexScreener.
Mint/burn conservation audits pool boundary-header, hash-pinned supply and boundary-recheck reads in a serialized pre-pass before contract scans. The three phases consume chunks of at most 100 calls per POST from the lane global request budget, not per-config budgets, with a 45-second pre-pass allowance capped by the lane deadline and a 15-second per-POST timeout capped by the remaining allowance. They introduce no new trigger or parallel connection peak. Each config still reuses fetched logs and independently fences/persists/advances its cursor. Exhausted audit capacity remains unverified rather than causing an unlimited retry loop or expanding the lane budget. See Raw Token Conservation.
The same cleanup rule applies to Worker-side integration clients. Telegram delivery, X posting, and GitHub feedback submission should consume or cancel response bodies before returning so transports do not retain unread streams or unbounded response bytes.
Cron Budgeting
Discovery and reserve run budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| DEX discovery overall deadline | 12 minutes | worker/src/cron/dex-discovery/orchestrator.ts | Shared deadline for the discovery pass before persistence/cleanup tail work |
| DEX discovery per-coin budget | 25 seconds | worker/src/cron/dex-discovery/orchestrator.ts | Prevents one slow coin from consuming the whole staging lane |
| DEX discovery stale-pool refresh | 150 seconds slice of the run budget; at most 125 requests x 30 pool addresses (3,750 pools) per run; 8 s timeout/request, 0 retries; stops after 3 consecutive provider failures or when the CoinGecko onchain circuit opens | worker/src/cron/dex-discovery/refresh-stale-pools.ts | Runs before cohort crawling, one request in flight at the 250 ms CoinGecko onchain pacing, so the trigger's 2/6 connection declaration is unchanged. The 2026-09-28 backlog (~2,330 pools on 66 networks) packs into ~122 requests, one run at the measured ~0.5 s per paced request (~1 minute); afterwards only pools that reach 20 h without a cohort crawl are due, and a pool answered for without a refresh backs off (6 h doubling to 4 days) instead of taking a slot every run |
| Stellar Horizon discovery pacing | 1 request start / second; 8-second stage timeout | worker/src/cron/dex-discovery/crawl-horizon-pools.ts, worker/src/lib/rate-limit.ts | Keeps the native Stellar AMM census within Horizon's public 3,600-request/hour limit and inside the shared per-coin deadline |
| Live reserve sync internal run budget | 9 minutes | worker/src/cron/sync-live-reserves-config.ts | Default cursoring budget; if the remaining budget drops below one adapter attempt, the untouched tail is marked deferred and resumed from cursor on the next run, leaving at least two minutes of wrapper headroom for D1 cleanup and cron logging. Optional finalization cleanup/history pruning is skipped when the D1 tail budget is already exhausted, and the skip is recorded in cron metadata. |
| Live reserve adapter I/O peak | 2 outbound operations per adapter attempt | worker/src/cron/reserve-adapters/concurrency.ts, shared/lib/cron-jobs.ts | Coin loop is serialized, but individual adapters can fan out internally; shared fetch/RPC helpers enforce the per-attempt limiter |
Telegram planning budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| Telegram authoritative target planning | 32 durable transitions per dispatch invocation; 90 candidate chats/page; 45 handoff targets/page; 100 statements per D1 transaction | shared/lib/telegram-delivery-policy.ts, worker/src/cron/dispatch-telegram-authoritative-planning.ts, worker/src/cron/telegram-alert-target-plans/ | Existing due deliveries drain before source-specific candidate capture and target planning. Candidate pages union only the source's direct, resolved-preset, and global scopes. Capture/planning reuse fan-out inputs only while preference generations match. Page materialization packs complete idempotent plan units into bounded D1 transactions; handoff validates exact plans before one set-based suppression pass and one atomic enqueue/backoff/target-state batch per page. |
| Telegram authoritative retention | 24 hours workflow state; 14 days settled exact replay and legacy terminal targets; 30 days unreferenced unresolved residue; 90 days ambiguous/audit; 100,000 high-volume rows/table/day | worker/src/cron/telegram-retention-cleanup.ts | Terminal subscriber/page/item/expiry workflow state ages out first. Settled target/plan/source bundles and terminal pre-authoritative targets retain 14 days and are deleted child-first. Active pending/claimed/sending or execution_unknown effects remain protected; degraded and other ambiguous audit evidence retains 90 days. Expired source-less queued jobs and expired unresolved sources without any dependent rows retain 30 days. Deletes use 10,000-row SQL sub-batches and expose cutoff, oldest-row, cap, duration, and isolated-error telemetry. |
| Telegram personalized recap planning | 90 due preferences/page, 10 pages (900 recipients), 3,000 Tape rows/page cap-plus-one, 90 min Tape freshness, 6 h pending TTL; shared-slot 5 min less risk runtime and 30 sec reserve | shared/lib/telegram-recap-policy.ts, worker/src/handlers/scheduled/five-minute-telegram.ts, worker/src/cron/telegram-recap-planner.ts | D1-only deterministic planning with zero AI/external planning calls. Priority 100 keeps recap work below risk, legacy, and admin pending rows; incomplete Tape windows defer instead of silently truncating. A tokened run defers after locked/incomplete/failed risk dispatch and cannot start without positive shared-slot budget. |
| Telegram eventless dispatch fast path | 5-minute dispatch cadence with fan-out skipped on every no-change run | worker/src/cron/dispatch-telegram-alerts.ts, worker/src/cron/dispatch-telegram-queue-paths.ts | Quiet runs drain/clean due or expired pending rows and refresh only available snapshots. An unavailable safety source preserves its held baseline; a healthy safety reseed writes directly. Neither case creates a source event, captures subscribers, or builds fresh targets. |
| Telegram safety source stale threshold | 2 canonical V9 publication intervals (60 minutes) | worker/src/lib/alert-safety-source-cache.ts, shared/lib/cron-jobs.ts | Safety alerts remain suppressed until compute-safety-score-v9 publishes a fresh generation-valid canonical snapshot |
Safety Score production budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| Canonical V9 publication cache budget | <= 1.9 MB stored; <= 1.35 MB compressed; <= 8 MB uncompressed | worker/src/lib/safety-score-v9/publication-codec.ts | The full public V9 payload is canonicalized, checksum-verified, and stored as gzip/base64 in D1. The reader accepts the prior deployment envelope during the one-way cache-key migration; every successful canonical publication rewrites it in the current format. |
| V9 production scheduling | Supply at +8; DEX-bound input and transfer-materiality preparation at +16 and +46; canonical publication at +22 and +52; D1 dependency/version/global-memory-lane admission; absolute slot deadlines | worker/wrangler.toml, shared/lib/scheduled-runner-registry.ts, worker/src/lib/v9-slot-window.ts | Successful DEX publication passes its exact generation ID directly to V9 input preparation. That stage observes transfer materiality with a 3/6 peak; the failure-independent chart writer runs serially, so the ordinary trigger peak stays at 3/6, rising to the reviewed 5/6 only while the bounded same-hour stage-recovery re-run is active inside the DEX consumer. Supply and publication each request a three-minute outer window, the publication runner retains its independent two-minute compiler timeout, and runV9AfterCoreWithinWindow() clamps every requested window to the next quarter-hour boundary. Lane ordering, leases, and the publication-admission fences are owned by Worker Infrastructure: Cron Scheduling. |
| Isolated V9 supply-attribution producer | 3/6 outbound connections per capture; independent DDR runs at :13/:28/:43/:58 with 0/6 outbound connections; 12-minute complete-success due interval, 14-minute complete-rejection retry, 45-minute compiler acceptance window; V9 compilation consumes the generation from D1 | worker/src/cron/sync-v9-supply-attribution.ts, worker/src/cron/compute-depeg-resolver.ts, worker/src/lib/safety-score-v9/supply-attribution-generation.ts, worker/src/lib/safety-score-v9/supply-attribution.ts, worker/src/lib/safety-score-v9/wm-supply-observer.ts, worker/src/lib/safety-score-v9/centrifuge-supply-observer.ts, worker/src/lib/safety-score-v9/xaut-supply-observer.ts, shared/lib/cron-jobs.ts | The attribution slot reads the prior exact V8 input, observes the exact expected asset inventory, appends the bounded journal, and atomically writes a private content-addressed generation. The independent DDR slot lazily loads and runs compute-depeg-resolver from the latest sync-stablecoins capability metadata. DDR has zero outbound fetch budget and runs in a separate scheduled invocation, so the attribution trigger remains 3/6 and DDR remains 0/6; the same service may reuse a warm isolate, but the large run-scoped graphs no longer coexist within one invocation. The attribution due intervals sit under the 15-minute trigger grid, so every attribution firing captures and the 22,52 publication consumes the :08/:38 capture roughly 14 minutes old instead of the previous cycle's at roughly 29. The capture must stay before the 16,46 prepare slot: the compiler admits a generation only when captureClockSec <= fixedInput.clockSec, since a publication must not depend on an observation taken after its own input snapshot, and prepare-safety-score-v9-input stamps that clock. A capture moved into the prepare-to-publication gap is rejected as capture-clock-after-consumer, and the cadence-defer branch then skips the publication on every subsequent cycle — observed in production on 2026-08-09 when the grid was briefly moved to 5,20,35,50. XAUT consumes the configured Tether transparency response before RPC work, requires its issuer timestamp within 48 hours, and hashes the exact response body. It then selects a finalized Ethereum block and uses one Multicall plus serialized proxy/code-identity waves with a 3-connection peak; canonical total supply, pinned treasury not-issued inventory, adapter lockbox balance, token links, endpoint, implementations, and confirmation header are bound to that block. Issuer authorized/not-issued amounts must reconcile exactly to the finalized total/treasury reads before the adapter share is divided by circulating liabilities and bound to the complete reviewed XAUt0 representation-group inventory without inferring destination shares. XAUT's explicit one-hour observation limit is preserved when an accepted attribution becomes chain-supply fact evidence; every other asset retains the generic chain-supply freshness window. The wM observer visits its four EVM deployments sequentially at safely lagged, hash-bound blocks; each route's Multicall, proxy-runtime-code read, and implementation-slot read share a 3-connection peak, followed by a same-block implementation runtime-code read. The Centrifuge observer serializes all JTRSY or ACRDX routes and keeps the same 3-connection peak: one EVM Multicall runs beside runtime-code and implementation-slot reads, while Solana reads mint and direct authority in one finalized context. A finalized Solana bank context can be a skipped slot, so the observer resolves the newest produced finalized block within the prior 64 slots before binding the observation time and hash; the three calls remain sequential and do not increase peak connections. It admits only the complete official burn/mint inventory with pinned Spoke authorization, runtime code, non-proxy state, and Token-2022 authority; lock/mint or adapter inventories are ineligible. EVM observations remain at or before the source fixed clock. wM's finalized Solana observation may be later only while the complete packet remains inside the 120-second capture envelope; Centrifuge packets require every route at or before the clock. Complete reviewed packets are all-or-nothing, and every rejected attempt records immutable bounded admission/fallback provenance plus an allowlisted exact observer leaf code; known rejected source times reuse the bounded timestamp field. A generation whose only producer rejection is XAUT transparency-stale remains cron-healthy and records diagnostic rejected-asset counters because the aggregate-only fallback is bounded and complete; unavailable sources, invalid payloads, identity drift, RPC gaps, and reconciliation failures remain blocking rejected assets for producer health. The isolated V9 compiler accepts only a schema-valid, inventory-compatible fresh generation and re-derives its USD allocations against the exact V8 aggregate liabilities. The generation's captured registry fingerprint is provenance, not an admission gate: it is global, so a release editing any registry input rotates it for every asset including untouched ones, and gating on it discarded the whole generation for one publication cycle after each deploy. Registry relevance is enforced per asset instead, where re-derivation recomputes the route inventory digest and identity pins against the live registry. A complete same-fixed-input generation that finished shortly after the scoring clock is a neutral cadence defer, preserving the prior publication instead of publishing aggregate-only partial ratings. A missing, malformed, rejected, stale, or inventory-incompatible generation fails the whole map closed to aggregate-only attribution with a clause-specific reason; a single accepted asset that fails re-derivation against its own observation window, route inventory, or identity pins fails only that asset closed and is reported in supplyAttributionGeneration.invalidAssetIds. Neither compiler path performs network fan-out. |
| V9 production regression envelope | Full active registry plus accepted-publication comparison and canonical V9 publication pipeline under 128 MiB Node old-space | worker/src/lib/__tests__/safety-score-v9-resource-budget.test.ts | The out-of-process bundled Node 24 guard inflates a prior accepted publication, projects the compact gate/delta baseline, releases the full prior graph, compiles the canonical response, and compresses the replacement publication. Production follows the same accepted-first ordering, and held-publication validation reads only the stored envelope identity instead of inflating the prior response again. This is a deterministic regression envelope, not proof of Cloudflare's total per-isolate memory limit; the fenced dedicated trigger still requires observed clean production executions before operational acceptance. |
| V9 supply-attribution source projection | < 5% of the exact input's uncompressed bytes in the full-registry regression fixture | worker/src/lib/safety-score-v9/supply-attribution-source.ts, worker/src/lib/__tests__/safety-score-v9-supply-attribution-source.test.ts | Input preparation atomically writes an identity-linked projection containing only attribution-cohort identities and supply rows. The +8 producer fails closed on a missing, malformed, or stale projection and does not inflate the full compiler input before RPC capture. This supersedes the older direct exact-input read while preserving the same base-input, source-generation, registry, and clock fences. |
| Accepted V9 offline-replay retention | One accepted base envelope plus one enrichment delta; each reuses the fixed-input codec's 1.9 MB stored / 1.35 MB compressed / 8 MB raw ceilings | worker/src/lib/safety-score-v9/publication-replay-capture.ts, worker/src/lib/report-cards-fixed-input-cache-codec.ts | The :22/:52 publisher serializes only already-held enrichment projections, not another full capture. When serialization succeeds, the existing compressed base string and delta are written in the publication's atomic D1 batch; serialization failure warns and leaves both prior rows unchanged without blocking canonical cards. The 128 MiB Node old-space guard includes production-scale synthetic journals/provenance and a delta raw-size floor above 300 KB; this is regression evidence, not a Cloudflare isolate-memory measurement. Scenario evaluation remains exclusively in offline Node. |
GET /api/yield-rankings (detailed and summary) validates the yield cache, then hydrates from report-cards:v9:score-index, not the full compressed publication. The same compact source serves grades, chain health, donor eligibility, mint/burn FTQ, OG scores and score-only Telegram commands; report-card details, dependency graphs and /why still need full cards. One D1 query reads the index and health plus server-extracted publication identity, result digest and timestamp. Only exact index/envelope identities and digests are usable; health must name that accepted generation. A health-clock discrepancy uses the existing held verdict, never a fresh claim.
Missing, invalid or generation/digest-mismatched indexes fail closed without full-parse fallback. Rankings keep yield observations but blank safety/PYS to null/NR, including inside the publish-time fallback window, with safety-score-index-missing, safety-score-index-invalid, safety-score-index-publication-mismatch or safety-score-index-health-mismatch in hydration provenance. Index read/health failures also have explicit reasons; /api/safety-grades returns uncached 503 with a machine-readable reason. Matching live and held publications retain existing response bytes and freshness policy.
Deploy transition: no migration/backfill is required. The first accepted :22 or :52 compile atomically creates the index with the publication, normally within one half-hour plus compilation time. Until then, score-only consumers report unavailable; a held/rejected compile does not manufacture an index and extends that unavailable window until acceptance. Existing full report cards remain readable. Node 24.16, 128 MiB old-space, five sequential production-row ranking requests measured 56.91 MiB peak pre-GC and 45.11 MiB retained after explicit GC (2026-10-02); this is comparative evidence, not proof of the 128 MB Cloudflare isolate limit.
Delivery and supporting producer budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| Telegram registration reconciliation peak | 1 outbound Bot API call at a time | worker/src/lib/telegram/webhook-registration.ts, shared/lib/cron-jobs.ts | Runs serially before dispatch-telegram-alerts when 15-minute cache markers expire; modeled as the budget-only telegram-registration-reconciliation entry in the same five-minute Telegram connection group and surfaced through /api/status.budgetOnlySurfaces. |
| Telegram digest outbox retry peak | 1 Bot API call at a time; up to 4 due editions per five-minute poll | worker/src/lib/telegram/digest-outbox.ts, worker/src/handlers/scheduled/digest-trigger-poll.ts | Runs serially on the existing */5 digest-trigger slot. Retryable HTTP failures retain the exact stored chunks and honor retry_after; ambiguous and permanent outcomes stop for operator review. Retained terminal backlog degrades budget-surface telemetry but does not repeatedly feed the Telegram provider circuit when no network attempt occurred. |
| Daily digest social attachment peak | 1 outbound request at a time | worker/src/cron/daily-digest.ts, worker/src/lib/digest-safety-map.ts, worker/src/lib/twitter.ts, shared/lib/cron-jobs.ts | The 08:05 UTC digest chain serializes manifest GET, dated-image HEAD, optional X image GET/media upload/post, and Telegram delivery after the Anthropic request. The extra map requests do not raise the slot's 1/6 connection peak; every response is consumed or cancelled before the next phase. Missing/stale map state and X media failures fall back to text-only social delivery. |
| Persisted cron metadata ceiling | <64 KiB per cron_runs.metadata payload | worker/src/lib/cron-metadata-persistence.ts, worker/src/lib/cron-logger.ts | Global compaction retains counts, bounded samples, and latest drill-down state while preventing large asset/config arrays from driving unbounded D1 growth. |
| GBP SONIA retained-fallback degradation | 2 consecutive daily retained-fallback runs | worker/src/cron/fetch-tbill-rate.ts | The first gbp-sonia-compounded-index-failed-retained run writes cache["fetch-tbill-rate:gbp-retained-fallback-streak"]; the second consecutive run remains visible through degraded cron status. Fresh GBP market data resets the streak. |
| Telegram pulse heavy-section cadence | 15 minutes | worker/src/api/telegram-pulse.ts, worker/src/lib/telegram/usage-analytics.ts | Current aggregate pulse counts still refresh every 5 minutes. Top coins, completed-day persisted lifecycle history, and Mini App daily counters reuse a rendered pulse section for up to 15 minutes; pending-delivery count can reuse the dispatch lane's pending-capacity snapshot. Lifecycle history is bounded to 90 complete UTC days and is never reconstructed from current subscriber cohorts. |
| Telegram load simulation targets | 500, 1,000, 5,000, 10,000 active watchers plus production calibration at 855 subscribers / 338 candidates / 300 targets | shared/lib/telegram-delivery-policy.ts, shared/lib/telegram-recap-policy.ts, scripts/ci/check-telegram-load.ts | npm run check:telegram-load estimates drain time, per-invocation estimatedCpuMs, and D1 operations for risk, admin, and personalized-recap scenarios. The production-calibrated dispatch gate additionally enforces candidate-horizon reduction, one fan-out load per capture page without invalidation, at most eight D1 round trips per bounded handoff page, and projected wall time below three minutes. Recap scenarios enforce a 5,000-recipient target across all-due, risk burst, 429 storm, preset-heavy, global-scope, no-change, and stale-Tape cases, with aiCalls=0 and externalPlanningFetches=0; they also emit a non-enforced 10,000-recipient advisory. The recap planner uses a 900-recipient/5-minute-run page budget, a 6-hour pending TTL, and priority 100. The CPU dimension (C102) reads cpu_ms from worker/wrangler.toml and fails when required scenarios exceed 0.5×cpu_ms. scripts/lib/telegram-load-guard.mts owns the reviewed dependency triggers for local and CI execution. |
| Telegram group admin membership cache | 5 minutes for cached denial/admin-list copy; mutating auth revalidates per webhook | worker/src/lib/telegram/chat-member.ts, worker/src/api/telegram-webhook-auth.ts | /subscribe, /unsubscribe, /set in group/supergroup chats use a fresh getChatMember lookup for mutation authorization and fail closed if Telegram cannot confirm admin status. Soft-launch warnings and denial copy can still use the cached getChatAdministrators list (telegram:chat-admins:<chat_id>). These calls happen on webhook ingress, not inside the dispatch cron lane. |
| Live reserve history retention | 30 days | worker/src/lib/live-reserves/store-shared.ts, worker/src/lib/live-reserves/store-write.ts | reserve_composition_history and reserve_sync_attempt_history are pruned during reserve-sync cleanup |
| Redemption route-status producer | D1-free/static plus live-reserve metadata | worker/src/lib/redemption-backstop/route-status.ts, worker/src/cron/sync-redemption-backstops.ts | v4 route status remains four-hour snapshot data. sync-redemption-backstops does not add outbound route-status feed fetches; route availability comes from existing live-reserve adapter metadata, reviewed static policy, and market-implied severe-depeg overlays. |
Blacklist scan budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| Blacklist sync runtime budget | 10 minutes | worker/src/cron/sync-blacklist.ts | Guardrail below the 12-minute trigger wrapper timeout |
| Blacklist sync subrequest budget | 900 | worker/src/cron/sync-blacklist.ts, worker/src/lib/evm-logs.ts | Covers explorer/RPC calls for a single run |
| Blacklist provider pacing/concurrency | 3 requests/second, 1 live request | worker/src/cron/sync-blacklist.ts, worker/src/lib/evm-logs.ts | The serial limiter's declared concurrency matches the live connection posture; throughput and live connections are reported separately in cron metadata |
| Blacklist RPC topic aggregation | One OR-topic eth_getLogs request per range when supported | worker/src/cron/blacklist/evm-source.ts, worker/src/lib/alchemy-logs.ts | Required topic0 signatures share one completeness frontier; explorer fallback remains per-topic where OR filters are unavailable |
| Blacklist Alchemy split cap | 64 provider calls per log scan, depth 8 | worker/src/lib/alchemy-logs.ts | Split traversal is sequential and also bounded by deadline and the shared 900-subrequest budget; hitting any cap returns incomplete coverage without advancing beyond proof |
| Blacklist Arbitrum scan window | 25,000,000 explorer/Alchemy-primary blocks; 250,000 fallback-RPC blocks per config/run | worker/src/cron/blacklist/evm-source.ts | Every topic shares the minimum proven frontier; the stored cursor never exceeds the 15-minute safe head |
| Blacklist amount-recovery batch | 100 queued rows / pass | worker/src/lib/blacklist/amount-recovery.ts, worker/src/lib/blacklist/amount-repair-queue.ts | Priority/retry state is durable; event and queue outcomes are paired in shared D1 batches and stay under the sync subrequest/runtime budget |
Mint-burn and producer metadata budgets
| Area | Current repo budget | Source | Notes |
|---|---|---|---|
| Mint/burn global request budget | 200 | worker/src/cron/sync-mint-burn.ts | Shared per-run request ceiling |
| Mint/burn D1 retention | events: 8 days, 10,000/batch, 50,000/critical run; hourly: 95 days, 10,000/batch, 25,000/critical run | worker/src/cron/mint-burn/retention.ts, worker/src/cron/sync-mint-burn.ts | Oldest-first cleanup runs only in the critical producer. Event deletion requires settled valuation, an existing stablecoin/chain/hour aggregate, and a persisted Tape cursor at or beyond the event timestamp; protected debt can outlive the nominal cutoff. Cleanup errors degrade successful ingestion and remain visible in cron metadata. |
| Mint/burn runtime self-budget | 9 minutes; 60 seconds minimum next-config window | worker/src/cron/mint-burn/run-configs.ts, worker/src/lib/alchemy-logs.ts | Guardrail below the 10-minute cron wrapper timeout so the runner can stop before starting a config that would not leave enough persistence/logging tail room. Mint/burn eth_getLogs calls receive the same runtime deadline so a started config cannot continue log-scan recursion past the self-budget. |
| Mint/burn conservation pre-pass | 100 calls/POST; 15 seconds/POST; 45 seconds/run | worker/src/lib/mint-burn-conservation.ts, worker/src/cron/mint-burn/run-configs.ts | Three serialized phases per chain; cost is the sum of each phase's chunks against the global 200-request ceiling. Headers are deduplicated by block; supply calls remain per contract/boundary. Per-config cursor fences are unchanged. |
| Mint/burn per-config budget (critical) | 60; 150 for bridge-aware critical configs | worker/src/cron/sync-mint-burn.ts | Prevents one hot config from consuming the full run while allowing bridge-aware high-volume configs enough tx-context headroom |
| Mint/burn tx-context batch size | 20 tx hashes/request; 3 concurrent batch requests | worker/src/lib/mint-burn-pipeline/classification.ts | Keeps bridge-aware classification under the Worker connection pool and avoids one HTTP request per parsed USDC-style mint/burn transaction |
| Mint/burn per-config budget (extended) | 25 | worker/src/cron/sync-mint-burn.ts | Lower ceiling for long-tail backlog drain |
| Mint/burn max scan range | 50,000 blocks | worker/src/cron/sync-mint-burn.ts | Keeps per-request log scans bounded |
| Mint/burn timestamp resolution scope | Non-dust candidate logs only | worker/src/cron/mint-burn/sync-config.ts | Dust-only blocks are excluded before eth_getBlockByNumber lookups so irrelevant logs cannot block sync-state advancement |
Mint/burn SQL IN chunk size | 90 ids | worker/src/cron/sync-mint-burn.ts | Current safeguard for large batched SQL |
| Mint/burn event insert batch size | 50 statements | worker/src/lib/mint-burn-pipeline/persistence.ts | Each insert binds 18 values; chunked to stay below D1 batch bind ceilings |
| Mint/burn extended attempt SLO | 2 due runs / 75 minutes | worker/src/cron/mint-burn/run-state.ts | The next run resumes at the first capacity-deferred config; active provider deferrals are explicitly exempted until their recorded expiry |
| Stablecoin producer metadata ceiling | <60 KiB before scheduler enrichment; <64 KiB persisted | worker/src/cron/sync-stablecoins/metadata.ts, worker/src/lib/cron-metadata-persistence.ts | The stablecoin producer reserves 4 KiB for wrapper-owned lease, slot, invocation, and timeout metadata. Its size guard compacts diagnostics before that enrichment so top-level publication and active-price coverage survive unchanged into cron_runs; because the compacted shape still scales with the missing-asset count, the guard enforces the budget through an ordered degradation ladder (duplicate ledger missing-ID list, verbose gap details, attempt records — counts and truncation markers retained) and fails closed with a scalar-only envelope marked sizeGuardEvidenceDropped when no rung fits. The global <64 KiB persistence guard remains the final fallback. |
Per-Job Cron Timeouts
Canonical per-job wrapper-timeout roster. Each cron job receives an AbortSignal from logCronRun() that fires after its CRON_TIMEOUT_MS entry; a job that exceeds it is aborted and logged with status='error', and the signal is threaded through to fetchWithRetry() so in-flight HTTP requests are also cancelled. Wrapper timeouts are wall-clock safety caps, distinct from lease heartbeats, slot deadlines, and the cpu_ms CPU class; docs/worker-infrastructure.md owns those mechanism distinctions and links here for the roster. Values are derived from CRON_JOB_DEFINITIONS plus CRON_TIMEOUT_OVERRIDES_MS in worker/src/lib/cron-timeouts.ts (jobs without an override use the 5-minute default), and the scheduled-runner contract test fails when a new status-tracked job would fall back implicitly.
| Job | Timeout | Reason | Sources |
|---|---|---|---|
| Default (no override) | 5 min | Standard jobs complete in <60s | worker/src/lib/cron-timeouts.ts (DEFAULT_CRON_TIMEOUT_MS) |
sync-stablecoins | 8 min | Core quarter-hour pipeline entrypoint now includes N-source weighted primary pricing, supplemental overlays, multi-pass enrichment, and depeg processing; explicit headroom avoids timing out on bounded fallback work | worker/src/lib/cron-timeouts.ts |
sync-dex-liquidity-stage | 13 min | External source loading and pool-graph construction, with headroom to persist the bounded generation | worker/src/lib/cron-timeouts.ts |
sync-dex-liquidity | 13 min | D1-only scoring, proof joins, and generation-fenced liquidity/price/history publication | worker/src/lib/cron-timeouts.ts |
sync-dex-discovery | 13 min | Multi-source pool staging with explicit 12-minute self-budget so the wrapper still has headroom to log a controlled degraded/error result. Every provider body is read through a bounded reader with an explicit per-response cap (512 KiB for depth=true CoinGecko tickers, 2 MiB for CoinGecko onchain pool pages and DexScreener token-pair payloads), and the sweep yields to the event loop between coins so the slot fence heartbeat and child lease renewal keep flowing on a busy run | worker/src/lib/cron-timeouts.ts |
reserve-recovery | 13 min | Isolated five-minute reserve interruption-recovery lane at a 2/6 connection peak. WORKER_RESERVE_RECOVERY_MODE=recover enables compatible suffix/sidecar replay; other values keep only stale-slot sweeping. Targeted reserve-slot recovery waits for five minutes without a heartbeat (five heartbeat intervals) before reconciliation; a heartbeat younger than that remains live. Before eligibility inspection, up to 25 incompatible running/recovering/ready/platform-abandoned checkpoints are retired atomically only after a newer current-hash checkpoint completed the full configured queue and its slot finished; aggregate slot degradation does not block supersession. Unexpired child/recovery leases and fresh slot heartbeats block retirement. The transition fences exact pending attempts without deleting leases. Remaining incompatible debt with zero ready/eligible work reports degraded liveness rather than silently returning healthy. | worker/src/handlers/scheduled/reserve-recovery.ts, worker/src/lib/cron-timeouts.ts |
cron-sentinel | 13 min | Declarative freshness sentinel and repair-debt lane that replaced seven retired watchdog jobs; D1-only work with zero outbound connections, so the ceiling only guards slow D1 reads/writes across its lanes | worker/src/lib/cron-timeouts.ts |
sync-blacklist | 12 min | Multi-chain scan + balance enrichment; isolated trigger allows extended runtime | worker/src/lib/cron-timeouts.ts |
sync-live-reserves | 12 min | Multi-adapter reserve fetching with per-adapter timeouts; explicit wrapper budget for the serialized reserve loop before the rest of the 4-hourly slot | worker/src/lib/cron-timeouts.ts |
sync-mint-burn | 10 min | Multi-contract EVM log scan; isolated trigger allows extended runtime, with a 9-minute internal guard before the wrapper timeout | worker/src/lib/cron-timeouts.ts |
sync-mint-burn-extended | 10 min | Long-tail mint/burn lane with its own run-state and the same 9-minute internal guard | worker/src/lib/cron-timeouts.ts |
sync-yield-data | 10 min | Multi-source yield data aggregation; dedicated post-V9 timeout after moving off the half-hourly lane | worker/src/lib/cron-timeouts.ts |
sync-yield-supplemental | 12 min | Supplemental yield source sync on the 4-hour lane; dedicated timeout for optional protocol families, covering more sources per invocation | worker/src/lib/cron-timeouts.ts |
snapshot-public-dataset | 10 min | PUBLIC_DATASET_CRON_TIMEOUT_MS: the 8-minute stablecoins-cache retry window plus 2 minutes for D1 reads, gzip/hash work, and the immutable insert | worker/src/lib/cron-timeouts.ts, worker/src/lib/public-dataset-snapshot-budget.ts |
sync-cl-exit-depth | 9 min | Score-bearing measured-execution lane; the producer stops RPC work at eight minutes, leaving a minute to publish a complete quote generation and record the cron result. A five-minute pacing projection additionally stops starting further quote stages when observed provider throughput cannot finish the remaining ladder/refinement work by then, so a degraded provider publishes a partial generation and releases the lease instead of contending with the D1-heavy :10/:13/:16 lanes for the full budget (MAX_PACED_QUOTE_RUNTIME_MS in worker/src/cron/measured-execution/sync.ts) | worker/src/lib/cron-timeouts.ts, worker/src/cron/measured-execution/sync.ts |
dispatch-telegram-alerts | 4.5 min | Dedicated five-minute lane with a 4-minute send-loop soft deadline, 30 seconds for durable finalization, and a 30-second lease heartbeat; pending-drain and fresh-send loops stop starting Telegram batches near the four-minute mark, releasing pending claims or queueing the untouched fresh tail so slow Bot API runs yield the next trigger interval. The shorter crash fence prevents an unabortable operation from suppressing several later slots | shared/lib/telegram-delivery-policy.ts (TELEGRAM_DISPATCH_TIMEOUT_MS), worker/src/lib/cron-timeouts.ts, worker/src/handlers/scheduled/context.ts |
daily-digest | 14 min | Expanded LLM generation + persistence/distribution, still below the 15-minute scheduled-trigger ceiling | worker/src/lib/cron-timeouts.ts |
weekly-recap | 14 min | Weekly Anthropic recap in the daily 08:10 UTC slot, with an in-code Monday UTC gate; uses the same Anthropic budget and two-channel publication tail as daily, so the wrapper preserves the same post-LLM headroom | worker/src/lib/cron-timeouts.ts, worker/src/cron/weekly-recap.ts |
docs/worker-infrastructure.md links here instead of repeating this roster.
D1 overload retry posture
Cron persistence helpers retry transient D1 queue pressure through runWithOverloadRetry() in worker/src/lib/d1-overload-retry.ts. Retried errors include D1 DB is overloaded, Requests queued for too long, D1 storage-operation reset timeouts (D1 DB storage operation exceeded timeout ...), Cloudflare D1 internal-reference errors (D1_ERROR: internal error; reference = ...), and transient D1 transport loss (Network connection lost). Retry backoff is abort-aware, and batchExecute() can accept an AbortSignal so chunked persistence can stop before the next D1 batch after cron timeout or lease loss. Every status-tracked job in shared/lib/cron-jobs.ts must have an explicit CRON_TIMEOUT_MS ceiling; the scheduled-runner contract test fails when a new cron job would fall back implicitly. If a leased job ignores timeout or lease-loss aborts, runCronWithLease() records an abandoned failure and leaves the lease row to expire by TTL instead of releasing it while late writes may still be running. Long-job lease heartbeats renew every 30 seconds and abort controlled work after three consecutive renewal failures. The duration watchdog excludes stale-slot reconciled synthetic child rows from runtime averages, keeps 7-day runtime totals for diagnostics, but requires repeated cap hits or unrecovered runBudgetTruncated rows (those without a persisted deferral cursor, cursorTailState <> 'complete') to reach the alert threshold inside the 24-hour recency window — cursor-complete truncations are the designed graceful-deferral path and no longer alert; cron_slot_executions remains the separate scheduled-slot abandonment signal. DEX source-stage chunks use deterministic one-row direct conflict upserts with progress writes every 24 chunks plus the final interval, and an ambiguous final manifest update is accepted only after an exact readback proves the ready/consumed schema, slot, chunk, record, and byte totals. DEX-liquidity publication streams 15 candidate rows at a time into five three-row statements, staying below D1's 100-bind ceiling while reducing statement and batch-call overhead; it still validates the complete per-run generation before updating the public current table. Live reserve, redemption-backstop, cache-sentinel, and DEWS persistence paths should route bursty run-manifest writes, cleanup, prune, and chunked batch work through this helper or batchExecute() so one transient D1 queue spike does not fail a whole scheduled run.
D1 query budgeting
Public health/status diagnostics intentionally trade a few minutes of operator telemetry freshness for lower D1 pressure:
queryBlacklistGapMetrics()supports a core diagnostic mode that skips the amount-status and amount-source distribution scans when callers only need total/recoverable/recent gap counts for health scoring.sync-blacklistmaterializes producer-owned blacklist gap snapshots into the existing D1cachetable only after every required config completes successfully in the run, no state CAS conflicts occur, and enough snapshot budget remains. Any skipped or incomplete config degrades the run and withholds publication. The snapshot timestamp is the oldest required config's successful-scan time, preventing a recent cron finish from masking a stale source cohort. It writes both full summary metrics and core health/status metrics from one live full query./api/health,/api/status, and/api/blacklist-summaryprefer those producer snapshots, then fall back to the short request cache and finally the live query if the producer row is missing or stale./api/blacklist-summaryalso has a producer-owned summary payload snapshot in the same D1cachetable. The handler preserves the public response shape, serves a fresh producer payload directly, serves a stale-but-present producer payload without rewriting the producer cache from public traffic, and falls back to live construction only when no valid snapshot has ever been written. Freshness headers are based on the producer snapshot timestamp, so stale-but-served fallback snapshots still emit the normalWarningheader once they exceed the blacklist summary max age.- The short 5-minute D1 request cache remains only as a fallback pressure valve. It should not be treated as the freshness authority for blacklist health; producer materialization is the preferred source when present.
- Cron health reads the latest 10 runs per tracked job through the existing
(job, started_at DESC)index instead of applying a window function across the retainedcron_runstable. - Tron current-balance refresh does not populate historical event amounts. The operator-only transfer-replay repair accepts at most eight exact unresolved events, with confirmed block-bound raw balances, matching freeze receipts, exhaustive bounded transfer history, and a complete-history balance reconciliation. It uses one guarded atomic D1 import for the audit, event updates, and derived-cache invalidation; see Blacklist Tracker for evidence and freshness requirements.
- DEWS producers maintain
stress_signals_latestas the full latest-row materialization for Telegram dispatch and smoothing, sparsestress_signalshistory at least hourly plus material changes, and two exact generations instress_signal_publication_rows. Pointer-bound public and PSI reads verify the exact buffered generation; a partial newer write cannot replace the previous proof. A missingdews:published-generationpointer is treated as first-run bootstrap and can use the unbounded latest-row fallback; corrupt or unreadable publication pointers return unavailable current rows instead of silently switching to an unbounded generation. - The active baseline contains the chain/hour
mint_burn_hourly, stablecoin/time yield, depeg pagination/open-event indexes, andstress_signals_latestmaterialization introduced by historical migration 0144. Keep matching SQL comments stable so D1 insights can attribute reads to source-owned query families; use the migration manifest for lineage. - D1 capacity samples are hourly and bounded. Utilization boundaries, the growth-regression windows, and the conservative exhaustion forecast are documented in
docs/operator procedures/d1-capacity-and-runtime-experiments.md. The scheduled status lane refreshes the control-plane observation; public health reads only the bounded cached assessment. - The five-minute Telegram lane reuses same-run pending-capacity and safety-source reads between dispatch, degradation watchdog, and pulse publication. Healthy no-change dispatch runs skip fan-out and fresh-send assembly while preserving pending drain, TTL cleanup, snapshot freshness, and watchdog-visible metadata. Source-event runs capture only direct/preset/global candidates relevant to the active families, reuse unchanged page fan-out inputs between eligibility and routing, and emit nested
authoritativePlanningphase timings/counts in dispatch metadata. - Telegram registration reconciliation still keeps separate webhook, command, profile, and menu cache checks so each helper remains directly callable and testable. Coalesce those cache reads only with a helper that preserves the current standalone semantics and Bot API rate-limit backoff behavior.
- Cron progress writes still go through the shared scheduled-runner logging path. There is no Telegram-specific progress-write opt-out today; add a generic, documented opt-out before reducing progress persistence for one lane.
When adding a new health/status loader, prefer a producer-cadence cache or an indexed top-N read over table-wide aggregates. Table scans are acceptable for low-frequency admin-only diagnostics, but not for public health probes, status self-checks, or same-origin browser polling.
Large cache-backed endpoints can opt into a response-ready companion cache when the producer has already validated the canonical payload. /api/stablecoins writes stablecoins:response-ready:v2 alongside the canonical stablecoins cache row after schema validation; the reader serves that raw body only when both rows share the same updated_at, then injects freshness metadata without reparsing the full payload. Companion rows are best-effort optimizations: read or write failures fall back to the canonical cache path and should not fail the producer. Endpoints with live transforms should not use this shortcut unless the companion body already includes those transforms.
Upstream Fetch Budgets
Discovery and price-provider budgets
| Path | Current repo throttle / budget | Source | Notes |
|---|---|---|---|
| CoinGecko onchain discovery | 250 ms between requests | worker/src/lib/rate-limit.ts | Used by discovery crawlers |
| CoinGecko onchain stale-pool refresh | 250 ms between requests; /onchain/networks/{network}/pools/multi/{addresses} with 30 addresses/request | worker/src/cron/dex-discovery/refresh-stale-pools.ts, worker/src/lib/coingecko-onchain.ts | Same key, circuit (coingecko-onchain), and 2 MiB bounded body reader as the discovery CG stage; 30 is the documented every-plan ceiling |
| CoinGecko onchain discovery stage timeout | 8 seconds per crawl stage | worker/src/cron/dex-discovery/staged-pool.ts | DISCOVERY_STAGE_TIMEOUT_MS.cgOnchain, inside the 25 s per-coin and 12 min run budgets |
| CoinGecko backfill throttle | 200 ms between requests | worker/src/lib/rate-limit.ts | Used by CoinGecko backfill/admin flows |
| CoinGecko native-peg quotes | 50 ids/request, 10 s timeout/request, 1 retry; one shared batch per (vs_currencies, id set) per stablecoin sync run | worker/src/lib/native-peg-quotes.ts, worker/src/cron/sync-stablecoins.ts | createNativePegQuoteSession() memoizes each batch response for the run so native-peg hardening, depeg detection, and pending-depeg confirmation reuse one fetch instead of one each; non-2xx responses, malformed payloads, and transport failures are never memoized |
| GeckoTerminal crawl throttle | 2000 ms between requests | worker/src/lib/rate-limit.ts | Conservative crawl pacing |
| GeckoTerminal discovery stage timeout | 8 seconds per crawl stage | worker/src/cron/dex-discovery/staged-pool.ts | DISCOVERY_STAGE_TIMEOUT_MS.geckoTerminal, inside the 25 s per-coin and 12 min run budgets |
| Protocol-redeem live override stage | 10 s total budget on up to 4 parallel candidate lanes (the sync-stablecoins maxConnections declaration); grouped circuit for most external live RPC-backed providers, with dedicated Kava and AZND circuits | worker/src/lib/authoritative-price-sources/index.ts | Missing-price candidates run before already-priced candidates and provider families are interleaved within each partition. Lanes pull candidates in that same priority order and take the next candidate as soon as one frees, so one slow chain route cannot serialize ahead of the local/cache-backed repairs and the shared budget is spent on work rather than queue order. Missing-only thin routes are excluded when a usable incumbent exists. Local par and inherited tracked-base overrides are cache/local decisions; circuit-open RPC providers fail closed. |
| Kava USDX oracle adapter | 4 sequential requests, 2.2 s timeout/request, 0 retries, 256 KiB max response/request; inside the shared 10 s live-override budget | worker/src/lib/authoritative-price-sources/kava-pricefeed.ts | Reads head, market, aggregate, and raw-oracle state serially. Wrong chain/market/oracle identity, a head older than 2 minutes, insufficient oracle expiry, or excessive dispersion fails closed. |
| Reviewed exact price routes | Inside the shared 10 s live-override budget; adapter-specific bounded RPC calls and executable notionals | worker/src/lib/authoritative-price-sources/ | sAID and AZND pin exact contracts/tokens and reject stale state, identity drift, missing trusted parents, insufficient capacity/depth, route-specific impact/divergence failures, failed public redemption, or RPC failure. AZND calls stay serial to preserve connection headroom. |
| DexScreener / CG-tickers fallback stage timeout | 6 seconds per fallback stage | worker/src/cron/dex-discovery/staged-pool.ts | DISCOVERY_STAGE_TIMEOUT_MS.dexscreener and .cgTickers; both late-stage fallbacks share the same per-coin deadline |
| Jupiter price fallback | 50 ids/request, 5 s timeout/request, 0 retries; up to 25 low-depth primary augmentation targets/run; at most 3 sequential 3 s slot-RPC probes/pass | worker/src/cron/sync-stablecoins/enrich-prices-jupiter-pass.ts | Solana-only enrichment pass between CMC and DexScreener; can append agreeing Jupiter evidence to low-depth primary prices and fails closed when every bounded slot reference is unavailable |
| DexScreener price-enrichment pass | 1 request/run, up to 30 same-chain addresses/request, 5 s timeout, 45 s total budget, 0 retries | worker/src/cron/sync-stablecoins/enrich-prices-dexscreener-pass.ts | Best-effort final fallback for missing prices through exact token-address lookups; chains and large within-chain cohorts rotate, symbol search is retired, and hard 429 / WAF 1015 refusals end the pass |
| Address-price augmentation group | CoinGecko Onchain exact-address provider enabled in production; 90 s shared group budget, 5 s timeout/request, 0 retries | worker/src/lib/address-price-providers/index.ts, worker/wrangler.toml | Runs during primary pricing for assets with missing prices, low-confidence prices, or previous source depth below 3. Observations expiring before the next generation order targeting cohorts but are not an inclusion reason. Durable target cursors rotate only inside missing, expiring, low-depth, and remaining-priced cohorts. The public GeckoTerminal corroboration pass is not run inline because its repeated public errors and response retention exceeded the Worker memory boundary; its address-price provider has since been removed entirely. |
Metered supplemental provider budgets
| Path | Current repo throttle / budget | Source | Notes |
|---|---|---|---|
| Dwellir supplemental RPC plan (plan-specific) | Developer plan: 25,000,000 JSON-RPC responses per UTC month, 100 responses/second, eth_getLogs range ≤ 500 blocks inclusive per request, JSON-RPC batch ≤ 100 items; overage billed at $5/1M responses. debug_/trace_ are available on the plan and unused by the Worker. | Dwellir plan terms (vendor-owned, not this repo) | Plan-specific row, deliberately not the repo's authority: vendor quotas and prices change independently of this repo, so verify the tier, included quota, and any overage stop in the vendor dashboard before spend-sensitive changes. The Worker never reads the plan; it enforces the repo cap in the next row instead. |
| Dwellir repo credit cap and ledger | DWELLIR_DEFAULT_MAX_CREDITS_PER_MONTH = 20,000,000 credits per UTC month (80% of the plan's included quota, so the app stops before overage billing starts), overridable by a positive-integer DWELLIR_MAX_CREDITS_PER_MONTH; ledger row rpc:dwellir:credits:v1:<YYYY-MM> in the cache table | worker/src/lib/rpc-provider-budget.ts | This constant — not the vendor quota — is the authority the Worker enforces. One credit per JSON-RPC response item, including errors; increments compare-and-swap on the monthly row with bounded retries; an unreadable or corrupt row fails closed (usable: false, credits reported as null); over cap or unreadable omits Dwellir from every built config, and a failed flush returns the drained credits to the isolate-local pending counter. DWELLIR_API_KEY is the kill switch: without it the ledger is never even read. |
| Dwellir supplemental failover and observation lane | Appended strictly after registry endpoints and adapter pins (last position); a timeout or network failure demotes that origin for the rest of the run; the observation lane costs about 29 chains × at most 3 Dwellir requests per hourly run ≈ 2,000 credits/day. | worker/src/lib/chain-registry.ts, worker/src/lib/evm-rpc.ts, worker/src/cron/rpc-provider-parity/probe.ts | Scheduled runtimes build supplemental endpoints only when the key is present, the ledger is usable, and the dwellir-evm circuit is closed; routes never enable them, and the circuit is non-public-impacting. Dwellir HTTP 403 is reported as provider-capability and never retried. Blacklist, mint/burn conservation, and readers with their own fetch loops select registry endpoints only, so log-lane history and the 500-block plan cap cannot silently truncate a scan. |
| vaults.fyi supplemental yield | 13 estimated credits/run on the four-hour lane; 2,500 estimated credits/UTC month | worker/src/cron/yield-sync/vaults-fyi.ts | Generation-fences one reservation owner with compare-and-swap, finalizes only the matching owner, dynamically throttles to the sustainable remainder, and fails closed before paid requests when the monthly ledger is corrupt; no provider-authoritative usage counter is ava… |
Market feeds and DEX fetch budgets
| Path | Current repo throttle / budget | Source | Notes |
|---|---|---|---|
| Coinbase CEX ticker products | 1 product request in flight, 10 s timeout/request, 1 retry | worker/src/lib/cex-tickers.ts | Keeps the Coinbase thunk inside the sync-stablecoins primary-provider cap instead of multiplying the quarter-hourly trigger fanout |
| Secondary FX mirror race | 3 mirror requests in parallel | worker/src/cron/sync-fx-rates-sources.ts, shared/lib/cron-jobs.ts | jsDelivr @latest, direct Pages mirror, and date-pinned jsDelivr package race inside the sync-fx-rates max connection declaration |
| Chainlink FX/commodity overlay | 3 feed pipelines in parallel | worker/src/lib/chainlink-feeds.ts, shared/lib/cron-jobs.ts | Keeps the five-feed overlay inside the sync-fx-rates three-connection cron metadata while preserving bounded parallel recovery |
| CoinGecko Onchain address augmentation | 5 requests/run, 30 addresses/request | worker/src/lib/address-price-providers/index.ts | Keyed exact onchain token lookup, separate from the serialized GeckoTerminal pool probe |
| CoinMarketCap fallback | Up to 2 requests in an eligible hour: 1 category request plus 1 targeted request for at most 25 rotated unresolved slugs; 10 s timeout/request, 0 retries | worker/src/cron/sync-stablecoins/enrich-prices-cmc-pass.ts | Usable returned category rows survive an unseen tail. Targeted quotes require exact identity, active status, freshness, positive volume, required supplied-contract agreement for known deployments, and peg validation. Verified targeted rows preserve their upstream timestamp through the one-hour cooldown; 429 still honors Retry-After. |
| DEX primary source JSON reads | 30 s per attempt through fetchJsonWithRetry(); Curve API fan-out capped at 4 chain requests | worker/src/cron/dex-liquidity/fetch-primary.ts, worker/src/lib/concurrency.ts | DeFiLlama Yields and Protocols are consumed before the bounded Curve phase starts, keeping Curve below the declared DEX job peak. The defillama-protocols cache stores a compact slug/category snapshot for yield coverage audits, and raw Curve response trees are released once their derived lookup maps are built |
| Direct DEX API fetch phase | 1 protocol family at a time, 5 nested connection static peak within a provider, 90 s provider timeout wrapping 15 s per-request timeouts, deterministic page caps (50 default); Fluid ticker retries cap provider Retry-After sleeps at 5 s per retry | worker/src/cron/dex-liquidity/orchestrator-phases/direct-api.ts, worker/src/cron/dex-liquidity/direct-api-policy.ts, worker/src/cron/dex-liquidity/direct-api-paginated.ts, protocol fetchers, shared/lib/cron-jobs.ts | Runs hourly at :10 inside sync-dex-liquidity-stage through a direct-API-local circuit/timeout wrapper; mapWithConcurrency(..., 1, ...) owns serial provider execution, and each result is reduced to tracked pools plus compact raw counts and authoritative exact keys before the next family starts. check:cron-connections enforces the declared 5/6 peak because one provider can use the nested width. |
| Measured DEX execution RPC lanes | Active: 3 EVM chain lanes every 30 minutes with the existing 1,300 request, 6,400 quote-subcall, and 8-minute caps. Shadow: shadow-only EVM once daily. | worker/src/cron/measured-execution/sync.ts, shared/lib/cron-jobs.ts | The isolated physical aliases 5 * * * * and 35 * * * * run the active 3/6 EVM lane while retaining logical :00/:30 slot identity. Shadow target inventories publish at 06:16 UTC; the shadow EVM collector runs at 08:10 UTC (the Solana and Tron native lanes were removed in Liquidity Score v6.0). Shadow generations never enter the active scorer. Quote publications omit manifest-proven budget-deferred rows and reconstruct them on read, while measured outcomes and real failures remain durable. |
| Shared retry JSON/text response body | 16 MiB default; configurable per request with maxResponseBytes | worker/src/lib/fetch-retry.ts, worker/src/lib/response-body.ts | fetchJsonWithRetry() and fetchTextWithRetry() reject declared or streamed overflow, cancel the body, and retry through the existing warning path; JSON is parsed only after a complete in-cap body. Raw fetchWithRetry() responses remain caller-owned and uncapped. Callers that read a raw body themselves must read through readResponseTextWithinLimitWithSignal() with an explicit cap; the discovery providers do exactly that — CoinGecko onchain pool pages and DexScreener token-pair payloads at 2 MiB, depth=true CoinGecko tickers at 512 KiB — so a mis-served response fails that one provider check instead of being buffered and parsed into the isolate. |
| Generic circuit breaker | <!-- GENERATED-START: worker-circuit-breaker-limit -->opens after 3 consecutive failures, probes every 30 minutes<!-- GENERATED-END: worker-circuit-breaker-limit --> | worker/src/lib/circuit-breaker.ts | Used to stop hammering degraded upstreams |
JSON/text fetch callers that need per-request timeout coverage across body consumption should use fetchJsonWithRetry() or fetchTextWithRetry() rather than calling fetchWithRetry() and then consuming the returned Response separately. Provider execution wrappers (providerJson() and providerTextBounded()) keep their provider timeout active through body reads and record body-read timeouts as provider failures.
Longer sequential failover chains cost latency, not correctness. Each extra URL in a chain pays its own timeout plus one network retry before the next endpoint is tried (worker/src/lib/evm-rpc.ts defaults to a 10 s timeout and maxRetries: 1 in network-only mode, so one exhausted endpoint is on the order of 20 s), and a read that fails over across every operator pays that for each one. The Dwellir trial is mitigated twice: it is appended last, so it only runs where every reviewed endpoint already failed, and a timeout or network failure demotes that Dwellir origin for the remainder of the run, so one slow Dwellir minute cannot multiply across the rest of a lane's reads.
Response-Body Limits
fetchJsonWithRetry() / fetchTextWithRetry() default to a 16 MiB body cap, and the DEX source stage names a tighter cap for every source it reads so that one mis-served response (HTML error page, doubled payload, proxy interstitial) cannot be buffered and parsed inside the 128 MB isolate. Caps are stated per source because the legitimate shape differs by an order of magnitude between a ticker list and a whole-catalog catalog payload.
Measured 2026-09-23 by calling the same public endpoints from a workstation with the repository's own query and page parameters (worker/src/cron/dex-liquidity/constants.ts query builders, pageSize/page_size values as configured). The The Graph gateway refused the 2026-09-23 measurement without GRAPH_API_KEY, so the Uni V3 / V4 / PancakeSwap page caps were first justified against the measured page budget of the same 1,000-row shape; the 2026-09-27 subgraph-lane repair re-measured the credentialed gateway directly with the repository's own key and confirmed the estimate (table row below).
| Source (Worker call site) | Endpoint shape | Measured | Cap | Constant |
|---|---|---|---|---|
| DeFiLlama Yields | GET /pools | 11,816,252 B / 17,188 pools | 16 MiB (≈1.4x) | DEFILLAMA_YIELDS_MAX_RESPONSE_BYTES (fetch-primary.ts) |
| DeFiLlama Protocols | GET /protocols | 8,911,702 B / 8,339 rows | 12 MiB (≈1.4x) | DEFILLAMA_PROTOCOLS_MAX_RESPONSE_BYTES (fetch-primary.ts) |
Curve getPools/all/:chain | 14 chain bodies, 4 at a time | 4,806,780 B (ethereum, largest) | 8 MiB (≈1.75x) | CURVE_MAX_RESPONSE_BYTES (fetch-primary.ts) |
| Uni V3 / Uniswap V4 / PancakeSwap subgraph pages | first: 1000 pool page | measured 2026-09-27 with GRAPH_API_KEY against the pinned Base deployments: Uni V3 574,513-574,539 B, Uniswap V4 462,509 B per 1,000-pool page (2026-09-23 estimate of 0.5-1 MB confirmed) | 8 MiB (≈14x) | SUBGRAPH_PAGE_MAX_RESPONSE_BYTES (constants.ts) |
| Direct-API paginated runner default | any paginated provider | Raydium pageSize=1000 2,138,442 B; Meteora page_size=500 995,192 B; Balancer 1,000-row list page 182,322 B for 246 rows | 8 MiB (≈3.9x) | DIRECT_API_DEFAULT_MAX_RESPONSE_BYTES (direct-api-policy.ts) |
Raydium pools/info/list | pageSize=1000 | 2,138,442 B (standard), 2,013,086 B (concentrated) | 8 MiB (≈3.7x) | RAYDIUM_MAX_RESPONSE_BYTES (fetch-raydium.ts) |
Meteora dlmm.datapi.meteora.ag/pools | page_size=500 | 995,192 B / 500 rows | 4 MiB (≈4x) | METEORA_MAX_RESPONSE_BYTES (fetch-meteora.ts) |
Balancer api-v3.balancer.fi GraphQL | first: 1000 pool page | 182,322 B / 246 pools (≈740 B/row) | 4 MiB | BALANCER_MAX_RESPONSE_BYTES (fetch-balancer.ts) |
Orca api.orca.so/v2/solana/pools | size=200 | 671,680 B / 200 rows (declared 162,596 B compressed) | 4 MiB (≈6x) | ORCA_MAX_RESPONSE_BYTES (fetch-orca.ts) |
| Fluid tickers | GET /v2/:chainId/dexes/stats/tickers | 15,029 B (ethereum, largest of six chains) | 256 KiB (≈17x) | FLUID_MAX_RESPONSE_BYTES (fetch-fluid.ts) |
Overflow semantics (R1-R8: unavailable is not zero, a failed read never becomes a positive claim):
- The reader rejects a declared over-cap
Content-Lengthbefore reading and otherwise streams and aborts at the cap, so an over-cap body is never parsed and never becomes partial data; the connection returns to the pool immediately. - The DEX stage turns the overflow into that source's own failure:
fetchSubgraphEntities()reportsfailed: truewith the machine-readablefailureReason: "body-over-cap"(also"http"/"graphql"),readDexApiJson()returns"<context> response body exceeded <n> bytes", andfetchFluidPools()recordsfluid <chain> response body exceeded <n> bytes. Consumers keep the existing degraded/failed-source accounting (failedSources,criticalSourceFailures, circuit outcomes) instead of publishing a short pool list. worker/src/lib/response-body.tsexportsResponseBodyTooLargeErrorandisResponseBodyTooLargeError()so callers can classify the overflow without string matching;fetchJsonWithRetry()remains the owner of the cap check, retries, and body cancellation.- Callers that must classify the final failure pass
throwOnFinalNetworkError: true(DEX subgraph pages, Fluid tickers, PancakeSwap subgraph pages); the retry loop still retries the earlier attempts, and the last attempt's typed error reaches the caller's failure path.
Long synchronous work in these phases yields between pages (yieldToEventLoop() in the subgraph page loop and the direct-API pagination loop) so slot heartbeats and abort timers keep firing during a multi-page fan-out instead of waiting for the whole provider family to drain.
Why the DEX caps exist. A mid-run isolate death leaves no terminal write, so the slot is reconciled as platform-abandoned roughly five minutes later and the cron surfaces as red with no source-level diagnostic. On 2026-09-23 at 00:10:15 UTC Cloudflare recorded an exceededMemory invocation outcome that killed sync-dex-liquidity-stage during subgraph-enrichment (cpu 4.69 s, duration 8.69 s, 186 subrequests, reported peak allocation 171.8 MB against the 128 MB isolate limit); the reconciliation row carries failureCategory: "platform-abandoned", progressStage: "subgraph-enrichment", and slotKey: "halfHourlyOffset". That stage loads whole catalogs into memory — 11.8 MB of DeFiLlama Yields JSON, 8.9 MB of Protocols, ~14.5 MB of Curve bodies, then the accumulating subgraph family maps — so a source that over-delivers is exactly the shape that ends a run. The caps above bound each of those reads individually; see docs/architecture.md for the lane-level memory topology.
Memory-Kill Triage (2026-09-23)
Three isolate deaths were recorded that day. Cloudflare invocation analytics reports the exact invocation second for the affected rows, which is what makes attribution possible without Workers Logs; the scheduled dataset carries the same cpu time for scheduled-handler kills, and cron_runs plus cron_slot_executions carry the slot identity when one exists.
| Invocation (UTC) | Workload | cpu / duration | Subreqs | Reported peak | D1 evidence |
|---|---|---|---|---|---|
2026-09-23 00:10:15 | sync-dex-liquidity-stage (halfHourlyOffset, progress stage subgraph-enrichment) | 4.69 s / 8.69 s | 186 | 171.8 MB | error row with failureCategory: "platform-abandoned", childDisposition: "abandoned", closed by the :16 reconciliation |
2026-09-23 07:52:46 | safety-score-v9-publication Workflow instance v9-publication-1790149920 (compile step) | 25.78 s / 25.78 s | 0 | 217.0 MB | workflow terminal row started_at 07:52:46, publicationStatus: "published" after the engine retried the killed attempt |
2026-09-23 11:22:53 | safety-score-v9-publication Workflow instance v9-publication-1790162520 (compile step) | 18.86 s / 18.86 s | 0 | 223.2 MB | workflow terminal row started_at 11:22:51, publicationStatus: "published" |
Reading the triage:
- Only the
00:10kill has cron-slot identity. The two Workflow kills are invocations of the same script that the:22/:52scheduled invocation creates immediately after it finishes compiling, so they leave nocron_slot_executionsrow and noplatform-abandonedreconciliation; a green terminal row is written once the Workflow engine retries the killed step. Per-invocation cpu equal to duration on both rows is the signature of a pure-compute compile step, and the neighbouring half-hours show the same ~20-25 s, ~211-221 MB invocation succeeding, so this lane sits permanently near the limit rather than failing on a single bad input. - The
00:10profile is the opposite shape: only 4.7 s of cpu across 8.7 s of wall time with 186 subrequests, i.e. the isolate died while provider bodies were flowing, which is why this workstream bounds every one of those reads. - Residual risk: the V9 shadow compile re-runs the full V9 compiler inside a Workflow invocation in the same minutes as the scheduled compile, and its memory is dominated by compiler graph modules outside this workstream's reviewed surface. Bounding those bodies does not change that lane's peak.
What this means operationally
sync-dex-liquidity-stageconsumes discovery output, loads external source families, constructs the ordered pool graph, and stores it as generation-fenced 192-KiB chunks hourly at:10. Its idempotent D1 reads and writes retry transient D1 overload/internal errors (runWithOverloadRetry) instead of failing the run, including thedex_pool_registrystaged-pool read.sync-dex-liquidityis D1-only except for its bounded same-hour stage-recovery re-run. At every hourly:16it refreshes prices and publishes liquidity scores/history, re-running a terminally failed or never-started:10stage for the same source slot under the stage job's lease before publishing; at:46it returns the exact current generation for V9 preparation without rewriting DEX surfaces, and — when:16never published — publishes the hour's still-unconsumed stage, re-running it first when that stage is terminal or never started.- Rate-bearing Curve StableSwap-NG enrichment runs only after the source graph and direct-provider phase have completed. It serializes chains, pins a fresh head/header, reads bounded pool batches (
get_balances,stored_rates,A,fee,offpeg_fee_multiplier, and orderedcoins) at that block, then confirms the same header hash before accepting a model. RPC, freshness, token-order, rate, fee, or dynamic-fee failures publish no model and retain the capability gate; this sequential phase does not increase the source trigger's declared five-of-six connection peak. - Active
sync-cl-exit-depthconsumes the previously published active retained-pool target generation and atomically publishes the next sparse measured-quote generation every 30 minutes. The hourly:16scoring consumer joins only fresh score-eligible evidence and then publishes the next active target inventory; the dailyevm-shadowgeneration never enters the active scorer. - EVM admission may reserve one currently published score-bearing direction packet closest to adapter-specific expiry, capped at 20 estimated requests within the 1,220-request admission ceiling. The reviewed legacy 3pool directions remain atomic; the reservation does not advance the cursor, and all remaining targets keep whole-coin rotation.
- Hook-free Ethereum Uniswap V4 shares the EVM lane without widening its limits: source enrichment is serialized after the existing subgraph families, PoolManager/StateView/Quoter verification is deduplicated per deployment, and state/quote calls use the existing eight-call Multicall batches under the 1,300-request and 16-MiB JSON-body ceilings. A transport-failed quote batch recursively fragments within the 80-request reserve; recovered sub-batches remain usable, while terminal singleton failures stay degraded.
- The exact legacy Curve 3pool adapter shares one Ethereum block and one deployment/registry verification across its direction packet. Its two output directions join atomically, including when reconstructing a retained route-only packet. The measured packet becomes score-facing only after both directions have three complete and three successful fresh cycles; until then the reserve simulation stays score-facing. Its last-known-good quote and history share the uniform three-hour ceiling used by every measured adapter, allowing repeated half-hour observations to survive scheduler jitter; the retained profile keeps its original quote block and timestamp, then falls back to the reserve model on expiry. Exact absent bytecode is semantic drift and cannot retain last-known-good evidence, while an unavailable RPC response remains an operational failure. This does not widen the EVM request/runtime ceilings.
sync-dex-discoveryis deliberately best-effort. Short per-source request timeouts and the 12-minute shared budget are there to force a partialdegradedresult before the platform can hard-kill the invocation. Lower-priority tier-2/tier-3 candidates are deterministically sharded across their cadence windows so one modulo run does not inherit the entire tier queue at once. The stale-pool refresh pass ahead of the cohort queue is capped by its own request count and wall-clock slice; a provider failure or open circuit ends only that pass (stalePoolRefresh.outcomein run metadata), and a registry read/write failure additionally marks the rundegraded(dex-discovery-stale-pool-refresh-failed). Cohort crawling continues in every case.- Missing-price fallback is intentionally time-bounded so a bad upstream day cannot consume the whole
sync-stablecoinsslot. - Replay-safe price continuity has a hard six-hour ceiling and also obeys any shorter per-source
maxTrustedAgeSec. Low/fallback or non-replay-safe sources are never eligible. The separate verified-CMC provider cache is limited to the original quote's one-hour age, preserves its upstream timestamp, stays fallback confidence, and revalidates identity and peg bounds on reuse. activePriceCoverageis independent from cron execution and cache publication. A missing active price degrades public health while the valid remainder of thestablecoinspayload stays available and a successfully published cron run remainsok.- Exact adapters are fail-closed availability paths. A stale head, RPC/provider outage, identity drift, missing parent, depleted bridge, insufficient pool depth, or excessive quote impact produces no price; operators must not compensate by increasing replay age or substituting nominal parity.
- Any new provider added to discovery or price enrichment should come with both a throttle and a hard stop budget.
Request Timeouts Worth Preserving
| Area | Current timeout | Source |
|---|---|---|
| CoinMarketCap price fallback | 10_000 ms | worker/src/cron/sync-stablecoins/enrich-prices-cmc-pass.ts |
| Jupiter price fallback | 5_000 ms | worker/src/cron/sync-stablecoins/enrich-prices-jupiter-pass.ts |
| DexScreener price fallback requests | up to 5_000 ms per request | worker/src/cron/sync-stablecoins/enrich-prices-dexscreener-pass.ts |
| Direct DEX API requests | 15_000 ms per request | worker/src/cron/dex-liquidity/direct-api-policy.ts |
| Measured DEX execution RPC requests | EVM up to 15_000 ms, clipped to the run's absolute 8 minute deadline | worker/src/cron/measured-execution/sync.ts |
| Ops admin proxy reads | 20_000 ms for /api/status and /api/status-history; 45_000 ms for /api/audit-depeg-history | shared/lib/api-endpoints/definitions.ts (opsProxyTimeoutMs) |
| Live reserve adapter attempt | 20_000 ms | worker/src/cron/sync-live-reserves-config.ts |
| Live reserve D1 finalize timeout | 30_000 ms | worker/src/cron/sync-live-reserves-config.ts |
| Blacklist explorer / RPC reads | 15_000 ms | worker/src/lib/fetch-retry.ts (default timeout) |
| Daily digest LLM call (outer) | 12 * 60_000 ms | worker/src/lib/constants.ts |
| Daily digest per-attempt fetch | 11 * 60_000 ms | worker/src/cron/digest/platform.ts (DIGEST_FETCH_PER_ATTEMPT_TIMEOUT_MS) |
Live reserve timeout values are resolved through LiveReserveSyncBudgetConfig. Production uses the checked-in defaults above; tests and operational wrappers can inject smaller or larger positive finite values to validate deferred-tail and D1-finalize behavior without changing cron code.
/api/audit-depeg-history is also hard-capped at 25 items per request (limit default and max) so the CoinGecko-backed admin audit stays page-sized over the Pages proxy.
Anthropic / Digest Runtime
Current digest generation constraints that are actually encoded in repo code:
- model: per-job typed configuration, with both
DAILY_DIGEST_LLM_CONFIGandWEEKLY_RECAP_LLM_CONFIGdefaulting toclaude-opus-5-5(Claude Opus 5.5). Opus 5.5 runs safety classifiers (cyber, bio,reasoning_extraction), so the request path treatsstop_reason=refusalas policy rather than infrastructure. - thinking: adaptive (
thinking.type = "adaptive") - reasoning effort: high (
output_config.effort = "high"). On a 2026-09-28 replay of production prompts, Opus 5.5 atxhighemitted up to 19,996 output tokens (weekly) and 11,844 (daily), which no ceiling inside the cost envelope holds; athighit emitted at most 6,510 and omitted the forward-look line less often than Opus 5 atxhigh(see measured telemetry below). - Anthropic LLM deadline:
12 * 60_000 ms(12 min) per edition, onecreateTimeoutSignalinrequestDigestCopy(worker/src/cron/digest/llm-request.ts) shared by the original and corrective legs, every HTTP retry, and every backoff sleep, so a corrective leg cannot restart the clock and run into the cron lease - per-attempt fetch timeout:
11 * 60_000 ms(11 min) to response headers only;fetchWithRetryclears it once headers arrive, so stream reading is bounded by the edition deadline - retry depth for the digest Anthropic call:
2(max 3 HTTP attempts); every HTTP attempt is recorded independently. A 200 stream that ends beforemessage_stop(dropped connection) is retried like a fetch-level failure while the edition budget covers a full request; its fragment is never parsed - corrective retry skip: if first-pass elapsed
>= 50%of the outer budget (6 min), the in-process retry after quality failures is skipped; the parse is accepted withqualityIssuesflaggeddegraded - daily cron lease (wrapper timeout):
14 * 60_000 ms(14 min), leaves ~2 min under Cloudflare's 15-min scheduled-event ceiling for D1 persistence, Twitter/Telegram delivery, and cron_runs logging. - daily-digest heartbeat override:
heartbeatSec = 30,maxRenewFailures = 3(seeworker/src/handlers/scheduled/context.ts— default policy unchanged for other jobs) - weekly cron lease:
14 * 60_000 ms(14 min), matching the daily wrapper so the weekly two-channel publication tail keeps the same post-LLM headroom under the 15-minute ceiling - max_tokens:
DIGEST_MAX_TOKENS=16000daily and weekly, about 2.5x the largest measured Opus 5.5highgeneration.max_tokensis not a spend guard on its own, because an attempt whose usage never came back may have billed a full generation and is still retried (up to 3 HTTP attempts per leg, 2 legs, so 6 billable generations worst case). Spend is bounded instead byDIGEST_MAX_EDITION_OUTPUT_TOKENS= 2 xDIGEST_MAX_TOKENS(32000), an aggregate per-edition output budget enforced by reservation: a request starts only when its entiremax_tokensstill fits; a request that reached Anthropic without reporting usage (fetch-level failure, or a 200 stream that ended beforemessage_stop) is charged the full ceiling; known usage is charged across every model attempt inusage.iterations; an HTTP-status rejection is charged nothing (no generation occurred, so overload retries stay functional). Two full requests means a corrective retry always fits after a completed first pass and one retry fits after an unknown-usage attempt. Envelope: six billed input charges at the largest observed prompts (15,265 daily, 6,784 weekly) plus 32,000 output tokens, daily plus weekly/7, at Opus 5.5 prices ($4 input, $20 output per MTok) is about $1.12/day, below the $1.15 ceiling. It is a planning figure for one invocation per edition, not an enforced calendar-day cap: manual force-runs and Monday weekly resumes (at most three weekly generations per edition day) each carry their own budget, and a mid-output fallback can bill one partial generation inside a request before it is charged.stop_reason=max_tokensremains a hard pre-parser failure and therefore a loud truncation tripwire. - measured token/cost telemetry:
- Opus 5 at
xhigh/16k in production (2026-09-01..28): 28 dailies with output p50 5,401 and max 11,565; 4 weeklies with max 8,466; $0.238/day blended recorded cost (a lower bound: one 2026-09-14 attempt ended without usage);missing-forward-lookon 3 of 6 dailies 2026-09-23..28. - 2026-09-28 replay of production prompts (dailies 2026-09-21, 09-24, 09-28; weekly 2026-09-07; first pass, 32k diagnostic ceiling): Opus 5
xhigh5,433-9,350 daily and 7,157 weekly,missing-forward-lookon 3 of 3 dailies. Opus 5.5xhigh10,960-11,844 daily and 19,996 weekly (up to 2.8x Opus 5), 97-168 s. Opus 5.5high(seven daily runs including four at the production 16k ceiling) 3,483-6,510 daily and 6,455 weekly, 31-58 s, about $0.16-0.20 per edition,missing-forward-lookon 2 of 7 dailies, one hardunverifiable-movement-claimin seven first passes. - Earlier migrations: Opus 4.8
xhigh/64k emitted up to 18,258 daily output tokens ($0.550/day blended); Opus 5high/16k dropped the forward-look on both sampled dailies (2026-08).
- Opus 5 at
- refusal fallback: requests set
fallbacks: "default"with beta headerserver-side-fallback-2026-07-01. Server-side fallback preserves one streaming request and one outer timeout; a manual second model call could exceed the 12-minute Anthropic budget, the 14-minute wrapper, or Cloudflare's 15-minute scheduled-trigger ceiling after HTTP retries. After a mid-output handoffmessage_startstill names the requested model, so the served model comes from thefallbackcontent block (to.model) and the finalfallback_messageentry inusage.iterations; each attempt in that list is billed at its own model's rates. If fallback still ends instop_reason=refusal, all partial text is discarded,stop_details.categoryis retained, no edition is published, and the policy outcome neither damages nor heals the Anthropic circuit breaker.reasoning_extractionrefusals are not retried by server-side fallback. - cadence: daily scheduled run plus deferred manual admin trigger (see "Manual trigger runtime model" below)
- cost accounting: pricing is external and can change independently of this repository. The worker records every original, corrective, and HTTP attempt in cron progress/run metadata and stores successful-edition provenance in
daily_digest.digest_meta.llm: requested and served model, effort, max tokens, input/cache-read/cache-write/output tokens, fallback handoffs, per-modelusage.iterations, attempt identities, stop reason, refusal category, latency, HTTP status, and computed USD cost. Cost is summed acrossusage.iterationsat each model's checked-in rates (the onemessageentry on a non-fallback stream carries a null model and is priced as the requested model); a pre-output decline is billed only for some refusal categories, so a fallback cost is an upper bound. Provider billing remains authoritative if the checked-in price table drifts.
Manual trigger runtime model
POST /api/trigger-digest does not execute the digest synchronously. It writes a bounded intent record (pending, attempts, nextAttemptAt, and lastError) into the digest:force-run-request cache row and returns 202. A dedicated */5 * * * * cron slot (digestTriggerPoll) reads due intents and runs the digest under scheduled-event wall-clock (15 min). Transient network, timeout, 5xx, rate-limit, and D1 failures retry with 2 * poll interval * attempts backoff for at most three attempts; validation, authorization, and other permanent failures dead-letter immediately, and exhausted retries remain as retained dead_letter state. A poll with no pending intent is a neutral idle slot and cannot create a synthetic not-started daily-digest failure; if a digest created durable progress and then lost ownership, stale-slot reconciliation still records that real abandoned attempt. The existing daily-digest lease remains authoritative: lease contention leaves the intent untouched for the next poll. Outcome is persisted to digest:last-trigger-result for D1 inspection and future ops-UI surfacing; the current admin panel still shows only the enqueue result from the browser session.
This two-step model exists because the repo treats long HTTP-triggered ctx.waitUntil() digest execution as unsafe on Cloudflare Workers. The external platform assumption is that HTTP request tail work can be canceled after a short post-response window, while scheduled events get the full scheduled-event wall-clock; digest runs can take several minutes, so enqueue + scheduled polling is the repo-verified safe path.
Source: worker/src/api/admin-actions.ts (enqueue-only HTTP handler), worker/src/handlers/scheduled/digest-trigger-poll.ts (polling consumer).
Source: worker/src/lib/constants.ts (Anthropic timeout/retries), worker/src/lib/cron-timeouts.ts (CRON_TIMEOUT_MS per-job lease budget), worker/src/cron/digest/platform.ts (model/thinking/effort, per-attempt timeout, corrective-retry skip), worker/src/cron/daily-digest.ts and worker/src/cron/weekly-recap.ts (max_tokens), worker/src/handlers/scheduled/context.ts (PER_JOB_LEASE_OPTIONS heartbeat override)
This doc deliberately does not restate Anthropic account-tier RPM / token-plan numbers because those are not repo-enforced.
Design Guidance
Before adding a worker feature that touches external services:
- Pick the trigger slot first. Shared slots are a capacity decision, not just a schedule decision.
- Add explicit throttle constants and an overall time budget before writing the fetch loop.
- Prefer chunked / batched writes and bounded SQL fan-out.
- Add or reuse a circuit breaker when the feature depends on a flaky upstream.
- Run
npm run check:cron-connectionsand document the trigger-slot impact for any new outbound I/O. - Update this doc only with limits the repo actually enforces or depends on architecturally.
If you need current provider-plan quotas, verify them outside the repo before relying on them.