maurycy/fog-explorer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
42d20cb0ed8d5ac9a981ac80ade12ae8833e96c0
fog-explorer/fog/plan.md
Hussar 42d20cb0ed first commit
2026-04-12 15:35:50 +00:00

17 KiB
Executable File
Raw Blame History

name, overview, todos, isProject
name overview todos isProject
fog-expedition-nx-and-mission-engine Set up an Nx-based monorepo for the Fog Expedition Twitch extension, implement a NestJS API with a tick-based mission engine, and integrate Twitch Extension auth, persistence, and content balance libraries.
id content status
stage1-nx-foundation Set up Nx workspace, add Angular/Nest plugins, generate Angular panel app, NestJS API app, and shared @fog-explorer/api-interfaces library, plus Docker Compose for Postgres and Redis. completed
id content status
stage2-extension-frontend Build the Twitch panel Angular UI (panel shell, live log, survivor status) and integrate Twitch Extension auth plus an EBS HTTP client using shared interfaces. completed
id content status
stage3-mission-engine Implement the 60-second tick engine, stateless encounter resolver, Redis-backed mission/lobby store, and SWF group logic services in the API. completed
id content status
stage4-ebs-persistence Add NestJS controllers, Twitch JWT guard, Postgres models/migrations, and Twitch PubSub integration to replace chat commands with extension-driven actions. completed
id content status
stage5-content-balance Create the JSON encounter library, formalize perk and modifier types, and add utilities/tests to tune and validate mission outcome probabilities. completed
false

Fog Expedition Nx Workspace & ZPG Engine Plan

v2 note: Additions introduced in this revision are marked [NEW] throughout.

Stage 1: Workspace & Infrastructure (Nx Foundation)

  • Initialize Nx workspace & tooling
    • Ensure the root contains an Nx workspace with TypeScript support (nx.json, project.json/workspace.json, tsconfig.base.json, package.json).
    • Add Nx plugins for Angular and NestJS (e.g. @nx/angular, @nx/nest, @nx/node).
    • Configure package.json scripts for running Nx tasks (e.g. "nx": "nx", "start": "nx serve twitch-extension-panel").
  • Generate core apps
    • Create an Angular app for the Twitch Extension Panel at [apps/twitch-extension-panel/src/main.ts] via Nx generator (standalone components, no routing complexity initially).
    • Create a NestJS API app at [apps/api/src/main.ts] using Nx's Nest generator, structured with modules for Auth, Missions, and TickEngine.
  • Shared API interfaces library
    • Generate a shared TypeScript library @fog-explorer/api-interfaces at [libs/api-interfaces/src/index.ts].
    • Define core domain types in small focused files:
      • [libs/api-interfaces/src/lib/survivor.ts] containing Survivor, SurvivorStats, SurvivorState (Active, Injured, Sacrificed).
      • [libs/api-interfaces/src/lib/mission.ts] containing Mission, MissionState (Lobby, InProgress, Completed, Failed), EncounterResult.
      • [libs/api-interfaces/src/lib/perk.ts] containing Perk, PerkModifier, and TeamPerk flags for SWF mechanics.
    • Re-export all public contracts from [libs/api-interfaces/src/index.ts] for clean imports in both twitch-extension-panel and api.
  • Docker & Docker Compose for infra
    • Add a root docker-compose.yml defining services:
      • postgres: PostgreSQL with exposed port, DB name fog_expedition, volume, and healthcheck.
      • redis: Redis for mission timers and lobbies.
      • Optional api service wired to build from [apps/api/Dockerfile] and depend on postgres and redis.
    • Create Dockerfiles:
      • [apps/api/Dockerfile] building the NestJS API (install deps, build via Nx, run with node dist/apps/api/main.js).
      • [apps/twitch-extension-panel/Dockerfile] for building and serving the Angular extension bundle for local testing.
    • Define environment configuration via .env/.env.local for DB and Redis connection strings consumed by the API app.
  • [NEW] Local development without Twitch
    • Since window.Twitch.ext is unavailable in a local browser, a missing stub causes the panel to hang silently on startup. Solve this early to keep frontend development fluid.
    • Create a TwitchExtMock module at [apps/twitch-extension-panel/src/mocks/twitch-ext.mock.ts] that:
      • Exposes the same interface as window.Twitch.ext (onAuthorized, onContext, onVisibilityChanged, listen).
      • Returns a hardcoded fake JWT and viewer ID so all downstream services initialise normally.
      • Is swapped in via an environment.ts flag (environment.mock = true) — never shipped in production builds.
    • Add a local-dev npm script that sets the mock flag and serves the panel against a local API.

Stage 2: The Extension Frontend (Angular Panel)

  • Panel shell & layout
    • In twitch-extension-panel, create a PanelShellComponent (e.g. [apps/twitch-extension-panel/src/app/panel-shell.component.ts]) that:
      • Hosts the Live Log area and Survivor Status area.
      • Handles initial Twitch Extension context/bootstrap (mounts only after window.Twitch.ext.onAuthorized, or mock equivalent in dev).
    • Add presentational components:
      • LiveLogComponent for a scrolling log of encounters/events (Progress Quest style).
      • SurvivorStatusComponent for current survivor(s), health/injury state, mission status, and perks.
  • Twitch EBS integration & auth
    • Implement a TwitchAuthService (e.g. [apps/twitch-extension-panel/src/app/services/twitch-auth.service.ts]) that:
      • Wraps window.Twitch.ext calls (onAuthorized, onContext, onVisibilityChanged).
      • Stores the latest JWT and parsed payload (obfuscated Twitch extension user ID).
    • Implement an EbsApiService that:
      • Attaches the Twitch JWT as Authorization: Bearer <token> on all HTTP calls to the NestJS API.
      • Exposes methods like startMission, getMissionState, getPerkInventory using types from @fog-explorer/api-interfaces.
  • [NEW] EBS resilience & rate limiting
    • At scale, thousands of concurrent viewers can simultaneously poll GET /missions/state, overwhelming the API.
    • Cache GET /missions/state responses in Redis with a 2–3 second TTL to absorb viewer polling spikes.
    • The panel should degrade gracefully when the EBS is unreachable — display last-known state rather than a broken UI, and surface a subtle "reconnecting…" indicator.
    • Add client-side exponential back-off with jitter for all EBS fetch retries.
  • State & real-time mission display
    • Create a MissionStateStore using Angular signals (or RxJS BehaviorSubject as a fallback) to hold:
      • Current mission snapshot (MissionState + encounter history).
      • Survivor state(s) and perk inventory.
    • Wire the store to:
      • Pull initial state from the API (getMissionState).
      • Subscribe to Twitch PubSub events (via the Twitch JS helper) to apply incremental updates (new log lines, state transitions).
    • Bind PanelShellComponent, LiveLogComponent, and SurvivorStatusComponent to this store using computed signals or async pipe for efficient change detection.
  • [NEW] PubSub message size guard
    • Twitch Extension PubSub enforces a 5 KB per-message limit. Unchecked log growth silently drops messages.
    • Send only the latest N log lines per PubSub message (recommend N = 10–15).
    • If the panel misses messages (detected via a sequence counter), trigger a full getMissionState re-fetch as a fallback.

Stage 3: The ZPG Mission Engine ("Corso")

  • Global heartbeat & tick worker
    • In the NestJS API, add a TickEngineModule (e.g. [apps/api/src/app/tick-engine/tick-engine.module.ts]) and a TickService that:
      • Uses @nestjs/schedule (cron) or a background worker to trigger a global tick every 60 seconds.
      • Acquires a Redis-based distributed lock (e.g. SETNX tick_lock) to ensure only one instance runs the tick.
      • Scans active missions from Redis and dispatches them to the encounter resolver.
  • [NEW] Tick engine resilience
    • Two failure modes need explicit answers before production: long ticks and process restarts.
    • Set the Redis lock TTL to 50 seconds (10-second buffer before the next tick fires). If a tick is still running when the lock expires, emit a structured warning log — do not silently skip.
    • Make encounter resolution idempotent: store a tick sequence number per mission in Redis and skip re-processing if the current tick index has already been resolved. This protects against double-application on restart mid-tick.
    • Log tick start, duration, and number of missions processed as structured fields on every execution.
  • Encounter resolver (stateless core)
    • Implement a pure, stateless function in [libs/api-interfaces/src/lib/encounter-resolver.ts] or a new logic lib (e.g. libs/mission-logic):
      • Signature like resolveEncounter(survivor: SurvivorStats, perks: PerkModifier[], difficulty: Difficulty): EncounterResult.
      • Uses RNG with a pluggable seed source for deterministic tests.
      • Applies modifiers as P{success} = Base + Σ Modifiers clamped to sane bounds.
    • In the API, create an EncounterService that:
      • Calls the resolver for each mission tick.
      • Updates mission state (success/fail, injury, sacrifice) and logs events.
  • Group logic & SWF mechanics
    • Introduce a GroupSynergyService that:
      • Accepts a group of 2–4 survivors and their perks.
      • Aggregates "Team Perk" modifiers, applying them to all relevant rolls.
      • Computes group-level outcomes (e.g., shared progress, split rewards, synchronized failure states).
    • Extend the encounter resolver inputs to accept a groupContext that influences difficulty modifiers for group missions.
  • Redis for active missions & lobbies
    • Define Redis data structures in the API layer:
      • active_mission:{missionId}: JSON or hash storing mission state, participant IDs, nextTickAt.
      • mission_lobby:{lobbyId}: state for lobby members, mission template, and ready flags.
    • Implement a MissionStore repository that abstracts Redis operations (get/set/expire/lists) from the core mission logic.

Stage 4: Twitch Extension Persistence (EBS)

  • Extension action handlers instead of chat commands
    • In the NestJS MissionsModule, add controllers like [apps/api/src/app/missions/missions.controller.ts] exposing:
      • POST /missions/start to replace !explore (triggered by the panel "Start Mission" button).
      • GET /missions/state for current mission snapshot.
      • POST /missions/choose-perk (future) or similar progression actions.
    • Use DTOs shaped by @fog-explorer/api-interfaces types for request/response contracts.
  • Twitch JWT verification & identity mapping
    • Implement a TwitchAuthModule and TwitchAuthGuard that:
      • Verifies incoming JWTs from the extension using the Twitch extension secret.
      • Extracts the obfuscated Twitch user ID and extension channel ID, attaching them to the Nest request context.
  • [NEW] JWT secret rotation handling
    • Twitch can rotate the extension secret, causing all JWT verification to fail globally until the API is redeployed.
    • On a JWT verification failure, the guard should attempt to re-fetch the current secret from the Twitch API before returning 401.
    • Cache the fetched secret with a reasonable TTL (e.g. 5 minutes) so rotation recovery is near-instant.
    • Emit a structured alert log whenever a secret re-fetch occurs — this is a high-signal operational event.
  • Postgres schema
    • Design Postgres tables (via migrations/TypeORM/Prisma) in the API project:
      • users (internal ID, Twitch extension user ID, created_at).
      • survivors (FK to users, stats, perk slots, current state: active/injured/sacrificed). Use soft-delete / sacrificed_at timestamp rather than hard deletion to support history and leaderboards.
      • missions (FK to survivors or group, status, difficulty, timestamps).
      • mission_logs (FK to missions, tick index, encounter key, rendered text, RNG details if needed).
  • [NEW] mission_logs retention strategy
    • mission_logs can grow unboundedly — a channel with high traffic can generate millions of rows within weeks.
    • Add an archived_at nullable column to mission_logs from day one.
    • Implement a lightweight nightly archival job that marks logs for completed/failed missions older than N days as archived (or bulk-moves them to an archive table).
    • Add a Postgres index on (mission_id, archived_at) to keep active-mission log queries fast regardless of historical volume.
  • Twitch PubSub integration for ticks
    • Create a TwitchPubSubService in the API that:
      • Signs messages to Twitch's Extension PubSub endpoint using the extension client ID and secret.
      • Publishes log updates and mission state deltas on each tick for relevant channels.
    • On the frontend, extend TwitchAuthService or a dedicated PubSubService to:
      • Subscribe to mission update topics.
      • Forward received events into the MissionStateStore, appending log lines and updating survivor/mission state.
  • [NEW] Broadcaster configuration panel
    • Broadcasters frequently need to configure extension behaviour (difficulty presets, max survivors, opt-in features). This is often needed earlier than expected.
    • Create a separate Angular app (or a route within the panel) for the Twitch Broadcaster Config view.
    • Add a POST /channel/config EBS endpoint that accepts channel-level settings and stores them against the channel ID in Postgres.
    • The config schema should at minimum cover: mission difficulty preset, max party size, and feature flags for beta mechanics.
    • Apply these settings in MissionsModule when resolving encounters for that channel.

Stage 5: Content Library & Balance

  • JSON encounter library
    • Create a dedicated library @fog-explorer/encounter-library at [libs/encounter-library/src/lib/encounters.json] to hold all flavor text and base parameters.
    • Structure encounters as JSON records:
      • Keys like "cleanse_hex", "escape_hatch", etc.
      • Fields for base success chance, difficulty tags, flavor text variants, and perk tags.
    • Add a small TypeScript wrapper in [libs/encounter-library/src/lib/encounter-library.ts] to:
      • Load encounters.
      • Expose helper functions (e.g. getEncounterById, getRandomEncounterByTier).
  • [NEW] Encounter JSON schema versioning
    • Adding new fields to encounters.json silently breaks records that predate those fields, causing runtime surprises.
    • Define a JSON Schema (or Zod schema) for the encounter record format and commit it alongside encounters.json.
    • Add a build-time validation step (Nx target) that validates all encounter records against the schema on every build.
    • Increment a schemaVersion field whenever breaking fields are added, and provide a migration helper for existing records.
  • Perk system & balance utilities
    • In @fog-explorer/api-interfaces (or a dedicated @fog-explorer/perks lib), formalize perk modifier types:
      • Additive and multiplicative modifiers, flat reroll mechanics, group-boosting perks.
    • Implement a PerkMath helper that:
      • Aggregates modifiers into a final P{success} value.
      • Separates survivor-level modifiers from team-level modifiers to keep SWF behaviour clear.
    • Provide a small test harness (unit tests in the mission logic lib) to:
      • Validate that typical perk loadouts produce reasonable success/fail/injury rates.
      • Guard against accidental balance regressions.
  • [NEW] Simulation CLI for balance tuning
    • Unit tests confirm correctness but are too coarse for tuning probability distributions. A headless simulator is far more useful.
    • Add an Nx target: nx run mission-logic:simulate -- --runs=10000 --perkSet=flashlight,spine_chill
    • The simulator should output: mean success rate, injury rate, sacrifice rate, and a percentile breakdown of mission lengths.
    • Wire it into CI with a fixed seed so regressions in outcome distributions surface as test failures without flakiness.

[NEW] Cross-Cutting Concerns

These topics were absent from the original plan but span all stages and should be established early.

  • Observability
    • Install Pino as the NestJS logger (structured JSON output). Replace all console.log usage with pino log calls.
    • Define a standard log schema with fields: traceId, channelId, missionId, tickIndex, durationMs, event.
    • Instrument the tick engine specifically: log tick start, active mission count, tick duration, and any per-mission errors as structured fields on every execution.
    • Add basic application metrics (tick processing time, encounter outcomes, error rates) — even a simple in-memory counter exported to a /metrics endpoint is sufficient to start.

High-Level Architecture Diagram

flowchart LR
  viewerPanel["Twitch Panel (Angular)"] -->|"JWT via window.Twitch.ext"| apiGateway["NestJS API (EBS)"]
  apiGateway -->|"REST: /missions/*"| missionEngine["Mission Engine / Tick Service"]
  missionEngine -->|"state + logs"| postgresDb["Postgres"]
  missionEngine -->|"timers + lobbies"| redisStore["Redis"]
  missionEngine -->|"Tick updates"| twitchPubSub["Twitch Extension PubSub"]
  twitchPubSub -->|"Push events"| viewerPanel
  apiGateway -->|"channel config"| postgresDb
  broadcasterConfig["Broadcaster Config Panel"] -->|"POST /channel/config"| apiGateway
Reference in New Issue View Git Blame Copy Permalink