234 lines
17 KiB
Markdown
Executable File
234 lines
17 KiB
Markdown
Executable File
---
|
||
name: fog-expedition-nx-and-mission-engine
|
||
overview: Set up an Nx-based monorepo for the Fog Expedition Twitch extension, implement a NestJS API with a tick-based mission engine, and integrate Twitch Extension auth, persistence, and content balance libraries.
|
||
todos:
|
||
- id: stage1-nx-foundation
|
||
content: Set up Nx workspace, add Angular/Nest plugins, generate Angular panel app, NestJS API app, and shared @fog-explorer/api-interfaces library, plus Docker Compose for Postgres and Redis.
|
||
status: completed
|
||
- id: stage2-extension-frontend
|
||
content: Build the Twitch panel Angular UI (panel shell, live log, survivor status) and integrate Twitch Extension auth plus an EBS HTTP client using shared interfaces.
|
||
status: completed
|
||
- id: stage3-mission-engine
|
||
content: Implement the 60-second tick engine, stateless encounter resolver, Redis-backed mission/lobby store, and SWF group logic services in the API.
|
||
status: completed
|
||
- id: stage4-ebs-persistence
|
||
content: Add NestJS controllers, Twitch JWT guard, Postgres models/migrations, and Twitch PubSub integration to replace chat commands with extension-driven actions.
|
||
status: completed
|
||
- id: stage5-content-balance
|
||
content: Create the JSON encounter library, formalize perk and modifier types, and add utilities/tests to tune and validate mission outcome probabilities.
|
||
status: completed
|
||
isProject: false
|
||
---
|
||
|
||
## Fog Expedition Nx Workspace & ZPG Engine Plan
|
||
|
||
> **v2 note:** Additions introduced in this revision are marked **[NEW]** throughout.
|
||
|
||
---
|
||
|
||
### Stage 1: Workspace & Infrastructure (Nx Foundation)
|
||
|
||
- **Initialize Nx workspace & tooling**
|
||
- Ensure the root contains an Nx workspace with TypeScript support (`nx.json`, `project.json`/`workspace.json`, `tsconfig.base.json`, `package.json`).
|
||
- Add Nx plugins for Angular and NestJS (e.g. `@nx/angular`, `@nx/nest`, `@nx/node`).
|
||
- Configure `package.json` scripts for running Nx tasks (e.g. `"nx": "nx"`, `"start": "nx serve twitch-extension-panel"`).
|
||
- **Generate core apps**
|
||
- Create an Angular app for the Twitch Extension Panel at `[apps/twitch-extension-panel/src/main.ts]` via Nx generator (standalone components, no routing complexity initially).
|
||
- Create a NestJS API app at `[apps/api/src/main.ts]` using Nx's Nest generator, structured with modules for `Auth`, `Missions`, and `TickEngine`.
|
||
- **Shared API interfaces library**
|
||
- Generate a shared TypeScript library `@fog-explorer/api-interfaces` at `[libs/api-interfaces/src/index.ts]`.
|
||
- Define core domain types in small focused files:
|
||
- `[libs/api-interfaces/src/lib/survivor.ts]` containing `Survivor`, `SurvivorStats`, `SurvivorState` (`Active`, `Injured`, `Sacrificed`).
|
||
- `[libs/api-interfaces/src/lib/mission.ts]` containing `Mission`, `MissionState` (`Lobby`, `InProgress`, `Completed`, `Failed`), `EncounterResult`.
|
||
- `[libs/api-interfaces/src/lib/perk.ts]` containing `Perk`, `PerkModifier`, and `TeamPerk` flags for SWF mechanics.
|
||
- Re-export all public contracts from `[libs/api-interfaces/src/index.ts]` for clean imports in both `twitch-extension-panel` and `api`.
|
||
- **Docker & Docker Compose for infra**
|
||
- Add a root `docker-compose.yml` defining services:
|
||
- `postgres`: PostgreSQL with exposed port, DB name `fog_expedition`, volume, and healthcheck.
|
||
- `redis`: Redis for mission timers and lobbies.
|
||
- Optional `api` service wired to build from `[apps/api/Dockerfile]` and depend on `postgres` and `redis`.
|
||
- Create Dockerfiles:
|
||
- `[apps/api/Dockerfile]` building the NestJS API (install deps, build via Nx, run with `node dist/apps/api/main.js`).
|
||
- `[apps/twitch-extension-panel/Dockerfile]` for building and serving the Angular extension bundle for local testing.
|
||
- Define environment configuration via `.env`/`.env.local` for DB and Redis connection strings consumed by the API app.
|
||
- **[NEW] Local development without Twitch**
|
||
- Since `window.Twitch.ext` is unavailable in a local browser, a missing stub causes the panel to hang silently on startup. Solve this early to keep frontend development fluid.
|
||
- Create a `TwitchExtMock` module at `[apps/twitch-extension-panel/src/mocks/twitch-ext.mock.ts]` that:
|
||
- Exposes the same interface as `window.Twitch.ext` (`onAuthorized`, `onContext`, `onVisibilityChanged`, `listen`).
|
||
- Returns a hardcoded fake JWT and viewer ID so all downstream services initialise normally.
|
||
- Is swapped in via an `environment.ts` flag (`environment.mock = true`) — never shipped in production builds.
|
||
- Add a `local-dev` npm script that sets the mock flag and serves the panel against a local API.
|
||
|
||
---
|
||
|
||
### Stage 2: The Extension Frontend (Angular Panel)
|
||
|
||
- **Panel shell & layout**
|
||
- In `twitch-extension-panel`, create a `PanelShellComponent` (e.g. `[apps/twitch-extension-panel/src/app/panel-shell.component.ts]`) that:
|
||
- Hosts the Live Log area and Survivor Status area.
|
||
- Handles initial Twitch Extension context/bootstrap (mounts only after `window.Twitch.ext.onAuthorized`, or mock equivalent in dev).
|
||
- Add presentational components:
|
||
- `LiveLogComponent` for a scrolling log of encounters/events (Progress Quest style).
|
||
- `SurvivorStatusComponent` for current survivor(s), health/injury state, mission status, and perks.
|
||
- **Twitch EBS integration & auth**
|
||
- Implement a `TwitchAuthService` (e.g. `[apps/twitch-extension-panel/src/app/services/twitch-auth.service.ts]`) that:
|
||
- Wraps `window.Twitch.ext` calls (`onAuthorized`, `onContext`, `onVisibilityChanged`).
|
||
- Stores the latest JWT and parsed payload (obfuscated Twitch extension user ID).
|
||
- Implement an `EbsApiService` that:
|
||
- Attaches the Twitch JWT as `Authorization: Bearer <token>` on all HTTP calls to the NestJS API.
|
||
- Exposes methods like `startMission`, `getMissionState`, `getPerkInventory` using types from `@fog-explorer/api-interfaces`.
|
||
- **[NEW] EBS resilience & rate limiting**
|
||
- At scale, thousands of concurrent viewers can simultaneously poll `GET /missions/state`, overwhelming the API.
|
||
- Cache `GET /missions/state` responses in Redis with a 2–3 second TTL to absorb viewer polling spikes.
|
||
- The panel should degrade gracefully when the EBS is unreachable — display last-known state rather than a broken UI, and surface a subtle "reconnecting…" indicator.
|
||
- Add client-side exponential back-off with jitter for all EBS fetch retries.
|
||
- **State & real-time mission display**
|
||
- Create a `MissionStateStore` using Angular signals (or RxJS `BehaviorSubject` as a fallback) to hold:
|
||
- Current mission snapshot (`MissionState` + encounter history).
|
||
- Survivor state(s) and perk inventory.
|
||
- Wire the store to:
|
||
- Pull initial state from the API (`getMissionState`).
|
||
- Subscribe to Twitch PubSub events (via the Twitch JS helper) to apply incremental updates (new log lines, state transitions).
|
||
- Bind `PanelShellComponent`, `LiveLogComponent`, and `SurvivorStatusComponent` to this store using `computed` signals or `async` pipe for efficient change detection.
|
||
- **[NEW] PubSub message size guard**
|
||
- Twitch Extension PubSub enforces a 5 KB per-message limit. Unchecked log growth silently drops messages.
|
||
- Send only the latest N log lines per PubSub message (recommend N = 10–15).
|
||
- If the panel misses messages (detected via a sequence counter), trigger a full `getMissionState` re-fetch as a fallback.
|
||
|
||
---
|
||
|
||
### Stage 3: The ZPG Mission Engine ("Corso")
|
||
|
||
- **Global heartbeat & tick worker**
|
||
- In the NestJS API, add a `TickEngineModule` (e.g. `[apps/api/src/app/tick-engine/tick-engine.module.ts]`) and a `TickService` that:
|
||
- Uses `@nestjs/schedule` (cron) or a background worker to trigger a global tick every 60 seconds.
|
||
- Acquires a Redis-based distributed lock (e.g. `SETNX tick_lock`) to ensure only one instance runs the tick.
|
||
- Scans active missions from Redis and dispatches them to the encounter resolver.
|
||
- **[NEW] Tick engine resilience**
|
||
- Two failure modes need explicit answers before production: long ticks and process restarts.
|
||
- Set the Redis lock TTL to 50 seconds (10-second buffer before the next tick fires). If a tick is still running when the lock expires, emit a structured warning log — do not silently skip.
|
||
- Make encounter resolution idempotent: store a tick sequence number per mission in Redis and skip re-processing if the current tick index has already been resolved. This protects against double-application on restart mid-tick.
|
||
- Log tick start, duration, and number of missions processed as structured fields on every execution.
|
||
- **Encounter resolver (stateless core)**
|
||
- Implement a pure, stateless function in `[libs/api-interfaces/src/lib/encounter-resolver.ts]` or a new logic lib (e.g. `libs/mission-logic`):
|
||
- Signature like `resolveEncounter(survivor: SurvivorStats, perks: PerkModifier[], difficulty: Difficulty): EncounterResult`.
|
||
- Uses RNG with a pluggable seed source for deterministic tests.
|
||
- Applies modifiers as P{success} = Base + Σ Modifiers clamped to sane bounds.
|
||
- In the API, create an `EncounterService` that:
|
||
- Calls the resolver for each mission tick.
|
||
- Updates mission state (success/fail, injury, sacrifice) and logs events.
|
||
- **Group logic & SWF mechanics**
|
||
- Introduce a `GroupSynergyService` that:
|
||
- Accepts a group of 2–4 survivors and their perks.
|
||
- Aggregates "Team Perk" modifiers, applying them to all relevant rolls.
|
||
- Computes group-level outcomes (e.g., shared progress, split rewards, synchronized failure states).
|
||
- Extend the encounter resolver inputs to accept a `groupContext` that influences difficulty modifiers for group missions.
|
||
- **Redis for active missions & lobbies**
|
||
- Define Redis data structures in the API layer:
|
||
- `active_mission:{missionId}`: JSON or hash storing mission state, participant IDs, nextTickAt.
|
||
- `mission_lobby:{lobbyId}`: state for lobby members, mission template, and ready flags.
|
||
- Implement a `MissionStore` repository that abstracts Redis operations (get/set/expire/lists) from the core mission logic.
|
||
|
||
---
|
||
|
||
### Stage 4: Twitch Extension Persistence (EBS)
|
||
|
||
- **Extension action handlers instead of chat commands**
|
||
- In the NestJS `MissionsModule`, add controllers like `[apps/api/src/app/missions/missions.controller.ts]` exposing:
|
||
- `POST /missions/start` to replace `!explore` (triggered by the panel "Start Mission" button).
|
||
- `GET /missions/state` for current mission snapshot.
|
||
- `POST /missions/choose-perk` (future) or similar progression actions.
|
||
- Use DTOs shaped by `@fog-explorer/api-interfaces` types for request/response contracts.
|
||
- **Twitch JWT verification & identity mapping**
|
||
- Implement a `TwitchAuthModule` and `TwitchAuthGuard` that:
|
||
- Verifies incoming JWTs from the extension using the Twitch extension secret.
|
||
- Extracts the obfuscated Twitch user ID and extension channel ID, attaching them to the Nest request context.
|
||
- **[NEW] JWT secret rotation handling**
|
||
- Twitch can rotate the extension secret, causing all JWT verification to fail globally until the API is redeployed.
|
||
- On a JWT verification failure, the guard should attempt to re-fetch the current secret from the Twitch API before returning 401.
|
||
- Cache the fetched secret with a reasonable TTL (e.g. 5 minutes) so rotation recovery is near-instant.
|
||
- Emit a structured alert log whenever a secret re-fetch occurs — this is a high-signal operational event.
|
||
- **Postgres schema**
|
||
- Design Postgres tables (via migrations/TypeORM/Prisma) in the API project:
|
||
- `users` (internal ID, Twitch extension user ID, created_at).
|
||
- `survivors` (FK to `users`, stats, perk slots, current state: active/injured/sacrificed). Use soft-delete / `sacrificed_at` timestamp rather than hard deletion to support history and leaderboards.
|
||
- `missions` (FK to `survivors` or group, status, difficulty, timestamps).
|
||
- `mission_logs` (FK to `missions`, tick index, encounter key, rendered text, RNG details if needed).
|
||
- **[NEW] mission_logs retention strategy**
|
||
- `mission_logs` can grow unboundedly — a channel with high traffic can generate millions of rows within weeks.
|
||
- Add an `archived_at` nullable column to `mission_logs` from day one.
|
||
- Implement a lightweight nightly archival job that marks logs for completed/failed missions older than N days as archived (or bulk-moves them to an archive table).
|
||
- Add a Postgres index on `(mission_id, archived_at)` to keep active-mission log queries fast regardless of historical volume.
|
||
- **Twitch PubSub integration for ticks**
|
||
- Create a `TwitchPubSubService` in the API that:
|
||
- Signs messages to Twitch's Extension PubSub endpoint using the extension client ID and secret.
|
||
- Publishes log updates and mission state deltas on each tick for relevant channels.
|
||
- On the frontend, extend `TwitchAuthService` or a dedicated `PubSubService` to:
|
||
- Subscribe to mission update topics.
|
||
- Forward received events into the `MissionStateStore`, appending log lines and updating survivor/mission state.
|
||
- **[NEW] Broadcaster configuration panel**
|
||
- Broadcasters frequently need to configure extension behaviour (difficulty presets, max survivors, opt-in features). This is often needed earlier than expected.
|
||
- Create a separate Angular app (or a route within the panel) for the Twitch Broadcaster Config view.
|
||
- Add a `POST /channel/config` EBS endpoint that accepts channel-level settings and stores them against the channel ID in Postgres.
|
||
- The config schema should at minimum cover: mission difficulty preset, max party size, and feature flags for beta mechanics.
|
||
- Apply these settings in `MissionsModule` when resolving encounters for that channel.
|
||
|
||
---
|
||
|
||
### Stage 5: Content Library & Balance
|
||
|
||
- **JSON encounter library**
|
||
- Create a dedicated library `@fog-explorer/encounter-library` at `[libs/encounter-library/src/lib/encounters.json]` to hold all flavor text and base parameters.
|
||
- Structure encounters as JSON records:
|
||
- Keys like `"cleanse_hex"`, `"escape_hatch"`, etc.
|
||
- Fields for base success chance, difficulty tags, flavor text variants, and perk tags.
|
||
- Add a small TypeScript wrapper in `[libs/encounter-library/src/lib/encounter-library.ts]` to:
|
||
- Load encounters.
|
||
- Expose helper functions (e.g. `getEncounterById`, `getRandomEncounterByTier`).
|
||
- **[NEW] Encounter JSON schema versioning**
|
||
- Adding new fields to `encounters.json` silently breaks records that predate those fields, causing runtime surprises.
|
||
- Define a JSON Schema (or Zod schema) for the encounter record format and commit it alongside `encounters.json`.
|
||
- Add a build-time validation step (Nx target) that validates all encounter records against the schema on every build.
|
||
- Increment a `schemaVersion` field whenever breaking fields are added, and provide a migration helper for existing records.
|
||
- **Perk system & balance utilities**
|
||
- In `@fog-explorer/api-interfaces` (or a dedicated `@fog-explorer/perks` lib), formalize perk modifier types:
|
||
- Additive and multiplicative modifiers, flat reroll mechanics, group-boosting perks.
|
||
- Implement a `PerkMath` helper that:
|
||
- Aggregates modifiers into a final P{success} value.
|
||
- Separates survivor-level modifiers from team-level modifiers to keep SWF behaviour clear.
|
||
- Provide a small test harness (unit tests in the mission logic lib) to:
|
||
- Validate that typical perk loadouts produce reasonable success/fail/injury rates.
|
||
- Guard against accidental balance regressions.
|
||
- **[NEW] Simulation CLI for balance tuning**
|
||
- Unit tests confirm correctness but are too coarse for tuning probability distributions. A headless simulator is far more useful.
|
||
- Add an Nx target: `nx run mission-logic:simulate -- --runs=10000 --perkSet=flashlight,spine_chill`
|
||
- The simulator should output: mean success rate, injury rate, sacrifice rate, and a percentile breakdown of mission lengths.
|
||
- Wire it into CI with a fixed seed so regressions in outcome distributions surface as test failures without flakiness.
|
||
|
||
---
|
||
|
||
### [NEW] Cross-Cutting Concerns
|
||
|
||
These topics were absent from the original plan but span all stages and should be established early.
|
||
|
||
- **Observability**
|
||
- Install Pino as the NestJS logger (structured JSON output). Replace all `console.log` usage with pino log calls.
|
||
- Define a standard log schema with fields: `traceId`, `channelId`, `missionId`, `tickIndex`, `durationMs`, `event`.
|
||
- Instrument the tick engine specifically: log tick start, active mission count, tick duration, and any per-mission errors as structured fields on every execution.
|
||
- Add basic application metrics (tick processing time, encounter outcomes, error rates) — even a simple in-memory counter exported to a `/metrics` endpoint is sufficient to start.
|
||
|
||
---
|
||
|
||
### High-Level Architecture Diagram
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
viewerPanel["Twitch Panel (Angular)"] -->|"JWT via window.Twitch.ext"| apiGateway["NestJS API (EBS)"]
|
||
apiGateway -->|"REST: /missions/*"| missionEngine["Mission Engine / Tick Service"]
|
||
missionEngine -->|"state + logs"| postgresDb["Postgres"]
|
||
missionEngine -->|"timers + lobbies"| redisStore["Redis"]
|
||
missionEngine -->|"Tick updates"| twitchPubSub["Twitch Extension PubSub"]
|
||
twitchPubSub -->|"Push events"| viewerPanel
|
||
apiGateway -->|"channel config"| postgresDb
|
||
broadcasterConfig["Broadcaster Config Panel"] -->|"POST /channel/config"| apiGateway
|
||
``` |