---
name: fog-expedition-nx-and-mission-engine
overview: Set up an Nx-based monorepo for the Fog Expedition Twitch extension, implement a NestJS API with a tick-based mission engine, and integrate Twitch Extension auth, persistence, and content balance libraries.
todos:
  - id: stage1-nx-foundation
    content: Set up Nx workspace, add Angular/Nest plugins, generate Angular panel app, NestJS API app, and shared @fog-explorer/api-interfaces library, plus Docker Compose for Postgres and Redis.
    status: completed
  - id: stage2-extension-frontend
    content: Build the Twitch panel Angular UI (panel shell, live log, survivor status) and integrate Twitch Extension auth plus an EBS HTTP client using shared interfaces.
    status: completed
  - id: stage3-mission-engine
    content: Implement the 60-second tick engine, stateless encounter resolver, Redis-backed mission/lobby store, and SWF group logic services in the API.
    status: completed
  - id: stage4-ebs-persistence
    content: Add NestJS controllers, Twitch JWT guard, Postgres models/migrations, and Twitch PubSub integration to replace chat commands with extension-driven actions.
    status: completed
  - id: stage5-content-balance
    content: Create the JSON encounter library, formalize perk and modifier types, and add utilities/tests to tune and validate mission outcome probabilities.
    status: completed
isProject: false
---

## Fog Expedition Nx Workspace & ZPG Engine Plan

> **v2 note:** Additions introduced in this revision are marked **[NEW]** throughout.

---

### Stage 1: Workspace & Infrastructure (Nx Foundation)

- **Initialize Nx workspace & tooling**
  - Ensure the root contains an Nx workspace with TypeScript support (`nx.json`, `project.json`/`workspace.json`, `tsconfig.base.json`, `package.json`).
  - Add Nx plugins for Angular and NestJS (e.g. `@nx/angular`, `@nx/nest`, `@nx/node`).
  - Configure `package.json` scripts for running Nx tasks (e.g. `"nx": "nx"`, `"start": "nx serve twitch-extension-panel"`).
- **Generate core apps**
  - Create an Angular app for the Twitch Extension Panel at `[apps/twitch-extension-panel/src/main.ts]` via Nx generator (standalone components, no routing complexity initially).
  - Create a NestJS API app at `[apps/api/src/main.ts]` using Nx's Nest generator, structured with modules for `Auth`, `Missions`, and `TickEngine`.
- **Shared API interfaces library**
  - Generate a shared TypeScript library `@fog-explorer/api-interfaces` at `[libs/api-interfaces/src/index.ts]`.
  - Define core domain types in small focused files:
    - `[libs/api-interfaces/src/lib/survivor.ts]` containing `Survivor`, `SurvivorStats`, `SurvivorState` (`Active`, `Injured`, `Sacrificed`).
    - `[libs/api-interfaces/src/lib/mission.ts]` containing `Mission`, `MissionState` (`Lobby`, `InProgress`, `Completed`, `Failed`), `EncounterResult`.
    - `[libs/api-interfaces/src/lib/perk.ts]` containing `Perk`, `PerkModifier`, and `TeamPerk` flags for SWF mechanics.
  - Re-export all public contracts from `[libs/api-interfaces/src/index.ts]` for clean imports in both `twitch-extension-panel` and `api`.
- **Docker & Docker Compose for infra**
  - Add a root `docker-compose.yml` defining services:
    - `postgres`: PostgreSQL with exposed port, DB name `fog_expedition`, volume, and healthcheck.
    - `redis`: Redis for mission timers and lobbies.
    - Optional `api` service wired to build from `[apps/api/Dockerfile]` and depend on `postgres` and `redis`.
  - Create Dockerfiles:
    - `[apps/api/Dockerfile]` building the NestJS API (install deps, build via Nx, run with `node dist/apps/api/main.js`).
    - `[apps/twitch-extension-panel/Dockerfile]` for building and serving the Angular extension bundle for local testing.
  - Define environment configuration via `.env`/`.env.local` for DB and Redis connection strings consumed by the API app.
- **[NEW] Local development without Twitch**
  - Since `window.Twitch.ext` is unavailable in a local browser, a missing stub causes the panel to hang silently on startup. Solve this early to keep frontend development fluid.
  - Create a `TwitchExtMock` module at `[apps/twitch-extension-panel/src/mocks/twitch-ext.mock.ts]` that:
    - Exposes the same interface as `window.Twitch.ext` (`onAuthorized`, `onContext`, `onVisibilityChanged`, `listen`).
    - Returns a hardcoded fake JWT and viewer ID so all downstream services initialise normally.
    - Is swapped in via an `environment.ts` flag (`environment.mock = true`) — never shipped in production builds.
  - Add a `local-dev` npm script that sets the mock flag and serves the panel against a local API.

---

### Stage 2: The Extension Frontend (Angular Panel)

- **Panel shell & layout**
  - In `twitch-extension-panel`, create a `PanelShellComponent` (e.g. `[apps/twitch-extension-panel/src/app/panel-shell.component.ts]`) that:
    - Hosts the Live Log area and Survivor Status area.
    - Handles initial Twitch Extension context/bootstrap (mounts only after `window.Twitch.ext.onAuthorized`, or mock equivalent in dev).
  - Add presentational components:
    - `LiveLogComponent` for a scrolling log of encounters/events (Progress Quest style).
    - `SurvivorStatusComponent` for current survivor(s), health/injury state, mission status, and perks.
- **Twitch EBS integration & auth**
  - Implement a `TwitchAuthService` (e.g. `[apps/twitch-extension-panel/src/app/services/twitch-auth.service.ts]`) that:
    - Wraps `window.Twitch.ext` calls (`onAuthorized`, `onContext`, `onVisibilityChanged`).
    - Stores the latest JWT and parsed payload (obfuscated Twitch extension user ID).
  - Implement an `EbsApiService` that:
    - Attaches the Twitch JWT as `Authorization: Bearer <token>` on all HTTP calls to the NestJS API.
    - Exposes methods like `startMission`, `getMissionState`, `getPerkInventory` using types from `@fog-explorer/api-interfaces`.
- **[NEW] EBS resilience & rate limiting**
  - At scale, thousands of concurrent viewers can simultaneously poll `GET /missions/state`, overwhelming the API.
  - Cache `GET /missions/state` responses in Redis with a 2–3 second TTL to absorb viewer polling spikes.
  - The panel should degrade gracefully when the EBS is unreachable — display last-known state rather than a broken UI, and surface a subtle "reconnecting…" indicator.
  - Add client-side exponential back-off with jitter for all EBS fetch retries.
- **State & real-time mission display**
  - Create a `MissionStateStore` using Angular signals (or RxJS `BehaviorSubject` as a fallback) to hold:
    - Current mission snapshot (`MissionState` + encounter history).
    - Survivor state(s) and perk inventory.
  - Wire the store to:
    - Pull initial state from the API (`getMissionState`).
    - Subscribe to Twitch PubSub events (via the Twitch JS helper) to apply incremental updates (new log lines, state transitions).
  - Bind `PanelShellComponent`, `LiveLogComponent`, and `SurvivorStatusComponent` to this store using `computed` signals or `async` pipe for efficient change detection.
- **[NEW] PubSub message size guard**
  - Twitch Extension PubSub enforces a 5 KB per-message limit. Unchecked log growth silently drops messages.
  - Send only the latest N log lines per PubSub message (recommend N = 10–15).
  - If the panel misses messages (detected via a sequence counter), trigger a full `getMissionState` re-fetch as a fallback.

---

### Stage 3: The ZPG Mission Engine ("Corso")

- **Global heartbeat & tick worker**
  - In the NestJS API, add a `TickEngineModule` (e.g. `[apps/api/src/app/tick-engine/tick-engine.module.ts]`) and a `TickService` that:
    - Uses `@nestjs/schedule` (cron) or a background worker to trigger a global tick every 60 seconds.
    - Acquires a Redis-based distributed lock (e.g. `SETNX tick_lock`) to ensure only one instance runs the tick.
    - Scans active missions from Redis and dispatches them to the encounter resolver.
- **[NEW] Tick engine resilience**
  - Two failure modes need explicit answers before production: long ticks and process restarts.
  - Set the Redis lock TTL to 50 seconds (10-second buffer before the next tick fires). If a tick is still running when the lock expires, emit a structured warning log — do not silently skip.
  - Make encounter resolution idempotent: store a tick sequence number per mission in Redis and skip re-processing if the current tick index has already been resolved. This protects against double-application on restart mid-tick.
  - Log tick start, duration, and number of missions processed as structured fields on every execution.
- **Encounter resolver (stateless core)**
  - Implement a pure, stateless function in `[libs/api-interfaces/src/lib/encounter-resolver.ts]` or a new logic lib (e.g. `libs/mission-logic`):
    - Signature like `resolveEncounter(survivor: SurvivorStats, perks: PerkModifier[], difficulty: Difficulty): EncounterResult`.
    - Uses RNG with a pluggable seed source for deterministic tests.
    - Applies modifiers as P{success} = Base + Σ Modifiers clamped to sane bounds.
  - In the API, create an `EncounterService` that:
    - Calls the resolver for each mission tick.
    - Updates mission state (success/fail, injury, sacrifice) and logs events.
- **Group logic & SWF mechanics**
  - Introduce a `GroupSynergyService` that:
    - Accepts a group of 2–4 survivors and their perks.
    - Aggregates "Team Perk" modifiers, applying them to all relevant rolls.
    - Computes group-level outcomes (e.g., shared progress, split rewards, synchronized failure states).
  - Extend the encounter resolver inputs to accept a `groupContext` that influences difficulty modifiers for group missions.
- **Redis for active missions & lobbies**
  - Define Redis data structures in the API layer:
    - `active_mission:{missionId}`: JSON or hash storing mission state, participant IDs, nextTickAt.
    - `mission_lobby:{lobbyId}`: state for lobby members, mission template, and ready flags.
  - Implement a `MissionStore` repository that abstracts Redis operations (get/set/expire/lists) from the core mission logic.

---

### Stage 4: Twitch Extension Persistence (EBS)

- **Extension action handlers instead of chat commands**
  - In the NestJS `MissionsModule`, add controllers like `[apps/api/src/app/missions/missions.controller.ts]` exposing:
    - `POST /missions/start` to replace `!explore` (triggered by the panel "Start Mission" button).
    - `GET /missions/state` for current mission snapshot.
    - `POST /missions/choose-perk` (future) or similar progression actions.
  - Use DTOs shaped by `@fog-explorer/api-interfaces` types for request/response contracts.
- **Twitch JWT verification & identity mapping**
  - Implement a `TwitchAuthModule` and `TwitchAuthGuard` that:
    - Verifies incoming JWTs from the extension using the Twitch extension secret.
    - Extracts the obfuscated Twitch user ID and extension channel ID, attaching them to the Nest request context.
- **[NEW] JWT secret rotation handling**
  - Twitch can rotate the extension secret, causing all JWT verification to fail globally until the API is redeployed.
  - On a JWT verification failure, the guard should attempt to re-fetch the current secret from the Twitch API before returning 401.
  - Cache the fetched secret with a reasonable TTL (e.g. 5 minutes) so rotation recovery is near-instant.
  - Emit a structured alert log whenever a secret re-fetch occurs — this is a high-signal operational event.
- **Postgres schema**
  - Design Postgres tables (via migrations/TypeORM/Prisma) in the API project:
    - `users` (internal ID, Twitch extension user ID, created_at).
    - `survivors` (FK to `users`, stats, perk slots, current state: active/injured/sacrificed). Use soft-delete / `sacrificed_at` timestamp rather than hard deletion to support history and leaderboards.
    - `missions` (FK to `survivors` or group, status, difficulty, timestamps).
    - `mission_logs` (FK to `missions`, tick index, encounter key, rendered text, RNG details if needed).
- **[NEW] mission_logs retention strategy**
  - `mission_logs` can grow unboundedly — a channel with high traffic can generate millions of rows within weeks.
  - Add an `archived_at` nullable column to `mission_logs` from day one.
  - Implement a lightweight nightly archival job that marks logs for completed/failed missions older than N days as archived (or bulk-moves them to an archive table).
  - Add a Postgres index on `(mission_id, archived_at)` to keep active-mission log queries fast regardless of historical volume.
- **Twitch PubSub integration for ticks**
  - Create a `TwitchPubSubService` in the API that:
    - Signs messages to Twitch's Extension PubSub endpoint using the extension client ID and secret.
    - Publishes log updates and mission state deltas on each tick for relevant channels.
  - On the frontend, extend `TwitchAuthService` or a dedicated `PubSubService` to:
    - Subscribe to mission update topics.
    - Forward received events into the `MissionStateStore`, appending log lines and updating survivor/mission state.
- **[NEW] Broadcaster configuration panel**
  - Broadcasters frequently need to configure extension behaviour (difficulty presets, max survivors, opt-in features). This is often needed earlier than expected.
  - Create a separate Angular app (or a route within the panel) for the Twitch Broadcaster Config view.
  - Add a `POST /channel/config` EBS endpoint that accepts channel-level settings and stores them against the channel ID in Postgres.
  - The config schema should at minimum cover: mission difficulty preset, max party size, and feature flags for beta mechanics.
  - Apply these settings in `MissionsModule` when resolving encounters for that channel.

---

### Stage 5: Content Library & Balance

- **JSON encounter library**
  - Create a dedicated library `@fog-explorer/encounter-library` at `[libs/encounter-library/src/lib/encounters.json]` to hold all flavor text and base parameters.
  - Structure encounters as JSON records:
    - Keys like `"cleanse_hex"`, `"escape_hatch"`, etc.
    - Fields for base success chance, difficulty tags, flavor text variants, and perk tags.
  - Add a small TypeScript wrapper in `[libs/encounter-library/src/lib/encounter-library.ts]` to:
    - Load encounters.
    - Expose helper functions (e.g. `getEncounterById`, `getRandomEncounterByTier`).
- **[NEW] Encounter JSON schema versioning**
  - Adding new fields to `encounters.json` silently breaks records that predate those fields, causing runtime surprises.
  - Define a JSON Schema (or Zod schema) for the encounter record format and commit it alongside `encounters.json`.
  - Add a build-time validation step (Nx target) that validates all encounter records against the schema on every build.
  - Increment a `schemaVersion` field whenever breaking fields are added, and provide a migration helper for existing records.
- **Perk system & balance utilities**
  - In `@fog-explorer/api-interfaces` (or a dedicated `@fog-explorer/perks` lib), formalize perk modifier types:
    - Additive and multiplicative modifiers, flat reroll mechanics, group-boosting perks.
  - Implement a `PerkMath` helper that:
    - Aggregates modifiers into a final P{success} value.
    - Separates survivor-level modifiers from team-level modifiers to keep SWF behaviour clear.
  - Provide a small test harness (unit tests in the mission logic lib) to:
    - Validate that typical perk loadouts produce reasonable success/fail/injury rates.
    - Guard against accidental balance regressions.
- **[NEW] Simulation CLI for balance tuning**
  - Unit tests confirm correctness but are too coarse for tuning probability distributions. A headless simulator is far more useful.
  - Add an Nx target: `nx run mission-logic:simulate -- --runs=10000 --perkSet=flashlight,spine_chill`
  - The simulator should output: mean success rate, injury rate, sacrifice rate, and a percentile breakdown of mission lengths.
  - Wire it into CI with a fixed seed so regressions in outcome distributions surface as test failures without flakiness.

---

### [NEW] Cross-Cutting Concerns

These topics were absent from the original plan but span all stages and should be established early.

- **Observability**
  - Install Pino as the NestJS logger (structured JSON output). Replace all `console.log` usage with pino log calls.
  - Define a standard log schema with fields: `traceId`, `channelId`, `missionId`, `tickIndex`, `durationMs`, `event`.
  - Instrument the tick engine specifically: log tick start, active mission count, tick duration, and any per-mission errors as structured fields on every execution.
  - Add basic application metrics (tick processing time, encounter outcomes, error rates) — even a simple in-memory counter exported to a `/metrics` endpoint is sufficient to start.

---

### High-Level Architecture Diagram

```mermaid
flowchart LR
  viewerPanel["Twitch Panel (Angular)"] -->|"JWT via window.Twitch.ext"| apiGateway["NestJS API (EBS)"]
  apiGateway -->|"REST: /missions/*"| missionEngine["Mission Engine / Tick Service"]
  missionEngine -->|"state + logs"| postgresDb["Postgres"]
  missionEngine -->|"timers + lobbies"| redisStore["Redis"]
  missionEngine -->|"Tick updates"| twitchPubSub["Twitch Extension PubSub"]
  twitchPubSub -->|"Push events"| viewerPanel
  apiGateway -->|"channel config"| postgresDb
  broadcasterConfig["Broadcaster Config Panel"] -->|"POST /channel/config"| apiGateway
```