maurycy/fog-explorer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Hussar
2026-04-12 15:35:50 +00:00
commit 42d20cb0ed
80 changed files with 2210 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

234
fog/plan.md Executable file
View File

@@ -0,0 +1,234 @@
---
name: fog-expedition-nx-and-mission-engine
overview: Set up an Nx-based monorepo for the Fog Expedition Twitch extension, implement a NestJS API with a tick-based mission engine, and integrate Twitch Extension auth, persistence, and content balance libraries.
todos:
- id: stage1-nx-foundation
content: Set up Nx workspace, add Angular/Nest plugins, generate Angular panel app, NestJS API app, and shared @fog-explorer/api-interfaces library, plus Docker Compose for Postgres and Redis.
status: completed
- id: stage2-extension-frontend
content: Build the Twitch panel Angular UI (panel shell, live log, survivor status) and integrate Twitch Extension auth plus an EBS HTTP client using shared interfaces.
status: completed
- id: stage3-mission-engine
content: Implement the 60-second tick engine, stateless encounter resolver, Redis-backed mission/lobby store, and SWF group logic services in the API.
status: completed
- id: stage4-ebs-persistence
content: Add NestJS controllers, Twitch JWT guard, Postgres models/migrations, and Twitch PubSub integration to replace chat commands with extension-driven actions.
status: completed
- id: stage5-content-balance
content: Create the JSON encounter library, formalize perk and modifier types, and add utilities/tests to tune and validate mission outcome probabilities.
status: completed
isProject: false
---
## Fog Expedition Nx Workspace & ZPG Engine Plan
> **v2 note:** Additions introduced in this revision are marked **[NEW]** throughout.
---
### Stage 1: Workspace & Infrastructure (Nx Foundation)
- **Initialize Nx workspace & tooling**
- Ensure the root contains an Nx workspace with TypeScript support (`nx.json`, `project.json`/`workspace.json`, `tsconfig.base.json`, `package.json`).
- Add Nx plugins for Angular and NestJS (e.g. `@nx/angular`, `@nx/nest`, `@nx/node`).
- Configure `package.json` scripts for running Nx tasks (e.g. `"nx": "nx"`, `"start": "nx serve twitch-extension-panel"`).
- **Generate core apps**
- Create an Angular app for the Twitch Extension Panel at `[apps/twitch-extension-panel/src/main.ts]` via Nx generator (standalone components, no routing complexity initially).
- Create a NestJS API app at `[apps/api/src/main.ts]` using Nx's Nest generator, structured with modules for `Auth`, `Missions`, and `TickEngine`.
- **Shared API interfaces library**
- Generate a shared TypeScript library `@fog-explorer/api-interfaces` at `[libs/api-interfaces/src/index.ts]`.
- Define core domain types in small focused files:
- `[libs/api-interfaces/src/lib/survivor.ts]` containing `Survivor`, `SurvivorStats`, `SurvivorState` (`Active`, `Injured`, `Sacrificed`).
- `[libs/api-interfaces/src/lib/mission.ts]` containing `Mission`, `MissionState` (`Lobby`, `InProgress`, `Completed`, `Failed`), `EncounterResult`.
- `[libs/api-interfaces/src/lib/perk.ts]` containing `Perk`, `PerkModifier`, and `TeamPerk` flags for SWF mechanics.
- Re-export all public contracts from `[libs/api-interfaces/src/index.ts]` for clean imports in both `twitch-extension-panel` and `api`.
- **Docker & Docker Compose for infra**
- Add a root `docker-compose.yml` defining services:
- `postgres`: PostgreSQL with exposed port, DB name `fog_expedition`, volume, and healthcheck.
- `redis`: Redis for mission timers and lobbies.
- Optional `api` service wired to build from `[apps/api/Dockerfile]` and depend on `postgres` and `redis`.
- Create Dockerfiles:
- `[apps/api/Dockerfile]` building the NestJS API (install deps, build via Nx, run with `node dist/apps/api/main.js`).
- `[apps/twitch-extension-panel/Dockerfile]` for building and serving the Angular extension bundle for local testing.
- Define environment configuration via `.env`/`.env.local` for DB and Redis connection strings consumed by the API app.
- **[NEW] Local development without Twitch**
- Since `window.Twitch.ext` is unavailable in a local browser, a missing stub causes the panel to hang silently on startup. Solve this early to keep frontend development fluid.
- Create a `TwitchExtMock` module at `[apps/twitch-extension-panel/src/mocks/twitch-ext.mock.ts]` that:
- Exposes the same interface as `window.Twitch.ext` (`onAuthorized`, `onContext`, `onVisibilityChanged`, `listen`).
- Returns a hardcoded fake JWT and viewer ID so all downstream services initialise normally.
- Is swapped in via an `environment.ts` flag (`environment.mock = true`) — never shipped in production builds.
- Add a `local-dev` npm script that sets the mock flag and serves the panel against a local API.
---
### Stage 2: The Extension Frontend (Angular Panel)
- **Panel shell & layout**
- In `twitch-extension-panel`, create a `PanelShellComponent` (e.g. `[apps/twitch-extension-panel/src/app/panel-shell.component.ts]`) that:
- Hosts the Live Log area and Survivor Status area.
- Handles initial Twitch Extension context/bootstrap (mounts only after `window.Twitch.ext.onAuthorized`, or mock equivalent in dev).
- Add presentational components:
- `LiveLogComponent` for a scrolling log of encounters/events (Progress Quest style).
- `SurvivorStatusComponent` for current survivor(s), health/injury state, mission status, and perks.
- **Twitch EBS integration & auth**
- Implement a `TwitchAuthService` (e.g. `[apps/twitch-extension-panel/src/app/services/twitch-auth.service.ts]`) that:
- Wraps `window.Twitch.ext` calls (`onAuthorized`, `onContext`, `onVisibilityChanged`).
- Stores the latest JWT and parsed payload (obfuscated Twitch extension user ID).
- Implement an `EbsApiService` that:
- Attaches the Twitch JWT as `Authorization: Bearer <token>` on all HTTP calls to the NestJS API.
- Exposes methods like `startMission`, `getMissionState`, `getPerkInventory` using types from `@fog-explorer/api-interfaces`.
- **[NEW] EBS resilience & rate limiting**
- At scale, thousands of concurrent viewers can simultaneously poll `GET /missions/state`, overwhelming the API.
- Cache `GET /missions/state` responses in Redis with a 2–3 second TTL to absorb viewer polling spikes.
- The panel should degrade gracefully when the EBS is unreachable — display last-known state rather than a broken UI, and surface a subtle "reconnecting…" indicator.
- Add client-side exponential back-off with jitter for all EBS fetch retries.
- **State & real-time mission display**
- Create a `MissionStateStore` using Angular signals (or RxJS `BehaviorSubject` as a fallback) to hold:
- Current mission snapshot (`MissionState` + encounter history).
- Survivor state(s) and perk inventory.
- Wire the store to:
- Pull initial state from the API (`getMissionState`).
- Subscribe to Twitch PubSub events (via the Twitch JS helper) to apply incremental updates (new log lines, state transitions).
- Bind `PanelShellComponent`, `LiveLogComponent`, and `SurvivorStatusComponent` to this store using `computed` signals or `async` pipe for efficient change detection.
- **[NEW] PubSub message size guard**
- Twitch Extension PubSub enforces a 5 KB per-message limit. Unchecked log growth silently drops messages.
- Send only the latest N log lines per PubSub message (recommend N = 10–15).
- If the panel misses messages (detected via a sequence counter), trigger a full `getMissionState` re-fetch as a fallback.
---
### Stage 3: The ZPG Mission Engine ("Corso")
- **Global heartbeat & tick worker**
- In the NestJS API, add a `TickEngineModule` (e.g. `[apps/api/src/app/tick-engine/tick-engine.module.ts]`) and a `TickService` that:
- Uses `@nestjs/schedule` (cron) or a background worker to trigger a global tick every 60 seconds.
- Acquires a Redis-based distributed lock (e.g. `SETNX tick_lock`) to ensure only one instance runs the tick.
- Scans active missions from Redis and dispatches them to the encounter resolver.
- **[NEW] Tick engine resilience**
- Two failure modes need explicit answers before production: long ticks and process restarts.
- Set the Redis lock TTL to 50 seconds (10-second buffer before the next tick fires). If a tick is still running when the lock expires, emit a structured warning log — do not silently skip.
- Make encounter resolution idempotent: store a tick sequence number per mission in Redis and skip re-processing if the current tick index has already been resolved. This protects against double-application on restart mid-tick.
- Log tick start, duration, and number of missions processed as structured fields on every execution.
- **Encounter resolver (stateless core)**
- Implement a pure, stateless function in `[libs/api-interfaces/src/lib/encounter-resolver.ts]` or a new logic lib (e.g. `libs/mission-logic`):
- Signature like `resolveEncounter(survivor: SurvivorStats, perks: PerkModifier[], difficulty: Difficulty): EncounterResult`.
- Uses RNG with a pluggable seed source for deterministic tests.
- Applies modifiers as P{success} = Base + Σ Modifiers clamped to sane bounds.
- In the API, create an `EncounterService` that:
- Calls the resolver for each mission tick.
- Updates mission state (success/fail, injury, sacrifice) and logs events.
- **Group logic & SWF mechanics**
- Introduce a `GroupSynergyService` that:
- Accepts a group of 2–4 survivors and their perks.
- Aggregates "Team Perk" modifiers, applying them to all relevant rolls.
- Computes group-level outcomes (e.g., shared progress, split rewards, synchronized failure states).
- Extend the encounter resolver inputs to accept a `groupContext` that influences difficulty modifiers for group missions.
- **Redis for active missions & lobbies**
- Define Redis data structures in the API layer:
- `active_mission:{missionId}`: JSON or hash storing mission state, participant IDs, nextTickAt.
- `mission_lobby:{lobbyId}`: state for lobby members, mission template, and ready flags.
- Implement a `MissionStore` repository that abstracts Redis operations (get/set/expire/lists) from the core mission logic.
---
### Stage 4: Twitch Extension Persistence (EBS)
- **Extension action handlers instead of chat commands**
- In the NestJS `MissionsModule`, add controllers like `[apps/api/src/app/missions/missions.controller.ts]` exposing:
- `POST /missions/start` to replace `!explore` (triggered by the panel "Start Mission" button).
- `GET /missions/state` for current mission snapshot.
- `POST /missions/choose-perk` (future) or similar progression actions.
- Use DTOs shaped by `@fog-explorer/api-interfaces` types for request/response contracts.
- **Twitch JWT verification & identity mapping**
- Implement a `TwitchAuthModule` and `TwitchAuthGuard` that:
- Verifies incoming JWTs from the extension using the Twitch extension secret.
- Extracts the obfuscated Twitch user ID and extension channel ID, attaching them to the Nest request context.
- **[NEW] JWT secret rotation handling**
- Twitch can rotate the extension secret, causing all JWT verification to fail globally until the API is redeployed.
- On a JWT verification failure, the guard should attempt to re-fetch the current secret from the Twitch API before returning 401.
- Cache the fetched secret with a reasonable TTL (e.g. 5 minutes) so rotation recovery is near-instant.
- Emit a structured alert log whenever a secret re-fetch occurs — this is a high-signal operational event.
- **Postgres schema**
- Design Postgres tables (via migrations/TypeORM/Prisma) in the API project:
- `users` (internal ID, Twitch extension user ID, created_at).
- `survivors` (FK to `users`, stats, perk slots, current state: active/injured/sacrificed). Use soft-delete / `sacrificed_at` timestamp rather than hard deletion to support history and leaderboards.
- `missions` (FK to `survivors` or group, status, difficulty, timestamps).
- `mission_logs` (FK to `missions`, tick index, encounter key, rendered text, RNG details if needed).
- **[NEW] mission_logs retention strategy**
- `mission_logs` can grow unboundedly — a channel with high traffic can generate millions of rows within weeks.
- Add an `archived_at` nullable column to `mission_logs` from day one.
- Implement a lightweight nightly archival job that marks logs for completed/failed missions older than N days as archived (or bulk-moves them to an archive table).
- Add a Postgres index on `(mission_id, archived_at)` to keep active-mission log queries fast regardless of historical volume.
- **Twitch PubSub integration for ticks**
- Create a `TwitchPubSubService` in the API that:
- Signs messages to Twitch's Extension PubSub endpoint using the extension client ID and secret.
- Publishes log updates and mission state deltas on each tick for relevant channels.
- On the frontend, extend `TwitchAuthService` or a dedicated `PubSubService` to:
- Subscribe to mission update topics.
- Forward received events into the `MissionStateStore`, appending log lines and updating survivor/mission state.
- **[NEW] Broadcaster configuration panel**
- Broadcasters frequently need to configure extension behaviour (difficulty presets, max survivors, opt-in features). This is often needed earlier than expected.
- Create a separate Angular app (or a route within the panel) for the Twitch Broadcaster Config view.
- Add a `POST /channel/config` EBS endpoint that accepts channel-level settings and stores them against the channel ID in Postgres.
- The config schema should at minimum cover: mission difficulty preset, max party size, and feature flags for beta mechanics.
- Apply these settings in `MissionsModule` when resolving encounters for that channel.
---
### Stage 5: Content Library & Balance
- **JSON encounter library**
- Create a dedicated library `@fog-explorer/encounter-library` at `[libs/encounter-library/src/lib/encounters.json]` to hold all flavor text and base parameters.
- Structure encounters as JSON records:
- Keys like `"cleanse_hex"`, `"escape_hatch"`, etc.
- Fields for base success chance, difficulty tags, flavor text variants, and perk tags.
- Add a small TypeScript wrapper in `[libs/encounter-library/src/lib/encounter-library.ts]` to:
- Load encounters.
- Expose helper functions (e.g. `getEncounterById`, `getRandomEncounterByTier`).
- **[NEW] Encounter JSON schema versioning**
- Adding new fields to `encounters.json` silently breaks records that predate those fields, causing runtime surprises.
- Define a JSON Schema (or Zod schema) for the encounter record format and commit it alongside `encounters.json`.
- Add a build-time validation step (Nx target) that validates all encounter records against the schema on every build.
- Increment a `schemaVersion` field whenever breaking fields are added, and provide a migration helper for existing records.
- **Perk system & balance utilities**
- In `@fog-explorer/api-interfaces` (or a dedicated `@fog-explorer/perks` lib), formalize perk modifier types:
- Additive and multiplicative modifiers, flat reroll mechanics, group-boosting perks.
- Implement a `PerkMath` helper that:
- Aggregates modifiers into a final P{success} value.
- Separates survivor-level modifiers from team-level modifiers to keep SWF behaviour clear.
- Provide a small test harness (unit tests in the mission logic lib) to:
- Validate that typical perk loadouts produce reasonable success/fail/injury rates.
- Guard against accidental balance regressions.
- **[NEW] Simulation CLI for balance tuning**
- Unit tests confirm correctness but are too coarse for tuning probability distributions. A headless simulator is far more useful.
- Add an Nx target: `nx run mission-logic:simulate -- --runs=10000 --perkSet=flashlight,spine_chill`
- The simulator should output: mean success rate, injury rate, sacrifice rate, and a percentile breakdown of mission lengths.
- Wire it into CI with a fixed seed so regressions in outcome distributions surface as test failures without flakiness.
---
### [NEW] Cross-Cutting Concerns
These topics were absent from the original plan but span all stages and should be established early.
- **Observability**
- Install Pino as the NestJS logger (structured JSON output). Replace all `console.log` usage with pino log calls.
- Define a standard log schema with fields: `traceId`, `channelId`, `missionId`, `tickIndex`, `durationMs`, `event`.
- Instrument the tick engine specifically: log tick start, active mission count, tick duration, and any per-mission errors as structured fields on every execution.
- Add basic application metrics (tick processing time, encounter outcomes, error rates) — even a simple in-memory counter exported to a `/metrics` endpoint is sufficient to start.
---
### High-Level Architecture Diagram
```mermaid
flowchart LR
viewerPanel["Twitch Panel (Angular)"] -->|"JWT via window.Twitch.ext"| apiGateway["NestJS API (EBS)"]
apiGateway -->|"REST: /missions/*"| missionEngine["Mission Engine / Tick Service"]
missionEngine -->|"state + logs"| postgresDb["Postgres"]
missionEngine -->|"timers + lobbies"| redisStore["Redis"]
missionEngine -->|"Tick updates"| twitchPubSub["Twitch Extension PubSub"]
twitchPubSub -->|"Push events"| viewerPanel
apiGateway -->|"channel config"| postgresDb
broadcasterConfig["Broadcaster Config Panel"] -->|"POST /channel/config"| apiGateway
```