In our ongoing quest to create a seamless fusion of tabletop storytelling and generative AI, we’ve encountered a common challenge: how to make the visual environment feel alive without it being distracted by every minor player movement or technical debug message. Today, we are excited to share a significant step forward in our visual engine: the Visual Prompt Refresh.
The Problem: Static Environments
Previously, our image generation system relied heavily on the overall scene state. While this worked for static locations, it often led to ‘visual stagnation.’ If the player moved through a room or a combat encounter began, the AI image might remain stuck on the initial setting because the primary scene parameters hadn’t changed significantly enough to trigger a new generation. Conversely, we didn’t want every single player action—like a character drawing a sword—to trigger a full, expensive image redraw.
The Solution: The ‘Visual Moment’
We have implemented a new layer in our visual projection engine called the Visual Moment. This feature extracts a transient, descriptive cue from the Narrator’s (Master’s) most recent accepted narration. It is designed to be a bridge between the narrative flow and the visual output.
How it works:
- Narrative Recognition: The engine scans the latest narration for specific visual markers—landmarks, light, movement, or environmental shifts (e.g., ‘a lantern illuminating a stone bell tower’ or ‘a bridge across a river’).
- Filtering the Noise:s Crucially, the system ignores player actions, speculative descriptions, and technical debug messages. Only narrations that describe a significant visual change are considered ‘canonical’ for the image prompt.
- Seamless Integration:s This ‘moment’ is passed through our translation layer and integrated directly into the SDXL prompt via ComfyUI, ensuring that the next image generated reflects the current, active moment of the story.
Technical Overview
This update (part of our 0.5.x development cycle) involves several key components:
- Scene Engine:s The
engine/scene.pynow contains logic to extract these moments, ensuring they remain transient and do not pollute the long-term scene memory. - Translation Layer:s Our
images/english.pyhas been updated to include this visual context in the prompt payload. - Visual Generator:s The prompt construction in
images/comfy.pynow synthesizes these moments into the final visual description, creating a dynamic, reactive environment.
Current Status: Candidate Release
This feature is currently in the Candidate phase. It has passed initial unit testing, but awaits full end-to-end playtesting on mobile and desktop platforms to ensure the visual transitions feel natural and performant.
Stay tuned for more updates as we continue to refine the boundary between the Dungeon Master’s freedom and the Engine’s rules. The complexity should stay in the DM, not the rules.
