When we started experimenting with AERIS RPG, the idea sounded fairly simple:

Put a local language model behind a fantasy RPG and let it be the Dungeon Master.

Easy.

Well, no.

It turns out that getting an AI to write like a Dungeon Master is surprisingly easy.

Getting it to behave like one is something else entirely.

A story generator isn’t a Dungeon Master

A language model is perfectly happy to describe a dark forest, an abandoned village or a suspicious-looking stone.

And it’s often very good at it.

But a Dungeon Master does something more interesting.

Imagine the DM casually mentions a lantern appearing at the top of an old ruined tower.

Maybe it was just atmosphere.

The player, however, stops.

“Who lit that?”

And suddenly the lantern matters.

Perhaps the DM had absolutely no idea who lit it five seconds earlier. That’s fine. A human DM can improvise an answer, connect it to something else happening in the world and continue.

But from that moment on, the answer becomes real.

The player saw the lantern. Someone — or something — presumably lit it.

Whatever the DM invents next must now coexist with everything that has already happened.

This is the part of tabletop role-playing that interests us.

Not generating an infinite amount of fantasy text.

Improvising a world and then respecting what has become true.

AI has a conveniently flexible memory

Our artificial DM is quite capable of improvising.

Sometimes a little too capable.

Tell it that the player has cut a tripwire and it can produce a lovely description of the trap being disabled.

Come back a little later and there’s always the possibility that our enthusiastic Dungeon Master will dramatically discover…

…the same tripwire.

Apparently it repaired itself while nobody was looking.

Funny during development.

Probably less funny after ten hours of adventuring.

So AERIS is slowly becoming a hybrid.

We want the AI to remain free to invent characters, places, events and consequences. But some things cannot remain merely suggestions floating around inside a language model.

If the player takes an object, somebody owns it.

If Aric gets off his horse, he is no longer riding it.

If something is broken, opened, discovered or destroyed, the world should remember.

And if an apparently meaningless lantern becomes important because the player decides to investigate it, perhaps that lantern deserves to become part of the world’s history too.

The complexity should be in the DM, not in the rules

This has led us to one of the principles we’re using while developing AERIS:

The complexity should be in the DM, not in the rules.

We don’t want to solve every possible problem by building an enormous traditional RPG engine underneath the AI.

The rules should remain relatively simple.

The difficult part should be teaching the artificial Dungeon Master when to improvise, when to remember, when to ask for a dice roll, when to accept what has already happened and — perhaps most importantly — when not to invent something.

We’re still figuring that out.

And occasionally breaking it.

Then we gave it pictures.

Because apparently one AI-related problem at a time wasn’t enough.

AERIS can also ask ComfyUI, running locally, to generate an image inspired by what’s happening in the adventure.

Sometimes the results are surprisingly atmospheric.

Sometimes we get this.

Image generation, as it turns out, has its own interpretation of world consistency.

We’re working on that too.

The man lying next to the cart was unavailable for comment.

All of this runs locally

There is one additional complication.

We’re deliberately trying to do all this on our own hardware.

The Dungeon Master currently runs as a small local language model. ComfyUI generates the images locally too.

The machine responsible for this technological revolution is equipped with an Intel i7-8700, 32 GB of RAM and an NVIDIA RTX 2070 with 8 GB of VRAM.

So… not exactly an AI supercomputer.

That’s intentional.

There are already plenty of fascinating AI projects demonstrating what can be achieved with enormous models and expensive GPUs.

We’re interested in a slightly different question:

What can you build with the computer you already have?

A lot of people interested in local AI don’t have a high-end GPU sitting under their desk. Neither do we.

If we can eventually get a convincing artificial Dungeon Master, persistent world, locally generated illustrations and an actual playable RPG running on this machine, that’s arguably more interesting to us than discovering that it works on hardware costing several thousand euros.

So far, the answer seems to be:

quite a lot, provided you’re willing to argue with an AI about horses, daggers, previously cut pieces of string and the generally accepted number of wheels on a medieval cart.

The RTX 2070 has not been consulted about any of this.

We’ll keep experimenting.

Next time, we’ll introduce the three unfortunate individuals currently being used to test all of it:

Aric, Mira and Dawn.