Using general intelligence to train specialised intelligence

I've been working on a website for a traditional Greek card game called Diloti. This started at first as an experiment to see if AI coding agents can making building such a game easier. But as models became better and better, the experiments became more ambitious. Older models were struggling with simple challenges, like for example the ARC-AGI benchmark where the challenges are easy for humans but meant to be hard for LLMs. Seeing a lot of progress in these benchmarks made me think: what if LLMs could play the game well just by reading the rules?

I suggest you watch this video where I cover most of this. This article was primarily meant to get this video on my blog, so I drafted up a tiny summary. It's not bad though, I hope.

LLMs as Diloti players

I started with the simplest of experiments. I provided the rules of the game as part of the system prompt and a full representation of public game state in a JSON format. I asked the models to pick the legal move that they thought is best. Whenever I tried to not provide a list of legal moves generated by the game engine, the models would occasionally fail to derive a legal move from the game state and would propose an impossible move.

This showed promising results at first, with Claude Opus 4.5 leading in evals. I iterated with prompts for a bit, and ran various experiments to improve performance and got to a point where I was sastisfied that the models had some level of game understanding. So I started to play against the models.

That's when I discovered a limitation that I hadn't previously thought about: the good models were extremely slow and not fun to play against. I could double down and invest further, trying to get more gains through prompt engineering, move history, giving models memory from prior moves, etc. But if I added more layers, the interaction would likely become even slower. So I decided to pause on LLMs as players; that said, I might come back to this and report back on how newer models are performing.

Training a model using reinforcement learning

I didn't find the performance of LLMs very impressive in this unknown domain that's definitely not part of their training data since the game is not known outside Greece and only has a handful of online references. What I decided to experiment with next was to see if general intelligence could be used to train specialised intelligence. My goal was to find out whether LLMs can read the game rules and use the game engine that I've already built to train a specialised model that can beat humans in this game.

Were LLMs able to one-shot the training to a meaningful point?

No, not really. At least not on a consumer budget. Maybe with unlimited tokens and a big fat loop you'd be able to get further along, but neither Claude Code or Codex (ChatGPT Work) were able to crack this quickly.

That said, LLMs were incredibly helpful. I'm far from an expert in the field; I've read books, seen YouTube videos, trained a few simple models by following instructions or using simple libraries, but I've never been a machine learning professional and it's not something I get to practice with a lot. Despite that I was able to get to a seriously impressive result: a model that beats a "best-practice" scripted agent 98% of the time, and an even better search-augmented model that's able to beat that best model 90% of the time. The search-augmented model has been tested against humans and despite not consistently exceeding human performance, it has beat me and other players in online games. (~30-40% win rate in a small sample)

This result might seem low, but having been through many different model generations whose performance would likely not exceed 5-10%, I'm quite pleased with that! And there's still a lot more to unlock, thanks to GPT 5.6 Sol Extra High :)

Collaborating with LLMs on an unknown domain

This was very much a side project. Whenever I had 30 minutes to spare, I would discuss experiment ideas with Codex/GPT5 series, let things run overnight and come back and made adjustments the next day. With that kind of time investment, it's not possible to dive deep, understand all the details, or read related research papers. I had to operate at a higher level of abstraction, acting as the human expert that knows the game well, and probing the model to ensure its understanding is correct and that the training curriculum makes sense.

These are some of the tools that I built to interact with the models and guide them in the right direction. It's seriously easy to one-shot these kinds of tools, and I can attribute a lot of the success in improving model performance to these tools and the interactions that they allowed.

Training debugger

I built a debugger where I could follow the reinforcement learning training move-by-move. After each move I could see the exact signal that was being passed to the model and I would feed that back to the model whenever I believed rewards/penalties were wrong or disproportionate.

Eval editor

I created 1-2 dozen evaluation scenarios by hand in a rough JSON format. LLMs, including Opus 4.5/GPT5.2 Codex+ were not that good at understanding the rules of the game and generating coherent evaluation scenarios.

To get around the manual creation of many scenarios, I had Codex create a UI for specifying evaluation scenarios which made it a lot easier to introduce new evaluations.

Reward shaping scripts

After creating lots of evaluation scenarios, and seeing a lot of different model generations struggle to pass some of them, I decided to create a script that compares the rewards/penalties that are produced for each move in our evaluation scenarios. Whenever a move in an evaluation is considered positive, but the rewards provided during training are negative, that's flagged so that we can ensure that the model learns the right behaviours.

What's next?

The next goal is to seriously exceed human-level performance. I'll keep improving the model and use it in online games until I'm convinced it's really hard to beat consistently.

I'll also do some deep dives on some of the topics covered here and in the video. The goal of this blog was to provide an intro. I've been working on this project on and off for a while and though it's best to put it out there in whichever state it's in today. My hope is to keep improving it and share actually useful stuff and deep dives in the future :)