Claude has trouble playing Pokemon partially because it can’t see the screen very well. This made me wonder if Claude would be better at an ASCII game like Dwarf Fortress, where it doesn’t need to rely on image recognition.
To check this, I built an MCP server to let Claude control an interactive terminal, and installed a text version of Dwarf Fortress.
One of my favorite AI papers is “Lets Think Dot By Dot”, which finds that LLMs can use meaningless filler tokens (like “”.) to improve their performance, but I was overestimating the implications until recently and I think other people might be too.
The paper finds that LLMs can be trained to use filler tokens to increase their ability to do parallel reasoning tasks. This has been compared to chain of thought, but CoT allows models to increase sequential reasoning, which is more powerful. I now think this paper should be taken as evidence against LLMs ability to perform long-term reasoning in secret.
Update: I suspect the results I see here were mostly caused by models accidentally trained on CoT. See this shortform for details.
In a recent post, Zvi described what he calls “The Most Forbidden Technique”:
An AI produces a final output [X] via some method [M]. You can analyze [M] using technique [T], to learn what the AI is up to. You could train on that. Never do that.
You train on [X]. Only [X]. Never [M], never [T].
Why? Because [T] is how you figure out when the model is misbehaving.
If you train on [T], you are training the AI to obfuscate its thinking, and defeat [T]. You will rapidly lose your ability to know what is going on, in exactly the ways you most need to know what is going on.
The article specifically discusses this in relation to reasoning models and Chain of Thought (CoT): if we train a model not to admit to lying in its CoT, it might still lie in the CoT and just not tell us.
CloudFlare recently had an incident where some code expected that a list would never contain more than 20 items, and then it was presented with a list of more than 20 items. Internet commenters rushed to point out that the problem was that the code was written in Rust, or that the source code had the word unwrap in it. A surprising number of people argued that they should have just “handled” this error.
I think this is wrong, and it completely misses how software is made robust.
Exercise is hard but it’s even harder if you have to use your brain and muscles at the same time. I wish a personal trainer would just teleport into my house whenever I work out, tell me exactly what to do, and then record my progress (and complaints) to improve the program going forward. Apps are too rigid or too complicated; personal trainers are expensive and require scheduling; but using Claude Code as a personal trainer has worked out well for me.
