One of my favorite AI papers is “Lets Think Dot By Dot”, which finds that LLMs can use meaningless filler tokens (like “”.) to improve their performance, but I was overestimating the implications until recently and I think other people might be too.
The paper finds that LLMs can be trained to use filler tokens to increase their ability to do parallel reasoning tasks. This has been compared to chain of thought, but CoT allows models to increase sequential reasoning, which is more powerful. I now think this paper should be taken as evidence against LLMs ability to perform long-term reasoning in secret.
There’s an LLM training optimization which keeps surprising me. It makes loss look unrealistically good during training, especially when training reasoning models with SGD. I’m also starting to think it’s the reason LLMs have trouble fixing their mistakes and take their own outputs too canonically.
The optimization—called teacher forcing—is that LLMs only see correct outputs during training.
Current LLMs almost always process groups of characters, called tokens, instead of processing individual characters. They do this for performance reasons: Grouping 4 characters (on average) into a token reduces your effective context length by 4×.
