One of my favorite AI papers is “Lets Think Dot By Dot”, which finds that LLMs can use meaningless filler tokens (like “”.) to improve their performance, but I was overestimating the implications until recently and I think other people might be too.
The paper finds that LLMs can be trained to use filler tokens to increase their ability to do parallel reasoning tasks. This has been compared to chain of thought, but CoT allows models to increase sequential reasoning, which is more powerful. I now think this paper should be taken as evidence against LLMs ability to perform long-term reasoning in secret.
CloudFlare recently had an incident where some code expected that a list would never contain more than 20 items, and then it was presented with a list of more than 20 items. Internet commenters rushed to point out that the problem was that the code was written in Rust, or that the source code had the word unwrap in it. A surprising number of people argued that they should have just “handled” this error.
I think this is wrong, and it completely misses how software is made robust.
I was thinking about an approval-style voting system that could end with a large number of ties, and ran into the problem of how to break ties in a provably-fair way that doesn’t depend on candidates trusting each other and won’t make voters’ eyes glaze over.
I think I found a solution which is provably-fair, but it might still cross over into eye-glazing territory. I don’t know if this is practical (or novel), but I’m writing it up in case anyone else finds it interesting.
I recently moved near Seattle, where traffic is terrible but there are surprisingly many bike lanes. Unfortunately, there are also a lot of hills. For a while, I would occasionally bike to work, but the hills were intimidating and I had to re-motivate myself every morning. Around a year ago, I finally gave up and just got an e-bike.
I’ve been working remotely since before it was cool, and one thing I wish more people paid attention to is meeting equipment. It’s annoyingly common to join a remote meeting with someone on flaky WiFi, with a barely-understandable microphone, and a camera where they show up as a shadowy blob.
All of this is fixable, and if you work remotely it’s worth spending a little bit of money to do it. Remote meetings where you can see and (more importantly) hear each other clearly are much nicer, and lead to more natural and collaborative conversations.
