LLMs learn faster if we first pretrain them to imitate dense teacher-forced examples. I speculated that this would work on humans too, so I built a chess app where you try to imitate Stockfish. My theory is that this will help humans quickly become OK at chess, but they will reach a wall where practice on full games is more efficient than continued pretraining. I also think the app is fun.

This is probably not an efficient way to learn the basic rules of chess, and you’ll need to train openings separately. The idea of chess puzzle apps is hardly unique, but I don’t think anything else works in exactly the same way.

The app's preview card: a chess position with two candidate moves drawn as arrows numbered 1 and 2, beside matching numbered buttons labeled Bxe5 and Qb8 under the question "Which move is better?" Tagline: "Real positions, two candidate moves, instant feedback. Difficulty adapts to hold you near 80% accuracy."

Read more

I’ve been working on some fairly large vibe-coded apps (like Lion Reader), and my process has converged on:

  1. Write a GitHub issue
  2. (If complicated enough) tell an agent to make a plan and then update the issue
  3. Have another agent read the issue and implement it

As the features get more complicated, I spend more and more time on step (1), and I’m finding that just taking the time to write a detailed enough issue is 90% of the work (and if I have a problem, going back and writing a much more detailed issue usually fixes it). The thing I realized this morning is that writing these issues and working through the plans is very similar to participating in a system design interview: You don’t need to implement anything, but you do need to have a good high-level design, and think through all of the edge cases and tradeoffs.

A sketchnote equating two scenes with a large equals sign: on the left a nervous person presents an architecture diagram at a whiteboard to an interviewer, and on the right the same person relaxes at a laptop while the identical diagram floats out to a friendly robot.

Read more

The United States has a strange (legal) tax loophole where you can double-count capital gains when donating securities (with some restrictions): If you buy a stock, the value goes up, and you’ve held it for at least a year, you can donate it and claim a tax deduction on the current market value instead of the value of what you paid for it (the cost basis), and you don’t pay taxes on the gains.

Sketchnote: a stock bought for $15k rises to $30k in value, then splits into two coins — one dropped into a charity donation box labeled "deduct $30k", the other landing on a crossed-out IRS form labeled "skip the gains tax".

Read more

The combination of running AI agents which can install and run anything with supply chain attacks / the Hackening has made me much more paranoid about my personal machine than I used to be. I’ve been slowly improving this, and finally moved all of my AI agents into a VM today. I’m sharing this to show how far you can take this, even if you have requirements like GPU access or the ability to run (something close enough to) Docker.

This is a follow-up / longer version of a recent shortform.

Diagram of a workstation (host). Outside it are two boxes: "My devices (Tailscale)" and "Internet".      Inside the host, an agent VM (KVM, 20 vCPU, 64 GB), whose QEMU process runs unprivileged and AppArmor-confined. It contains: agents and their web UI, as a user without root; rootless Podman for their containers; their credentials; a separate admin account agents can't reach; its own Tailscale node serving the web UIs; and a phone passed through over USB. Its firewall allows the internet and one port to the GPU box.      Next to it, a GPU box: its own no-login account running rootless Podman, containing a gVisor sandbox with SSH reachable only from this machine, a GPU job queue and jobs, a storage key scoped to one bucket, and nvproxy to the GPU driver. Its firewall allows the internet only.      Along the bottom: my account (desktop and browser, not in the docker or libvirt groups, sudo needs a password); text-to-speech (Docker, GPU worker in gVisor, no network); and the GPU (8 GB, shared with the desktop).      Arrows: my devices to the VM, labeled "web UIs over Tailscale"; the VM and the GPU box each to the internet; the VM to the GPU box through an SSH proxy; the VM to text-to-speech; the GPU box and text-to-speech to the GPU.      Footer: Secure Boot, Livepatch, unused kernel modules blocked, firewall inbound only over Tailscale.

Read more

I’m working on an experiment comparing the internal representations of two architectures when solving a sequential algorithm, but training models to use a sequential algorithm is surprisingly hard. The optimization landscape makes it easier for models to learn parallel algorithms or memorize lookup tables, so I needed to make some specific architectural and training decisions to get models to actually learn the sequential algorithm. Even with all of these tricks, the results are seed-dependent and I needed to inspect the resulting models to prove that they did or didn’t learn the expected algorithm.

In this post, I’ll document what did and didn’t work, and the techniques I used to prove whether or not the model learned a sequential algorithm.

Two heatmaps titled "Logit lens: does the model compute one composition step per layer?", with layers L0 to L7 down the side and intermediate states t[1] to t[6] across the bottom. Model A, trained answer-only then full-sequence, has a dark diagonal staircase: 55% at L1/t[1], 56% at L2/t[2], 72% at L3/t[3], 64% at L4/t[4], 48% at L5/t[5], and 100% at L6/t[6]. Model B, trained with full-sequence loss only, sits at chance (~17%) everywhere except the t[6] column, which climbs from 32% at L3 to 100% at L7.

Read more