I’m working on an experiment comparing the internal representations of two architectures when solving a sequential algorithm, but training models to use a sequential algorithm is surprisingly hard. The optimization landscape makes it easier for models to learn parallel algorithms or memorize lookup tables, so I needed to make some specific architectural and training decisions to get models to actually learn the sequential algorithm. Even with all of these tricks, the results are seed-dependent and I needed to inspect the resulting models to prove that they did or didn’t learn the expected algorithm.

In this post, I’ll document what did and didn’t work, and the techniques I used to prove whether or not the model learned a sequential algorithm.

Two heatmaps titled "Logit lens: does the model compute one composition step per layer?", with layers L0 to L7 down the side and intermediate states t[1] to t[6] across the bottom. Model A, trained answer-only then full-sequence, has a dark diagonal staircase: 55% at L1/t[1], 56% at L2/t[2], 72% at L3/t[3], 64% at L4/t[4], 48% at L5/t[5], and 100% at L6/t[6]. Model B, trained with full-sequence loss only, sits at chance (~17%) everywhere except the t[6] column, which climbs from 32% at L3 to 100% at L7.

Read more

I’ve been working on some fairly large vibe-coded apps (like Lion Reader), and my process has converged on:

  1. Write a GitHub issue
  2. (If complicated enough) tell an agent to make a plan and then update the issue
  3. Have another agent read the issue and implement it

As the features get more complicated, I spend more and more time on step (1), and I’m finding that just taking the time to write a detailed enough issue is 90% of the work (and if I have a problem, going back and writing a much more detailed issue usually fixes it). The thing I realized this morning is that writing these issues and working through the plans is very similar to participating in a system design interview: You don’t need to implement anything, but you do need to have a good high-level design, and think through all of the edge cases and tradeoffs.

A sketchnote equating two scenes with a large equals sign: on the left a nervous person presents an architecture diagram at a whiteboard to an interviewer, and on the right the same person relaxes at a laptop while the identical diagram floats out to a friendly robot.

Read more