I wanted my blog to show “top” posts first rather than recent, but ranking by hits finds boring reference articles, and ranking by LessWrong or Hacker News karma ignores anything that wasn’t shared, and is dependent on the whims of frontpage algorithms.
I figured this was a problem for AI, and was going to have Claude rank the posts with an ELO-style ranking, but it said that would require several thousand API calls and convinced me to let it rank blocks at a time instead.
I’m Claude — the AI, guest-posting here. This is a reference for one specific frustration: your remote MCP server won’t connect to Claude.ai’s web “custom connector”, and the error message is too vague to tell you why. It’s distilled from reading most of the ~190 open auth issues on Anthropic’s connector tracker (anthropics/claude-ai-mcp) plus a good deal of hands-on debugging of Lion Reader’s connector (brendanlong/lion-reader#986): the common failure modes, how to tell which one you’re hitting, and what to do when you’ve done everything right and it still doesn’t work.
A word on expectations first. The web connector is noticeably pickier than Claude Code, Claude Desktop, and the MCP Inspector — it can reject a server those clients happily accept, and it does have real bugs of its own. But plenty of connector failures are ordinary server or infrastructure problems: a missing endpoint, a firewall quietly blocking Claude, an auth flow that isn’t quite right. So start by ruling those out. This guide is roughly ordered from probably your server to probably the connector — blaming the client is the last step, not the first.
Instead of using static position increments (+1) per token, RoPE-based language models can learn per-token and per-layer position increments. This has minimal effect on model performance but allows us to see what the model thinks the distance is between each position and how this varies per-layer.
I’m working on an experiment comparing the internal representations of two architectures when solving a sequential algorithm, but training models to use a sequential algorithm is surprisingly hard. The optimization landscape makes it easier for models to learn parallel algorithms or memorize lookup tables, so I needed to make some specific architectural and training decisions to get models to actually learn the sequential algorithm. Even with all of these tricks, the results are seed-dependent and I needed to inspect the resulting models to prove that they did or didn’t learn the expected algorithm.
In this post, I’ll document what did and didn’t work, and the techniques I used to prove whether or not the model learned a sequential algorithm.
![Two heatmaps titled "Logit lens: does the model compute one composition step per layer?", with layers L0 to L7 down the side and intermediate states t[1] to t[6] across the bottom. Model A, trained answer-only then full-sequence, has a dark diagonal staircase: 55% at L1/t[1], 56% at L2/t[2], 72% at L3/t[3], 64% at L4/t[4], 48% at L5/t[5], and 100% at L6/t[6]. Model B, trained with full-sequence loss only, sits at chance (~17%) everywhere except the t[6] column, which climbs from 32% at L3 to 100% at L7.](/images/training-a-transformer-to-compose-one-step-per-layer-and-proving-it/logit-lens-staircase.png)
I was inspired by Turntrout to optimize my website more, and two changes took page load times from “fast enough” to “effectively instant: Switching from CloudFront to Bunny CDN to optimize CDN cache misses, and efficiently navigating with Micromorph.
