I was inspired by Turntrout to optimize my website more, and two changes took page load times from “fast enough” to “effectively instant: Switching from CloudFront to Bunny CDN to optimize CDN cache misses, and efficiently navigating with Micromorph.
Clawed Abode: Claude Code is Too Cloudy
Running Claude Code locally is annoying since you have to deal with permissions and agents interfering with each other (and you have to be at your computer), but running Claude Code on the web is annoying because the cloud environment is so limited.
What if we could run Claude Code for the web but on our machines? Through the magic of Claude Code writing Claude Code code, I made a local app for this.
Announcing Clawed Abode: A web app you can run on your own home computer which runs Claude Code without permission prompts in ephemeral containers, and with the ability to install packages, run containers, use caches, and access the GPU.
%CPU Utilization Is A Lie
I deal with a lot of servers at work, and one thing everyone wants to know about their servers is how close they are to being at max utilization. It should be easy, right? Just pull up top or another system monitor tool, look at network, memory and CPU utilization, and whichever one is the highest tells you how close you are to the limits.
Make Data Pipelines Debuggable by Storing All Source References
A few jobs ago, I worked at company that collected data from disparate sources, then processed and deduplicated it into spreadsheets for ingestion by the data science and customer support teams. Some common questions the engineering team got were:
- Why is the data in some input CSV missing in the output?
- Why is data in the output CSV not matching what we expect?
To debug these problems, the process was to try to reverse engineer where the data came from, then try to guess which path that data took through the monolithic data processor.
This is the story of how we stopped doing that, and started storing references to all source data for every piece of output data.
Pixz for indexed and compressed archives
7 years ago (!) I wrote a post comparing ZIP and tar, plus gz or xz, and concluded that ZIP is ideal if you need to quickly access individual files in the compressed archive, and tar + compression (like tar + xz) is ideal if you need maximum compression. Since then, I discovered pixz, which seems to provide the best of both worlds: Maximum compression with indexing for quick seeking.
Write code to interact with 3rd parties using their data model
When writing code to interact with a third party like a SaaS provider, you typically need to write both code to handle the API the third party uses, and translate their data model into your data model. One thing I’ve found that makes this easier is to do each step separately: First write a library to interact with their API using their own data model, then separately write code to translate between your data model and theirs.
“Additional” space complexity is a bad metric
I’ve been subscribed to Interview Cake for years, and today they had a really interesting question: Given a list of n + 1 integers in the range 1...n, find one of the duplicates (there is guaranteed to be at least one) in O(n) time and O(1) additional space. The answer is really interesting, and I recommend trying it, but I don’t think it makes sense to care about additional space rather than total space, and I still think using a set to keep track of numbers (the most obvious solution) is the best solution in practice.
It’s worth it to use more memory
There’s a common programming interview question that asks you to find the single non-duplicated value in a list of duplicated integers. This can be done in O(n) time and O(1) space by XOR’ing the values, and doing so is almost always the wrong answer. A better solution when you can afford to do it is to use a hash table to count occurrences.
Indexing and sorting to find data quickly
To make your data faster to lookup, you can either store it in an order that makes it easier to search, or add one or more indexes. For practical work, you can let your file system do this for you, or use a pre-built database (either relational or not). I’ll describe from the lowest-level to highest level so you can understand what I’m suggesting, but my real-world answer is that I would store most kinds of data in a relational database like PostgreSQL and put indexes on any column that I want to do lookups by.
How to write examples for documentation
When you’re documenting a project so other people can use it, whether it’s a library or web service, one important thing to do is to give people good examples to work with. Not only does this save people time trying to cobble together their first working program, but it’s also a good way to show how the library is meant to be used, instead of just what’s technically possible.
Easy mistakes when writing OCaml C bindings
I recently spent several days improving the OCaml FreeTDS C bindings for work, and I thought it might be useful to share the problems I ran into and how to solve them.
I tried to order things so the most likely issues are listed first, but if you’re trying to debug some C binding crashes, I recommend just reading the whole thing.
How to use Core.Command.Param
Core.Command (and the closely-related Async.Command) is an OCaml library for creating command line programs with nice interfaces (including help text and argument parsing). This article is an overview of Command.Param, the newer interface for defining your command’s arguments (replacing Command.Spec).
FreeTDS 0.91’s TDS 7.1 support is buggy
This is a short post to document an issue we ran into at work so it will show up in search results for people who have this problem in the future.
tl;dr:
- Use FreeTDS 1.0 if you can.
- If you’re stuck on FreeTDS 0.91 (and possibly other pre-1.0 versions), don’t use any TDS version above 7.0.
The minimum viable XLSX reader
I recently finished a basic XLSX reader for OCaml and I thought it would be a good time to summarize what’s necessary to make a library like this, since the documentation is complicated and sometimes hard to find.
What is a monad?
From a non-theoretical engineer perspective, a monad is a kind of container that you can apply functions to to get a new monad. In particular, you need a way to put something into your monad and a way to apply changes to it. These are called return and bind. There is also a third operation, map, which is a combination of the two. Note that monads are allowed to be empty, which will make bind and map not do anything; and they can also contain more than one thing (in which case bind and map will call the function you give them more than once and merge the results).
ZIP vs tar for compressed archives
The main difference between the two formats is that in ZIP, compression is built-in and happens independently for every file in the archive, but for tar, compression is an extra step that compresses the entire archive. This means ZIP has better random access while tar+compression will generally provide better compression.
OCaml operator cheatsheet
One of the hardest parts of learning OCaml is figuring out what the infix operators do, since they’re just a string of symbols and you can’t find them with a Google search. This is my attempt to make a cheatsheet for whenever you’re wondering what a random series of symbols means.
Replacing computer setup with Ansible
I’ve been keeping a computer setup page on my website for a while, describing the exact steps needed to make a computer to work how I want after a new OS install. I’ve also had a note there for years saying eventually I should use Ansible for this. This weekend I finally got around to it, and learned a little bit about Ansible while I was at it.
Setting a minimum viewport width for responsive pages
My hobby recently has been making website scale better on phones, but I ran into a problem on several of them: The viewport meta tag assumes that a website either has a fixed width or that it scales to any possible width. I investigated several solutions and eventually settled on an (apparently) novel one: Check the screen size, and if it’s too small, add a new meta tag with a fixed viewport. I also wrote a JavaScript polyfill that lets me just set a min-width in the viewport tag.
Monitoring servers with Monit
This machine runs a lot of services and I don’t use all of them. After breaking several of them and not noticing (again), I decided to finally set up service monitoring. After some research, Monit was relatively easy to set up and seems to meet my needs. I figured other people might want some examples of how to use it, so this post describes how to set it up and you can see my config file at the end.
Getting patches into open source projects
There’s an open source project you want to use, but it’s missing a feature you need, or has a bug you need fixed. You can implement it yourself, but you want to make sure the patch will be accepted by the project’s maintainers. This guide will explain how best to minimize wasted effort and improve the chances of your code being accepted.
systemd user services are amazing
In the last few days, I’ve needed to set up several long-running services and I just wanted to take a minute to talk about how helpful systemd’s user services have been.
The things I wanted to run are:
- A Node.js server which is started with
npm run - A Node …
Website metadata and improved reading mode support
It’s that time of year again: Time to make needless improvements to my website. This time, I was annoyed about how my pages were showing up in Google, so I added Microdata to my articles. One thing lead to another, and I ended up with Microdata, Open Graph, a better sitemap, fewer useless pages, and much improved reading mode support.
Building Alljoyn packages with fpm
I’ve been using Alljoyn recently, but only the sources are distributed, and it’s fairly difficult to compile programs against it. The recommended way to handle this seems to be to manually copy various files into your root filesystem, but I prefer not to do that for obvious reasons. Instead, I’m going to build Alljoyn packages, using fpm.
Common informative metadata in MPEG-DASH
This is part two in a series on MPEG-DASH. The previous post described the structure of an MPEG-DASH MPD. This post describes informative metadata, like labels, languages, and copyright information. I plan to write one more post in this series, about less visible data like asset identifiers.
CircleCI makes automated builds easy (but not as easy as they could be)
Earlier today, I stumbled upon CircleCI and I was curious about how it compared to continuous integration products I’ve used in the past (Jenkins and Bamboo). So far I’m impressed.
Testing a Node.js project
After connecting CircleCI to my GitHub account, the only repo I owned …
Running ‘make check’ tests with Valgrind
While writing unit tests for my C code, I decided it would be nice to run the tests in Valgrind, so I could check for memory leaks too. It turned out to be easier than I expected, but still a lot harder than it should have been.
Unit Testing C with Check and Autotools
I recently finished (in terms of features) a major code project, but I’m not happy with the quality of the code yet. Besides the usual ways code can be buggy, it’s especially easy to crash a program written in C (which can be exploited if the program is available online, as this one probably should be).
RFC 7240’s “Prefer: wait” instead of “Timeout” header
I emailed the IETF HTTP group about my timeout header idea, and a few people said that RFC 7240’s “wait” preference does what I want. I’m not entirely convinced that this is what was intended by that spec, but it’s close enough. In response, I’ve updated my Express middleware to support the “Prefer” header and “wait” preference.
HTTP “Timeout” header for requesting resources from the future
Background
I’ve been participating in MPEG’s DASH group, and currently a lot of work has been focused on reducing live streaming latency. The latency problem in DASH is that clients have to poll servers to check for new media segments. If they poll too slowly, it introduces latency, but if they poll too quickly, it increases server load. When MPD’s are dynamic, a client needs to poll the server until it finds a new MPD, then request newly available segments, then only after the segment has downloaded enough to start, it can continue playback.
The structure of an MPEG-DASH MPD
The MPEG-DASH Media Presentation Description (MPD) is an XML document containing information about media segments, their relationships and information necessary to choose between them, and other metadata that may be needed by clients.
In this post, I describe the most important pieces of the MPD, starting from the top level (Periods) and going to the bottom (Segments). In a later post, I cover common informative metadata. Other topics that I might cover include MPD events, in-band events (‘emsg’), and encryption (DRM).
Videos as images in HTML
Most web browsers only support a few ancient image formats (mainly PNG, JPEG and GIF), but video formats have improved significantly since those formats were defined. Google is attempting to fix this with their WebP image format, based on VP8. Unfortunately, this only works with Google Chrome and Opera. Since what we want is to encode an image using the advancements in VP8 or h.264, I thought it would be interesting to try encoding single-frame videos and using them as images.
Prince of the Dark Kingdom EPUB and MOBI
I just finished book 6 of Mizuni-san’s Prince of the Dark Kingdom (Harry Potter fanfiction). I’ve been using an auto-generated EPUB, but the quality was pretty bad, since it didn’t understand the formatting. To fix that, I decided to make my own EPUB using some Python and some Pandoc.
Linker problems building Chromium on Fedora 20
I need to build Chromium on a machine that runs Fedora 20 today, and I had some trouble due to this error:
[12269/24399] LINK gles2_conform_support
FAILED: c++ -Wl,-z,now -Wl,-z,relro -Wl,--fatal-warnings -pthread -Wl,-z,noexecstack -fPIC -B/home/blong/workspace/chromium/src/third_party/binutils/Linux_x64 …Fluid CSS design and Feedbin’s mobile interface
On August 26th, I posted several feature requests for Feedbin. On August 27th, Feedbin became open source. Since the timing was just too perfect to ignore, I decided to fix some things myself. The first big improvement has been fixing Feedbin’s mobile interface, which previously looked like this …
Distributed Secure Email Protocol
I’ve been reading about BitMessage, an anonymous, encrypted peer-to-peer email protocol. Unfortunately, there are some major problems:
- BitMessage addresses are 36-character hashes, which isn’t very user friendly.
- Since the address is the hash of your public key, there’s no way to change your keys without creating a new address …
Debugging GStreamer crashes
I’ve been working on GStreamer a lot recently, and I’m slowly becoming comfortable with debugging problems with gdb. I assume this could be useful for other people, so this is part of a series on debugging GStreamer.
Using gdb to debug crashes
I recently had a problem with …
Python Encryption Example
Better responsive design
My last update made this site render better on small screens, but didn’t look right on Android. It looks like the problem is that mobile browsers do weird things on the assumption that website developers are idiots (generally a valid assumption). MDN has an article about how to fix it …
Responsive design
I decided a couple of days ago that this site really needed a responsive design, since it’s the cool thing to do and stuff.
diff --git a/themes/brendanlong/static/css/style.css b/themes/brendanlong/static
index 96a16b4..eb493b9 100644
--- a/themes/brendanlong/static/css/style.css
+++ b/themes …Automated testing
There’s a popular question on the Programmers Stack Exchange site asking, “Are unit tests really that useful?” and the second answer seems to be responding to a straw-man version of automated testing. The comments show that a lot of people seem to think it’s a reasonable description, so …
Timing attacks and usernames
Today, I found an interesting article about password timing attacks. The basic point is that if you check a password one character at a time, the amount of time it takes to receive a “bad password” response tells you how many characters you got right.
Advanced Makefiling
This weekend, my roommate asked me to make a website for him. I wrote it in PHP so I could just make some templates and he could import them, but it bothered me that I was rendering a static website with PHP. The easiest solution seemed to be running every …
Why I love NumPy
Fixing a clock with NTP
My roommate’s computer’s clock has been broken since we first installed Linux on it. It was never a big deal, but it’s been annoying me for a long time.
Recently, my laptop started to have a similar problem. A friend had used Windows on it, and so the clock was …
How to find palindromes in Python
In school this week, the idea of finding palindromes in Python came up, as a homework assignment in a class I took last semester. A friend asked me for some guidance in how to do this in Python, so I showed him my “simple” solution:
#!/usr/bin/env python3
from …