Thomas Burgess

Resume

2026-05-29T02:02:33Z

Thomas Burgess

Email: me@thomasburgess.dev
https://github.com/ThomasBurgess2000
https://www.linkedin.com/in/jthomasburgess

Lead Software Engineer

MeetKai, Jan 2025 - Present

Single-handedly made the iOS and Android companion apps, the backend voice services, the marketing website, and customized the RTOS firmware for an upcoming AR glasses platform.
Architecting a scalable API for our AI platform, with a focus on sandbox infrastructure for agents.
Creating web and mobile apps that build on our AI platform's SDK.

Technical Lead

MeetKai, Dec 2020 - Jan 2025

Lead development of bitnile.com, architecting full-stack solutions using Next.js and Babylon.js.
Manage a team of 8 engineers, driving technical decisions and mentoring junior developers.
Architected and implemented browser-based metaverse platform, for companies such as BYD, Sony, DRL and the Charlotte Hornets.

Data Team Lead

MeetKai, Jul 2021 - Mar 2022

Led team of 3 engineers in building data acquisition pipelines.
Wrote systems in Go and TypeScript to scrape, parse, and export data from various sites at scale.

Software Engineer

MeetKai, Dec 2020 - Jun 2021

Worked on the backend team.
Wrote code in Go and Python for NLP, data aggregation, and automation features.

Software Engineer Intern

Lockheed Martin Aeronautics, May 2020 - Dec 2020

Worked on the F-35 weapons team, completing coding, testing, and debugging tasks in C++.
Led effort to automate the build and test processes using Jenkins and Python.

Bachelor of Computer Science

Biola University

3.9 GPA, Stewart Science Honors Program, Torrey Honors College

Experienced with TypeScript, Next.js, Go, Temporal, React, Python, Babylon.js, Prisma, C++

How to Use an Ultrasonic Sensor with RPi and C++

2026-05-29T02:02:33Z

Follow this wiring diagram for the HC-SR04 ultrasonic sensor:

I used 1KΩ and 2KΩ resistors instead of 330Ω and 470Ω (because it's what I had on hand). Here's what that looked like:

Test the wiring with this simple Python script:

from gpiozero import DistanceSensor
from time import sleep

ultrasonic = DistanceSensor(echo=17, trigger=4, max_distance=4)

while True:
    print(f"{ultrasonic.distance * 100:.1f} cm")
    sleep(1)

Then install WiringPi , the fastest RPi GPIO access library:

git clone https://github.com/WiringPi/WiringPi.git
cd WiringPi
./build debian
mv debian-template/wiringpi_3.10_arm64.deb .
sudo apt install ./wiringpi_3.10_arm64.deb

Create a file with this program:

#include <wiringPi.h>
#include <iostream>
#include <iomanip>
#include <unistd.h>
  
#define TRIGGER_PIN 4    // GPIO 4
#define ECHO_PIN    17   // GPIO 17
#define TIMEOUT     25000 // Maximum time to wait for echo (in microseconds)
  
double measureDistance() {
    // Send trigger pulse
    digitalWrite(TRIGGER_PIN, LOW);
    delayMicroseconds(2);
    digitalWrite(TRIGGER_PIN, HIGH);
    delayMicroseconds(10);
    digitalWrite(TRIGGER_PIN, LOW);
    
    // Wait for echo start
    while (digitalRead(ECHO_PIN) == LOW);
    long startTime = micros();
    
    // Wait for echo end
    while (digitalRead(ECHO_PIN) == HIGH && micros() - startTime < TIMEOUT);
    long endTime = micros();

    // Calculate distance
    double duration = (endTime - startTime);
    double distance = (duration * 0.0343) / 2; // Speed of sound = 343 m/s
    return distance;
}

  

int main() {
    // Initialize WiringPi
    if (wiringPiSetupGpio() == -1) {
        std::cerr << "Failed to initialize WiringPi" << std::endl;
        return 1;
    }

    // Setup pins
    pinMode(TRIGGER_PIN, OUTPUT);
    pinMode(ECHO_PIN, INPUT);

    while (true) {
        double distance = measureDistance();
        std::cout << std::fixed << std::setprecision(1)
                  << distance << " cm" << std::endl;
        sleep(1);
    }

    return 0;
}

Compile it (linking wiringPi):

g++ -o ultrasonic main.cpp -l wiringPi

And run:

./ultrasonic

If everything worked correctly, you should see something like this:

66.7 cm
66.2 cm
65.8 cm
4.8 cm
24.4 cm
23.6 cm
4.9 cm

All code available on GitHub.

12.3.2024

Tracking Camera with Head Animation

2026-05-29T02:02:33Z

I wanted to make the player's head track with the camera. So first, I had to disconnect the animations from controlling the neck and overriding whatever rotations I was setting. Then I had to make the neck's rotation correspond to the rotation of the camera.

Defining this behavior turned out to be quite difficult. Firstly, I could not merely match the rotation of the camera to the neck because they used different systems to determine rotation. The ArcRotateCamera we are using has a target (the player) and 3 values to determine its position: alpha, beta, and radius. The neck on the other hand just uses a quaternion.

I also wanted different behavior based on whether the camera was behind the player or in front of the player. If the camera was behind the player, I wanted the head to point to where the camera was looking, if the camera was in front of the player I wanted the head to look at the camera.

So first I needed a way to determine whether the camera was in front of the player or behind the player. After trial and error, I determined that this was essentially converting the rotation measurements of the camera and the player to something I could directly compare.

The way I described this was: imagine 2 CD's on a CD tower rotating independently of each other (the flattening here was important). Now draw a dot on the edge of the top CD and color in half of the bottom CD. Essentially what I was trying to do was determine when the dot on the top CD was over the colored-in portion of the bottom CD.

This is how I got the direction that the player was facing:

let playerDirection = new Vector3(0, 0, -1);

playerDirection = playerDirection.rotateByQuaternionAroundPointToRef(playerRotation, new Vector3(0, 0, 0), playerDirection);

const playerDirectionInRadians = Math.atan2(playerDirection.x, playerDirection.z);

let normalizedPlayerDirection = playerDirectionInRadians % (2 * Math.PI);

if (normalizedPlayerDirection < 0) {
	normalizedPlayerDirection += 2 * Math.PI;
}

The alpha value for the camera was basically where the camera was in the horizontal orbit around the player. Normalizing this value to make sure the angle was between 0 and 2*Pi, correcting negative values, and then inverting it to reflect it over the axis made it match the player direction's format.

const alpha = camera.alpha;

let normalizedAlpha = (alpha + Math.PI / 2) % (2 * Math.PI);

if (normalizedAlpha < 0) {
	normalizedAlpha += 2 * Math.PI;
}

normalizedAlpha = 2 * Math.PI - normalizedAlpha;

Then I just had to get the absolute value of the difference between normalizedAlpha and normalizedPlayerDirection (and if the value was larger than Pi, subtract from 2*Pi to get the complementary angle, ensuring it represented the smallest angle between the two) and see if the value was greater or less than Pi/2. This would determine whether the camera was in front of or behind the player.

Then based on this information, I would set the neck's X rotation value to the beta or -beta value of the camera (along with an offset and clamping the values so the player didn't look up or down too much).

let correction = -1.1;

if (!isInFrontOfPlayer) {
	beta = -beta;
	correction = 1.9;
}

if (beta < 0.7 && isInFrontOfPlayer) {
	beta = 0.7;
}

if (beta > -0.8 && !isInFrontOfPlayer) {
	beta = -0.8;
}

const finalRotation = new Vector3(beta + correction, 0, 0);

While I was originally setting the neck rotation directly as described above, people complained that the neck's movement was too fast when the camera moved from the front of the player to the back (or vice versa) since it would involve a big rotation change. So I ended up sacrificing some responsiveness of the head movement for smoothness, by using a 1s animation with a cubic easing function to move the neck whenever the camera moved.

const finalQuaternion = Quaternion.FromEulerVector(finalRotation);

const easingFunction = new CubicEase();
easingFunction.setEasingMode(EasingFunction.EASINGMODE_EASEINOUT);

Animation.CreateAndStartAnimation(
	'neckRotationAnimation',
	window.localPlayer.neck,
	'rotationQuaternion',
	60,
	60,
	window.localPlayer.neck.rotationQuaternion,
	finalQuaternion,
	Animation.ANIMATIONLOOPMODE_CONSTANT,
	easingFunction,
);

7.27.23

LLMs are 800x Cheaper for Translation than DeepL

2026-05-29T02:02:33Z

When looking at the cost of translation APIs, I was floored by the prices. Azure is $10 per million characters, Google is $20, and DeepL is $25.

To come up with a rough estimate for a real-time translation use case, I assumed 150 WPM speaking speed, with each word being translated 3 times (since the text gets retranslated multiple times as the context lengthens). This resulted in the following costs:

Azure: $1.62/hr
Google: $3.24/hr
DeepL: $4.05/hr

Assuming the same numbers, gemini-2.0-flash-lite would cost less than $0.01/hr. Cost varies based on prompt length, but I'm actually getting just under $0.005/hr.

That's over 800x cheaper than DeepL, or 0.1% of the cost.

Presumably the quality of the translations would be somewhat worse, but how much worse? And how long will that disadvantage last? I can stomach a certain amount of worse for 99% cheaper, and it seems easy to foresee that LLMs will surpass the quality of the legacy translation models in the near future.

Right now the accuracy depends a lot on the prompting. I need to run a lot more evals, but so far in my tests I'm seeing that the translations I'm getting are as good (most of the time identical) or better than Google's the vast majority of the time. I'm confident I can get to 90% of Google's accuracy with better prompting.

I can live with 90% accuracy with a 99.9% cost reduction.

For many, 90% doesn't cut it for their translation needs and they are willing to pay a premium for the best. But the high costs of legacy translation APIs will become increasingly indefensible as LLM-based solutions improve, and we'll see translation incorporated in ways that were previously cost-prohibitive.

Sending Images to Smart Glasses via Bluetooth

2026-05-29T02:02:33Z

I recently tackled the challenge of figuring out how to send images to a pair of smart glasses we're developing. The significant hurdle was the lack of direct access to the firmware and extremely limited documentation on available commands or their formats. All I initially had was an outdated APK that no longer worked and some partial source code that only supported basic functionality.

This project was entirely new territory for me—I'd never worked with Bluetooth protocols, real-time operating systems (RTOS), or even Android development extensively. Fortunately, the availability of powerful language models (LLMs) made the exploration feasible.

Initial Attempts

I began by sending commands based on the limited documentation, hoping to trigger an image display. These attempts failed completely, giving no response or indication from the glasses that anything had been received—akin to shouting into a void.

Reverse Engineering

Realizing I needed deeper insight into what code could possibly be running on the glasses, I decompiled an old Android APK to investigate the original implementation for sending commands. Surprisingly, I discovered the APK didn't contain a working example of sending images. Instead, I found unused code snippets that suggested the capability existed at one point, but it had not been completed in this version of the app.

I noticed there were commands for drawing primitives (lines, rectangles, text) with clear examples, yet nothing similar for images. I reconstructed packet headers by referencing working commands and attempted to modify them to accommodate image data, but initially, this yielded no results.

Leveraging UUIDs and Subscriptions

Next, I shifted my approach toward trying to get any sort of feedback from the glasses. I learned that when connecting via Bluetooth Low Energy (BLE), devices respond with a list of subscription UUIDs. Unfortunately, these UUIDs lacked descriptive documentation.

By systematically logging every UUID interaction, I gradually identified patterns correlating specific UUIDs to particular functionalities. Even though the returned data was raw bytes and challenging to interpret directly, logging helped me detect responses indicating malformed requests. This marked significant progress—I finally had confirmation that the glasses were at least receiving and processing my messages.

Image Format and Compression Issues

Initially, I assumed the issue was image formatting. Since the APK command for image drawing was unused, I had no guidance on the expected image format. Additionally, the image underwent compression via a native binary library written in C—completely opaque to my investigation efforts.

Unable to decipher the compression algorithm through online searches, I extracted and incorporated the native binary directly into my workflow, hoping to correctly mimic the required compression step. Despite my efforts, repeated "malformed request" errors continued.

Breakthrough with Header Adjustment

I revisited the header configurations and realized I had mistakenly used a "draw canvas" command instead of the correct "raw image" command. Correcting this mistake resulted in the glasses returning a success message—my first tangible indication that the protocol was partially correct.

However, despite receiving success confirmations, no image appeared on the glasses. Suspecting color issues, I experimented by switching the bitmap color from black to white. Immediately, a visible line appeared on the glasses' screen.

Line to Box

However, I expected a 16x16 box and not a line. I discovered that for RAW_IMAGE messages the glasses expect every pixel to be defined. So when it received the 256 bytes, it started placing them on the first row of pixels. So if I wanted just a box in the middle of the screen, I needed to pad the box with black pixels.

This means sending a lot more pixels, which meant I immediately required sending multiple packets to display a single image on the screen. This broke the CRC calculation, because I was only calculating the CRC once for the whole image data, and apparently this firmware requires the CRC to be calculated for each packet. But after that, it worked.

7.2.25

Building Smart Glasses as a Solo Engineer

2026-05-29T02:02:33Z

For the better part of the last year, I have been the principal engineer (and for most of the project, the only engineer) for an upcoming smart glasses product. I've had to customize the firmware, build the app for communicating with the glasses, and build the backend services. This is despite having no previous work experience in hardware, app development, or three of the six programming languages I would end up using (C++, Go, Python, Kotlin, TypeScript, Swift).

I want to give credit where credit is due right at the outset: I could not have done this without the massive assistance of LLMs. The impact a single moderately skilled engineer can make in areas totally outside their realm of expertise is beyond what I thought possible.

I was so surprised at the scope of what I have built so far that I decided to document it at a high level, primarily for my own later recollection:

Dev Log

We are sourcing glasses from China that run a simple RTOS (real-time operating system) and have dual waveguide displays, speakers, and microphones. The firmware is basic, primarily consisting of Bluetooth connection code, a GUI for navigating an app menu, and BLE commands to send content to the glasses. The compiled binary is about ~3mb. The apps that were in the firmware were basically templates. For example, there were pre-made UI components for chat bubbles, but anything that would appear in those chat bubbles had to be sent from the phone. The glasses themselves have no internet access.

Thankfully the firmware edits were relatively minor, given the firmware is extremely painful to build and update OTA whenever I make a change. It's written in C++ and requires proprietary build software ($200 a month) to compile. I fixed some bugs (such as the pairing screen not being dismissible after the connection dropped temporarily), updated a couple UI components (through the extremely unintuitive "GUIX Studio"), and Americanized some aspects (display 12hr time formatting, temperature in Fahrenheit, etc.). The ability to dump the entire firmware into Cursor/Codex/Gemini CLI (whatever happens to be leading at the time) is priceless. It's a massive codebase whose structure I do not know, but I am able to ask things like "where are all the places that deal with navigation" and dig in from there.

The app has been the real challenge. We decided to build it as an Expo app to be cross-platform across iOS and Android, but given the nature of the app, I have to dive into native code for each platform quite often. At the time of writing actually, I've really only done the Android side of things, with some cursory research on how I might approach it for iOS.

If you're unfamiliar with how Bluetooth works, there's not just one protocol called "Bluetooth." There are a ton of protocols that a device could possibly use to communicate over Bluetooth. Discovering the intricacies of how to achieve consistent pairing, connecting, and communicating with the glasses was...a process. I had no documentation on the firmware besides an incomplete list of BLE (Bluetooth Low Energy) commands the glasses supported, and the rest I had to figure out by reading the firmware itself. And of course, this all had to be done through native code and is completely different on iOS and Android.

For most of what I did, I was only concerned with BLE. This covered things like sending the text to display on the glasses or setting the brightness level.

But I also had to make the glasses work as regular Bluetooth headphones. It took me a while to figure this out, but I eventually discovered the firmware blocked the ability to pair the glasses as headphones through a phone's native Bluetooth settings until the user had paired the glasses through the vendor's app, an intentional decision on the manufacturer's part.

I knew that in order to work as headphones, I had to connect two other Bluetooth profiles: A2DP and HFP. However, they had made it so these profiles were not available until you had connected another profile called SPP through the app.

This is different behavior than the Meta Ray-Bans for example, which you can pair as a regular Bluetooth headset without having to first pair the glasses through the Meta AI app. I think the theory was that customers would be confused if they paired the glasses through their phone's Bluetooth settings and audio worked, but then the glasses still complained about not being paired (since they still lacked the BLE connection).

Anyway, even once I figured that out, managing pairing state was tricky because of what iOS and Android let you see about the status of Bluetooth devices connected to the user's phone. For example, Android lets you see which devices are "bound" to the device (have ever been paired) but doesn't let you see which are currently connected. iOS doesn't let you see anything.

I had to create a flow in the app that would let the user establish BLE, SPP, A2DP and HFP connections to the glasses as simply as possible. On Android, this looked like sending a pairing request to the glasses, long-press confirming on the glasses (this completed BLE pairing), then adding the SPP connection, and then prompting the user to accept a pairing notification from the phone's native Bluetooth settings (for A2DP and HFP). As far as I can tell, this is as simple as I can make it, mirroring the behavior of the Meta Ray-Bans closely.

And that was just the initial connection. Knowing the state of the glasses at all times (are they connected over BLE but not A2DP/HFP? are they disconnected from SPP? did they disconnect because the user unpaired them in their phone's Bluetooth settings, because they got too far from the phone, because they turned off Bluetooth, or...?) involved quite the state machine.

But once I gave the phone the ability to talk to the glasses, I had to figure out what to say and how to say it. I had to figure out how to format the packets, how to sign them with CRC, how to split large payloads across chunks, what custom form of run-length encoding they were using for images, etc. etc. This was probably what I actually spent the most time on, because I actually worked on prototyping the app for a long time before I had access to the firmware itself. See Sending Images to Smart Glasses via Bluetooth for a full description of how I tried reverse-engineering one of the commands.

Once I knew how to communicate with the glasses, I needed to actually make them useful. We decided on an initial list of features we thought were achievable in a couple months:

Translation
Chat
Listener
Live Captions
Notifications
Calls
Weather
Music
Navigation

I will probably just link to a separate post about how I built the backend services for Translation, Chat, Listener, and Live Captions. If there's not a link here yet, it's because I haven't written that post. The gist is it was originally written in Python, then rewritten in Go, and had some neat techniques to achieve translation across multiple languages simultaneously in real-time.

The more standard smartwatch-like features that could live offline were more straightforward. Calling and Music were actually implemented entirely within the firmware already. But there were still a couple interesting elements:

For example, on Android you can get permission to read the user's notifications from other apps, allowing you to forward to the glasses for display. So all you have to do on the app side of things is let the user select which notifications they want to see, and forward those. On iOS however, Apple does not allow you to read the notifications from other apps. They have their own internal API they use for this sort of functionality with the Apple Watch, but they do not allow third-party developers access to it. Instead, they have their own old, poorly-documented method of broadcasting these notifications over Bluetooth directly to the paired devices that require them, bypassing the app. This meant it was harder to give the user control over the types of notifications they wanted to receive on the glasses, because instead of just being a menu in the app, it had to be done via firmware on the glasses.

Another interesting feature was navigation. The obvious way to approach this would be to use a navigation API like Google Maps and have some sort of navigation integration within the app. However, Google Maps is not free and I figured users would prefer to not have to switch how they navigated simply to get it to display in the glasses. And I did not want to have to implement a whole nav app if if I didn't have to.

So on Android, I leveraged the ability to read notifications again, and simply extracted the directions data from Google Maps notifications and forwarded them to the glasses, where they would be parsed and displayed. On Apple, this was not possible as already stated, so I'm going to have to make an Apple Maps integration (thankfully the Apple MapsKit is free for use in iOS apps).

Thus far I have MVPs of Translation, Chat, Listener, Notifications, and Navigation in the Android app, with Calling and Music being handled by the firmware.

I have a teammate who might take implementing the iOS side of things, but they're busy on another project, so it's possible I'll be doing that as well.

I've really enjoyed getting to stretch my abilities and will be thrilled if the glasses actually end up in the hands of consumers by the end of the year.

9.2.25

(Multiplayer) Provable Fairness Explainer

2026-05-29T02:02:33Z

Goal: Create a long list of random numbers that we can prove:

We have not tampered with since it was created.
We did not generate a lot of random number lists and then cherry pick the list that was most skewed in our favor.

Methodology: We generate a long chain of hashes from an initial seed that we keep private.

function generateHash(input: string): string {
  return createHash("sha256").update(input).digest("hex");
}

function generateHashChain(serverSeed: string, length: number): string[] {
  let currentHash = serverSeed;
  const chain: string[] = [];

  for (let i = 0; i < length; i++) {
    currentHash = generateHash(currentHash);
    chain.push(currentHash);
  }
  return chain;
}

We publish the last hash of that chain. We then reverse that chain, and each game we shift the chain, take that hash, and salt it with a publicly disclosed "client" seed.

function reverseHashChain(chain: string[]): string[] {
  return chain.reverse();
}

function saltHash(hash: string, clientSeed: string) {
  return createHmac("sha256", hash).update(clientSeed).digest("hex");
}

We then use that salted hash to generate the random crash point for that game.

function generateCrashPoint(serverSeed: string, clientSeed: string): number {
  const hash = saltHash(serverSeed, clientSeed);

  if (shouldInstaCrash(hash, houseEdge)) {
    return 1;
  }

  const h = parseInt(hash.substring(0, 52 / 4), 16);
  const e = Math.pow(2, 52);

  return Math.floor((100 * e - h) / (e - h)) / 100;
}

Each game we tell the user a hash of what the salted hash used to generate the next value will be. After the game, we reveal the actual hash used.Users can then verify that the hash chain has not been tampered with by first verifying that the secondary hash they were told before the game actually corresponds to the hash that was revealed after the game. Then they can put the revealed hash in a validation function:

function validateFairness(hash: string, clientSeed: string, numGames: number = 10): void {
  const chain = generateHashChain(hash, numGames);
  const reversedChain = reverseHashChain(chain);

  console.log(`Validating fairness for the last ${numGames} games:`);

  reversedChain.forEach((gameHash, index) => {
    const crashPoint = generateCrashPoint(gameHash, clientSeed);
    console.log(`Game ${index + 1}: Hash = ${gameHash}, Crash Point = ${crashPoint.toFixed(2)}`);
  });
}

This function generates the results of the preceding games leading up to the current one, and the user can verify they indeed match the history. If they generate all the preceding games, the hash of the very first one should be the hash that we publicly disclosed at launch (the last hash of the reversed chain).

So all of the above just proves that we didn't tamper with the list once it was created. But to prove that we didn't cherry pick the list, the publicly disclosed client seed that we use to salt the chain with has to be something that we couldn't have picked.

Typically companies have picked this client seed by publicly committing (normally on Twitter) to use the hash of a future Bitcoin block as the client seed. So this way they can commit to a number that they can't have known in advance.

Most critical security vulnerability:
The creation and storage of the pre-salted hash chain. The hash chain that we generate with our initial private seed has to be kept entirely secret, because whoever has access to it can predict the outcome of future games. I'm not sure what the best way to handle this is. We need some secure way of retrieving the next element in the chain without being able to output the entire chain, even internally.

4.19.2024

Handy Speech-to-Text to Tmux

2026-05-29T02:02:33Z

I installed Handy, a local speech-to-text app, on Kubuntu. It uses ONNX models (parakeet-tdt) to transcribe speech locally. Getting it to work required fixing a couple of issues, and then I built a small watcher script to pipe transcriptions into a tmux session.

Fixing the Crash

Handy was crashing silently every time I pressed the shortcut to start recording. The log at ~/.local/share/com.pais.handy/logs/handy.log would just stop dead mid-recording with no error. No segfault in dmesg, no coredump, nothing.

The root cause was that pipewire-alsa wasn't installed. Handy uses CPAL (a Rust audio library) which goes through ALSA, but without the PipeWire-ALSA bridge, it couldn't route audio and would silently die. Running Handy from a terminal showed the errors:

ALSA lib pcm.c:2722:(snd_pcm_open_noupdate) Unknown PCM pipewire
ALSA lib pcm_dmix.c:1000:(snd_pcm_dmix_open) unable to open slave

Fix:

sudo apt install pipewire-alsa

After that, the ALSA errors went away and recording worked.

Push-to-Talk Doesn't Work on KDE Wayland

Push-to-talk mode (hold key to record, release to stop) doesn't work properly on KDE Wayland. The key release event fires immediately, so the recording stops before any speech is captured. The log shows TranscribeAction::start and TranscribeAction::stop in the same second.

This is because Handy falls back to a regular window on KDE Wayland (Skipping GTK layer shell init on KDE Wayland) and global shortcut key-release detection is unreliable under Wayland's security model.

Workaround: use toggle mode instead (push_to_talk: false). Press once to start recording, press again to stop.

Handy Settings

In Handy's settings I changed:

Push to Talk: disabled (toggle mode, because of the KDE Wayland issue)
Paste output: disabled (so it only saves to history, doesn't paste into the focused window)
Debug mode: enabled (for the extra logging while troubleshooting)

The settings file lives at ~/.local/share/com.pais.handy/settings_store.json.

Sending Transcriptions to a Tmux Session

I wanted to be able to dictate into a specific terminal without switching focus to it. The idea: keep working in whatever window I'm in, press Ctrl+Space, speak, and have the text appear in a tmux session called "handy".

Handy saves every transcription to a SQLite database at ~/.local/share/com.pais.handy/history.db. The schema is simple:

CREATE TABLE transcription_history (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    file_name TEXT NOT NULL,
    timestamp INTEGER NOT NULL,
    saved BOOLEAN NOT NULL DEFAULT 0,
    title TEXT NOT NULL,
    transcription_text TEXT NOT NULL,
    post_processed_text TEXT,
    post_process_prompt TEXT
);

I wrote a watcher script at ~/.local/bin/handy-to-tmux that polls this database for new entries and sends them to the tmux session via tmux send-keys. It debounces rapid writes (Handy sometimes writes partial results) so only the final transcription gets sent.

Usage:

handy-to-tmux start    # start watcher in background
handy-to-tmux stop     # stop it
handy-to-tmux status   # check if running
handy-to-tmux run      # run in foreground for debugging

It uses a pidfile at ~/.local/state/handy-to-tmux.pid to ensure only one instance runs at a time. Logs go to ~/.local/state/handy-to-tmux.log.

The tmux session just needs to exist first:

tmux new-session -d -s handy

Then whatever shell is running in that session receives the transcribed text as if it were typed in.

Targeting Different Tmux Sessions

I added the ability to route transcriptions to different tmux sessions using separate keyboard shortcuts. The target subcommand sets which session to send to and triggers Handy's recording in one step:

handy-to-tmux target code    # sets target to "code" and starts recording
handy-to-tmux target chat    # sets target to "chat" and starts recording

The target persists in ~/.local/state/handy-to-tmux.target until changed, so subsequent Ctrl+Space presses (without setting a new target) continue going to the same session. Defaults to "handy" if no target is set.

To set up per-session shortcuts, bind KDE global shortcuts to:

Ctrl+Alt+1 → handy-to-tmux target handy
Ctrl+Alt+2 → handy-to-tmux target code
etc.

Each shortcut starts recording and routes the result to its session. Press Ctrl+Space to stop recording.

Repo

Source is at https://github.com/ThomasBurgess2000/handy-to-tmux. Install with ./install.sh which copies the script to ~/.local/bin/.

2.16.2026

Deploying Next.js Apps to Local Debian Server

2026-05-29T02:02:33Z

To deploy my Next.js app to my local Debian server while still having my domain point to it, I had to first enable port forwarding on my router. I forwarded 80 to 80 and 443 to 443. Don't do what some guides say and do 80 to 3000, I can't remember why, but I think it had something to do with nginx.

I had to enable some ports through my ufw firewall, I don't really remember which. I think one was sudo ufw allow 'Nginx Full'. Then sudo ufw reload.

Speaking of, BEFORE STARTING THE NEXTJS app, go ahead and get nginx setup.

This is what my /etc/nginx/sites-available/nextjs-app looked like:

server {

    server_name mydomain.com www.mydomain.com;

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }
}

Then link it with sudo ln -s /etc/nginx/sites-available/nextjs-app /etc/nginx/sites-enabled/. Test with sudo nginx -t. Restart with sudo systemctl restart nginx.

I made sure my DNS records were set up to point to my IP:

Host: @, Type: A, Data: MY_IP_ADDRESS
Host: www, Type: www, Data: MY_IP_ADDRESS

Then I ran sudo certbot --nginx.

This would automatically update my nginx file to accept SSL, but after that I changed it to work with Next.js. This is what it looked like after certbot and the NextJS addition:

server {

    server_name mydomain.com www.mydomain.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot


}

Then when I had a built version of my app on the server, I ran pm2 start yarn --name "app_name" -- start from within the app directory. You can set it to always start on server boot with pm2 startup. Then save with pm2 save. Now I can do like pm2 start app_name, pm2 stop app_name, pm2 restart app_name, and pm2 reload app_name (last one is supposedly zero downtime).

After this, I was able to access my app at https://mydomain.com.

9.2.2024

I then wanted to deploy a second app to my server, using the subdomain example.anotherdomain.com.

In order to do this, I added a new A record in Cloudflare:

Host: example
Type: A
Data: MY_IP_ADDRESS

I cloned the repo I wanted to run to my server, and started it with:

pm2 start yarn --name "second_example_app" -- start
pm2 startup
pm2 save

Then I had to create a new Nginx server block:

sudo nano /etc/nginx/sites-available/example.anotherdomain.com

Where I put:

server {

    server_name example.anotherdomain.com;

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }

    location / {
        proxy_pass http://localhost:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/example.anotherdomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.anotherdomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = example.anotherdomain.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80;
    listen [::]:80;

    server_name example.anotherdomain.com;
    return 404; # managed by Certbot

Then I enabled the server block with:

sudo ln -s /etc/nginx/sites-available/example.anotherdomain.com /etc/nginx/sites-enabled

Tested with:

sudo nginx -t

Reloaded nginx:

sudo systemctl reload nginx

Got the SSL certificate:

sudo certbot --nginx -d example.anotherdomain.com

Restarted the app for good measure:

pm2 restart second_example_app

12.1.2024

I make no promises as to the security of either of these methods. If there are glaring vulnerabilities or easy ways to harden these, please shoot me an email at me@thomasburgess.dev.

Converting a Shader from Three.js to Babylon.js

2026-05-29T02:02:33Z

In Search of the Perfect Tree

There have been many, many attempts at creating a forest of performant trees in Babylon.js, to the point that there's a dedicated "Tree Generators" page in the official documentation.

However, I was not impressed by the looks of the existing solutions, and went in search of a higher quality tree.

I came across this blog post titled "Creating fluffy trees with Three.js" and it instantly stood out visually. It used a custom GLSL shader applied to an extremely basic low-poly model of a tree, and the result was a beautiful, stylized, and performant tree.

At this point I knew nothing about shaders, and assumed I could just copy over the shader code into BJS and it would work. This turned out to not quite be the case, so I'm going to walk through the steps I had to take to make the shader Babylon-compatible.

Using GLSL in Babylon.js

First I had to figure out how to use GLSL shaders in Babylon. The docs provide four ways of importing GLSL code into your project, and I found the easiest way to be to simply copy the code into a string and assign it to the BABYLON.Effect.ShadersStore, like this:

ShaderStore.ShadersStore['customVertexShader'] = `
    attribute vec3 position;
    attribute vec2 uv;
    attribute vec3 normal;

    #include<instancesDeclaration>
    uniform mat4 view;
    ...
    `;

Then to use that shader, you create a new ShaderMaterial, like so:

const shaderMaterial = new ShaderMaterial(
      'shader',
      scene,
      { vertex: 'custom', fragment: 'custom' },
      {
        attributes: ['position', 'uv', 'normal'],
        uniforms: ['view', 'projection', 'vLightPosition', 'u_color'],
        uniformBuffers: undefined,
        shaderLanguage: ShaderLanguage.GLSL,
      },
    );

I quickly realized I could not simply paste in the vertex.glsl file from the 3JS blog post though.

Converting the Vertex Shader for Babylon Use

Babylon has a Create Your Own Shader tool that helps identifying errors in your shader code, and I discovered that 3JS has different "uniforms" (variables passed from the application to the shader code) than those used by Babylon. Some of these are explicitly defined, others are built in.

Here's the entirety of the vertex shader I was trying to convert:

uniform float u_effectBlend;
uniform float u_remap;
uniform float u_normalize;

varying vec2 v_uvs;

float inverseLerp(float v, float minValue, float maxValue) {
  return (v - minValue) / (maxValue - minValue);
}

float remap(float v, float prevMin, float prevMax, float newMin, float newMax) {
  float t = inverseLerp(v, prevMin, prevMax);
  return mix(newMin, newMax, t);
}

void main() {
  v_uvs = uv;

  vec2 vertexOffset = vec2(
    remap(uv.x, 0.0, 1.0, -u_remap, 1.0),
    remap(uv.y, 0.0, 1.0, -u_remap, 1.0)
  );

  vertexOffset *= vec2(-1.0, 1.0);

  if (u_remap == 1.0) {
    vertexOffset = mix(vertexOffset, normalize(vertexOffset), u_normalize);
  }

  vec4 worldViewPosition = modelViewMatrix * vec4(position, 1.0);

  worldViewPosition += vec4(mix(vec3(0.0), vec3(vertexOffset, 1.0), u_effectBlend), 0.0);

  gl_Position = projectionMatrix * worldViewPosition;
}

All this shader does is translate the vertices relative to the camera so that the quads on the mesh each have a pseudo-billboard effect, giving the tree its fluffy look.

There are built-in uniforms that 3JS uses that Babylon doesn't have. Here the 3JS-specific ones were modelViewMatrix and projectionMatrix.

projectionMatrix can just be replaced by projection. In 3JS, modelViewMatrix is viewMatrix * modelMatrix, so in Babylon we would replace modelViewMatrix with view * world (reminder that matrix multiplication is not commutative).

I then declared view, world, and projection at the top with the other uniforms, and when I created the ShaderMaterial in Babylon, I passed those uniforms in. I also had to explicitly add position and uv as attributes (per-vertex data).

Once I replaced those uniforms, the next issue I ran into was varying vec2 v_uvs. varying variables are used to pass data from vertex shaders to fragment shaders, but the blog post I was following didn't actually have a fragment shader explicitly defined. It appears they were using a package that provided it, so we'll come back to that variable later.

But at the end of the conversion, the vertex shader looked like this:

ShaderStore.ShadersStore['customVertexShader'] = `
attribute vec3 position;
attribute vec2 uv;

uniform mat4 world;
uniform mat4 view;
uniform float u_effectBlend;
uniform float u_remap;
uniform float u_normalize;
uniform mat4 projection;

varying vec2 v_uvs;

float inverseLerp(float v, float minValue, float maxValue) {
  return (v - minValue) / (maxValue - minValue);
}

float remap(float v, float inMin, float inMax, float outMin, float outMax) {
  float t = inverseLerp(v, inMin, inMax);
  return mix(outMin, outMax, t);
}

void main() {
  v_uvs = uv;
  
  vec2 vertexOffset = vec2(
    remap(uv.x, 0.0, 1.0, -u_remap, 1.0),
    remap(uv.y, 0.0, 1.0, -u_remap, 1.0)
  );

  vertexOffset *= vec2(-1.0, 1.0);

  if (u_remap == 1.0) {
    vertexOffset = mix(vertexOffset, normalize(vertexOffset), u_normalize);
  }

  vec4 worldViewPosition = view * world * vec4(position, 1.0);

  worldViewPosition += vec4(mix(vec3(0.0), vec3(vertexOffset, 1.0), u_effectBlend), 0.0);

  gl_Position = projection * worldViewPosition;
}
`;

Then after creating the ShaderMaterial, I passed in the uniform starting values:

shaderMaterial.setFloat('u_remap', 1.0);
shaderMaterial.setFloat('u_normalize', 1.0);
shaderMaterial.setFloat('u_effectBlend', 1.0);

This resulted in a shader that made the quads of the mesh face the camera, like this in the 3JS demo (just without the colors):

Creating the Fragment Shader

Next I needed to make it actually look like foliage, and for this, I had to make the fragment shader that was omitted from the 3JS demo. This is what I came up with:

ShaderStore.ShadersStore['customFragmentShader'] = `
	precision highp float;
	varying vec2 v_uvs;
	uniform sampler2D textureSampler;
	uniform vec3 u_color;
	
	void main(void) {
	    vec4 texColor = texture2D(textureSampler, vUV);

	    float luminance = dot(texColor.rgb, vec3(0.299, 0.587, 0.114));
	
	    gl_FragColor = vec4(u_color * texColor.rgb, luminance);
	
	    if (luminance < 0.75) {
	      discard;
	    }
	}`;

This was the texture I had from the demo:

The fragment shader takes that texture, determines the "luminance" at coordinates v_uvs (from the vertex shader) and if it is sufficiently dark (black basically) it does not draw that fragment. This is essentially using a cookie cutter on the quads we saw earlier to turn them into leaf shapes.

It then multiplies it by the color you pass in to make the fragment that color.

The texture and color were passed into the shader from Babylon like this:

const alphaMap = new Texture('https://douges.dev/static/foliage_alpha3.png', scene, false, false, Texture.NEAREST_NEAREST);
alphaMap.hasAlpha = true;
shaderMaterial.alphaMode = Material.MATERIAL_ALPHABLEND;
shaderMaterial.setTexture('textureSampler', alphaMap);
const foliageColor = new Color3(63 / 255, 109 / 255, 33 / 255);
shaderMaterial.setColor3('u_color', foliageColor);

Enabling the Shader for Use with Instances

At this point I tried to create an instance of my tree mesh and discovered that my shaders as written didn't work. The foliage mesh was invisible.

It turns out there are some adjustments you have to make to your vertex shader in order for it to work with instances in Babylon. First, at the top of the GLSL code, you need to add #include<instancesDeclaration> and at the top of the main() function you need to add #include<instancesVertex>. You no longer have to pass in world (so you can delete it from the uniforms), and instead you will use finalWorld (provided by the include I believe).

Once that's done, you should be able to create an instance of the mesh that already has the ShaderMaterial applied and it should just work.

Lighting Effects

At this point the foliage was all one shade of green, and didn't have the nice shadows that were in the 3JS demo. I found an example of a toon shader in the CYOS tool, and incorporated that into my fragment shader, resulting in its final form:

precision highp float;

// Varying variables for lighting calculations
varying vec2 vUV;
varying vec3 vPositionW;
varying vec3 vNormalW;

// Uniforms
uniform sampler2D textureSampler;
uniform vec3 u_color;
uniform vec3 vLightPosition; // Add a light position uniform

void main(void) {

  // Toon shader thresholds and brightness levels
  float ToonThresholds[4];
  ToonThresholds[0] = 0.95;
  ToonThresholds[1] = 0.5;
  ToonThresholds[2] = 0.2;
  ToonThresholds[3] = 0.03;

  float ToonBrightnessLevels[5];
  ToonBrightnessLevels[0] = 1.0;
  ToonBrightnessLevels[1] = 0.8;
  ToonBrightnessLevels[2] = 0.6;
  ToonBrightnessLevels[3] = 0.35;
  ToonBrightnessLevels[4] = 0.2;

  // Light calculation
  vec3 lightVectorW = normalize(vPositionW - vLightPosition);
  float ndl = max(0., dot(vNormalW, lightVectorW));

  // Apply toon shading
  vec3 color = texture2D(textureSampler, vUV).rgb;
  for (int i = 0; i < 4; i++) {
	  if (ndl > ToonThresholds[i]) {
		  color *= ToonBrightnessLevels[i];
		  break;
	  }
  }
  
  // Original luminance and transparency logic
  float luminance = dot(color, vec3(0.299, 0.587, 0.114));
  if (luminance < 0.75) {
	  discard;
  }

  gl_FragColor = vec4(u_color * color, luminance);
}

The addition of new varying variables also required me to update my vertex shader by adding calculations for those variables. Here is the final version of the vertex shader:

attribute vec3 position;
attribute vec2 uv;
attribute vec3 normal;

#include<instancesDeclaration>
uniform mat4 view;
uniform float u_effectBlend;
uniform float u_remap;
uniform float u_normalize;
uniform mat4 projection;

varying vec2 vUV;
varying vec3 vPositionW;
varying vec3 vNormalW;

float inverseLerp(float v, float minValue, float maxValue) {
  return (v - minValue) / (maxValue - minValue);
}

float remap(float v, float inMin, float inMax, float outMin, float outMax) {
  float t = inverseLerp(v, inMin, inMax);
  return mix(outMin, outMax, t);
}

void main() {
  #include<instancesVertex>
  
  vec2 vertexOffset = vec2(
	remap(uv.x, 0.0, 1.0, -u_remap, 1.0),
	remap(uv.y, 0.0, 1.0, -u_remap, 1.0)
  );
  vertexOffset *= vec2(-1.0, 1.0);
  
  if (u_remap == 1.0) {
	vertexOffset = mix(vertexOffset, normalize(vertexOffset), u_normalize);
  }

  vec4 worldPosition = finalWorld * vec4(position, 1.0);
  vPositionW = worldPosition.xyz;
  
  vNormalW = normalize(vec3(finalWorld * vec4(normal, 0.0)));
  
  vec4 worldViewPosition = view * finalWorld * vec4(position, 1.0);
  worldViewPosition += vec4(mix(vec3(0.0), vec3(vertexOffset, 1.0), u_effectBlend), 0.0);

  vUV = uv;

  gl_Position = projection * worldViewPosition;
}

Applying those shaders to my mesh culminated in a beautiful tree:

You can find a full, functional demo here.

I hope you find this tutorial helpful, and please feel free to reach out with any questions!

12.20.23
Thomas Burgess, *@thomasburgess.dev

The Underappreciated UX of the ChatGPT Mobile Scroll Behavior

2026-05-29T02:02:33Z

The ChatGPT mobile app UX is severely underappreciated.

Perhaps the best example of this is the scroll behavior. They've put a lot of thought into something so deceptively simple that most people don't notice how complex it really is.

I didn't realize myself, until I tried implementing it from scratch.

It's easy to forget that ChatGPT is not a typical text message UI. The introduction of streamed responses immediately following the user's messages actually necessitates a lot of new behaviors.

In a typical text message interface, when you send a message your message gets added to the bottom of the list of messages. When the person you're messaging responds, it also gets appended to the end. This behavior is very straightforward, and there are packages like Shopify's FlashList that make adding this sort of feature plug-and-play.

However, ChatGPT is very different. I'm still discovering new states that affect the scroll behavior, but here are some of the ones I've noticed so far:

Most of the time when you send a new message, it appears at the top of your screen (as opposed to the bottom in a normal texting interface). The response streams in below it and can go off the screen (no scrolling yet).
However, if you send a message when the first response did not go off the screen, it does not scroll so that your message appears at the top of the screen. Instead, your message appears below the response, and the subsequent response generates below it and also does not scroll.
If a response goes off the screen and you send a new message without scrolling to read the end of the previous response, opening the keyboard overlays the messages and sending the new message does scroll so that it is visible at the top of the screen.
There is whitespace allowed below responses if they are short enough to not take up the full screen. However, if there is whitespace below a response and you scroll back in the conversation, when you return to the bottom of the conversation it does not allow you to scroll so far that the whitespace becomes visible again (the bottom of the response is at the bottom of the screen, and you can't scroll past it into whitespace).
When you return to an old conversation, it shows the end of the last response at the bottom of the screen such that there is no whitespace below it.
If you've scrolled to the bottom and there's no whitespace below the last response, opening the keyboard does make the conversation view shrink above it. If you're not scrolled to the bottom, it does not.

What's also interesting is none of the above applies to web. On web, your messages scroll to the top of the screen, and the responses stream below it without scrolling, even on really short responses. That's it.

More and more apps are shipping their own chat interfaces, creating demand for a similar plug-and-play package like FlashList. I might release one myself once I have all the kinks ironed out in my own app, but if someone knows of something available already, would love to give it a try.

12.22.25

2 Weeks of Tesla FSD after 2 Years with Comma's OpenPilot

2026-05-29T02:02:33Z

I've had the Comma 3 device installed in my 2019 Hyundai Santa Fe for over two years and nearly 11,000 miles. The self-driving software it runs is called "openpilot," and for most of that time, I've been using a fork called sunnypilot.

Two weeks ago, I bought my wife a 2025 Tesla Model Y and subscribed to FSD (Supervised). Here are my first impressions comparing the two self-driving systems and what you can expect from each.

Highway King vs Nagging Driver

The Comma 3 excels at one thing: highways. openpilot is fantastic at keeping you in your lane and maintaining speed. If that's what you want, you'll be thrilled. Much more than that, and you'll be disappointed.

Comma has long promised a full self driving experience complete with navigation, traffic light awareness, and 90-degree turns, but it continues to fall short on those counts. They hoped to have released a driving model that "learned e2e in the imagination of a transformer" by November. It's now December, and we're still waiting.

That being said, on highways I prefer openpilot to FSD for one main reason: driver monitoring.

Tesla's driver monitoring is clunky. They recently removed the requirement of having to hold the steering wheel the whole time, but the driver monitoring they've replaced it with is unpolished.

Theoretically it's supposed use the driver-facing camera and take into account several factors (the primary one being your eyes) to determine whether or not the driver is paying attention. If it thinks your attention has strayed, the screen nags you to "apply slight turning force" to the wheel. It's easy to overdo the "turning force" and disengage FSD entirely.

Comma's approach is simpler: driver monitoring is based solely on head orientation. Are you looking forward? Great. Are you looking down or rubbernecking too long? Warning is triggered, and simply looking forward again dismisses it.

Which actually brings us to another important distinction between FSD and openpilot: what happens when you touch the wheel. Like I mentioned before, applying any significant amount of steering force to the wheel in a Tesla disengages FSD entirely. With the Comma however, you can steer without disengaging, and with sunnypilot, you can even brake. This flexibility is crucial to how much I enjoy using it.

Comma: No Surprises

FSD can be nerve-racking on familiar routes because it occasionally does unexpected things, like stopping in the middle of the road. With openpilot, it’s more predictable. Coming up on a left-turn-only lane? I steer manually without disengaging, and openpilot keeps driving as expected.

With openpilot, I always know it’ll stay in its lane and slow for obstacles. With FSD, I generally know what it’ll do, but the unexpected happens and is extremely jarring when it does.

Tesla: Moonshot

Tesla wants to remove the steering wheel entirely—any takeover by the driver is considered a failure. This bold approach of leaping to the final product means it can do some genuinely impressive things: get around slow drivers, park itself, or accelerate like a rocket at green lights. But the confidence with which it undertakes these maneuvers can be misplaced.

Comma takes the opposite approach, adding capabilities iteratively. As their "make driving chill" slogan suggests, the last thing they want to do is stress you out. They handle the mundane, leaving you in control of the complexities. If you want the car to make an unprotected left turn, Comma isn't there yet.

The ability to set a destination and watch FSD handle everything is incredible, and I hope Elon’s promises of full autonomy come true. For now, though, it’s still hit-and-miss.

Price: Race to the Bottom

FSD peaked at $15,000, but now sells for $8,000, with most opting for the $99/month subscription. Since FSD is non-transferable between vehicles, you'd have to keep the same vehicle for over six years to make buying outright worthwhile.

When I bought the Comma 3, it cost $2,000. Last month, you could get the Comma 3X for $1,000 during Black Friday.

I suspect Comma has bottomed-out on price, but wouldn't be shocked if the FSD price continued to drop as competitors advance.

Final Recommendation

Comma works with many new car models across most car brands, while FSD is Tesla-exclusive. For most people, Comma isn't the best option—it's the only option.

If you own a Tesla, there's still a case for adding a Comma. But for my wife and me, we're still subscribing to FSD.

I like monitoring the cutting edge, and FSD has a cool factor that Comma lacks. It's fun watching the screen visualize your surroundings and seeing your car think through challenging situations. Does it feel safe or chill? I think "exciting" is the better term, which is not a trait everyone wants in their self-driving software.

The Comma looks and feels like a dev device. It is running open-source software after all. You see a glitchy green line overlaying a camera feed of the lane in front of you, and that's it.

Most people looking to add smarts to their car today should just get a Comma. FSD is more for those who want to get a taste of the future that isn't quite here.

Looking Forward

Long-term, I'm more confident in Tesla reaching true autonomy than Comma.

The Comma 3 and 3X have 2 front-facing cameras and a single rear-facing camera, all mounted to your windshield. Comma says they could use their rear-facing driver-monitoring camera for side views, but that hasn’t happened, supposedly due to a lack of volunteer data labelers. I have significant doubts whether the rear-facing camera has sufficient coverage to be able to handle all situations.

In terms of processing, Comma's hardware is essentially an old smartphone. It's worked so far, but whether it's sufficient for the next level of autonomy has yet to be seen.

Tesla, with its HW4 board and full-coverage cameras is clearly better equipped for the vision-only approach. Elon seems confident that true FSD is achievable on HW4, but has begun expressing doubts about HW3, which he previously assured buyers was FSD-capable. HW5 is already in the works.

For now, I'm keeping the Comma in my Hyundai and FSD in the Tesla, rooting for both and glad for the competition that is pushing the technology further.

TL;DR

Driving with openpilot is like collaborating with your car. Driving with FSD is like handing control over but having to yank it back occasionally.

12.4.24

EDIT 6.24.25

Tesla FSD has come a long way since this post, and most of the drawbacks I mentioned no longer apply. It no longer nags or attempts anything crazy. The only downside really is the price, but at this point I'm happy to pay it.