BIT-101

My Hifi Setup

kp@bit-101.com (Keith Peters) — Sat, 05 Sep 2026 10:33:31 -0400

I work from home, so I’m at my desk in my home office a lot. When I’m not in meetings, I like to listen to some music. And I like to have various sources of music available. This post shows the various pieces I use for the different music sources I listen to.

The Receiver

The core component of this whole system is my Yamaha R-S202 receiver. It’s not a crazy expensive, powerful, mega high res device, but it works well for me. It’s the bottom component here:

As a receiver, it has an AM/FM tuner and an amplifier with multiple inputs, including BlueTooth, and supports two sets of stereo speakers.

I have some decent cheap speakers mounted to the wall above my desk, and they sound good to me. The receiver has more than enough power for listening to music at my desk.

I don’t really use the tuner on this device often. Most FM radio is crap, and you’re limited to what’s in your area. I have another solution for listening to live radio that allows me to pull in more and better stations from around the world.

Internet Radio Receiver

In the above image, the box on the top is an Ocean Digital WR-800 internet radio reciever. I wrote more about this device in these two posts:

But to summarize, this device lets me pull in tens of thousands of live radio stations and live internet radio streams from around the world. I have 37 stations saved to my favorites and it’s great to just tune to one that has the kind of music I’m interested in and let it play while I work.

This is piped into my Yamaha receiver as input one. Although it does have built-in speakers, it sounds 100x better through the receiver and external speakers.

Jellyfin

I self host a Jellyfin media server with my music, movies and saved TV shows on it. For a while I’ve been trying to figure out the best way to get that music library into my reciever so I can listen to it on the external speakers. Things I’ve tried:

Using UPnP from my Ocean Digital device. I was able to get it to connect to my music library but browsing the library on that device was slow and horrible.
An old phone that streams from Jellyfin, and sends to the receiver via BlueTooth. This worked fine, but I didn’t like the form factor. A phone just sitting on my desk, moved around, you have to pick it up to see what it’s playing or start, stop, change song, etc. Yeah, first world problems, but it was just a bad UX.
Did the same thing with my laptop off to the side of my desk, piping Jellyfin to the reciever via BlueTooth. This was better than the phone actually. Large screen, keyboard shortcuts, stays put. But I use my laptop for other stuff, so every morning I’d have to set it up and connect it and then undo it at the end of the work day. Yeah, I know, woe is me, I have it rough.

I also thought of setting up a Raspberry Pi with some streamer software and probably an audio hat, that could be hard wired into my receiver. But I’d still want a display. Otherwise, I’m back to using a phone to see a UI on the Pi which is streaming from my Jellyfin server. Might as well cut out the middle man and just use the phone. Looked into getting a small touch screen to attach to the PI, but that started looking complex and expensive.

Then I thought, swap out the old phone for an old tablet. I had an old Kindle Fire HD10 tablet sitting around unused for years. I put the Jellyfin client on that and BlueToothed it into the receiver. Brilliant! It’s a nice big touch screen I can see and control without picking it up. It sits in one place on my desk and doesn’t move. I used this for a week or two and was pretty happy with it. My only complaint was that it was actually a little TOO big.

So I went on line and bought a used Samsung Galaxy Tab A 8.0 tablet. This is from 2019, an 8-inch Android tablet. Cheap as heck. Just the right size. I applied updates to get it up to Android 11 then installed LineageOS on it which gets it to Android 15 compatibility, with Google Apps installed. Now I was able to run Symfonium, a really great, customizable music player that can act as a Jellfin client.

Here it is under my big main monitor:

Again, it sits right in that spot on my desk, 24/7. I don’t use it for anything else. It’s a fully dedicated music player. I can see what’s playing, control it easily. It connects to my receiver with BlueTooth, sounds great. Exactly what I was looking for all along. It’s in a plastic stand right now. But I might build some kind of heavier wooden stand for it to give it a bit more stability and weight.

Recorded shows

A while back, I posted about a command line app I created to record live streams at scheduled times. I called it RecPlay.

I’m still using that very regularly. But I improved the setup to use cron jobs to record the same shows every week. Here’s how that works:

RecPlay is installed on a computer on my network. A small, low powered NUC on my LAN.
I have a script there called record that looks like this:

#!/bin/sh
# 1 = station, 2 = length, 3 = name

dir=/home/keith/recording/$3
mkdir -p $dir
cd $dir
echo $(date) >> record.log
/home/keith/go/bin/recplay --station $1 --length $2 --name $3 >> record.log

echo "recording complete!" >> record.log

echo "renaming to m4a..." >> record.log
for file in `ls ./*.mp4` ; do
  echo $file >> record.log
  mv $file ${file%.mp4}.m4a
done

echo "syncing..." >> record.log
rsync -avz --ignore-existing --remove-source-files -e "ssh -i /home/keith/.ssh/<sshkey>" ./*.m4a <path to media on nas> >> record.log

echo "====================\n" >> record.log

Some obfuscation there, but basically it lets you pass in a station, length and name of show.

It changes into a dedicated directory for that show name and calls recplay to immediately record that stream for the given duration.

When the recording is done it changes the extension to m4a (mp4 is good for capturing a wide variety of streams, but Jellyfin sees that as a video, and m4a as audio.)

Then it syncs it to my NAS where Jellyfin is running.

I log each step in case things go wrong.

Notice that I did not use the scheduling feature of recplay. It starts recording immediately. That’s because I schedule all these via crontab. Here’s what what that looks like:

0 9 * * 2  /home/keith/recording/record wvew 01:30:00 Rustle_in_the_Bushes
0 13 * * 3 /home/keith/recording/record wvew 02:00:00 Post_Punk_Romantics
0 6 * * 3  /home/keith/recording/record wmbr 01:00:00 It_Came_From_Planet_Claire
0 7 * * 3  /home/keith/recording/record wmbr 01:00:00 Morning_Wood
0 17 * * 4 /home/keith/recording/record wers 01:00:00 Throwback_Thursday
0 10 * * 6 /home/keith/recording/record wvew 02:00:00 Psychedelic_Train

So at the given time and day of week, it records a given show for a given duration, by calling the above script.

Each recording automatically becomes part of my Jellyfin library, so I can play them from my tablet if I’m at my desk, or from my phone if I’m out and about.

Pretty happy with this whole set up.

Yuki

kp@bit-101.com (Keith Peters) — Tue, 28 Jul 2026 17:59:00 -0400

I lost one of my best friends ever today.

I saved this poem years ago, knowing that it would bring me comfort on this day.

"The House-Dog’s Grave"
I’ve changed my ways a little: I cannot now
Run with you in the evenings along the shore,
Except in a kind of dream: and you, if you dream a moment,
You see me there.

So leave awhile the paw-marks on the front door
Where I used to scratch to go out or in,
And you’d soon open; leave on the kitchen floor
The marks of my drinking-pan.

I cannot lie by your fire as I used to do
On the warm stone,
Nor at the foot of your bed: no, all the nights through
I lie alone.

But your kind thought has laid me less than six feet
Outside your window where firelight so often plays,
And where you sit to read — and I fear often grieving for me —
Every night your lamplight lies on my place.

You, man and woman, live so long it is hard
To think of you ever dying.
A little dog would get tired living so long.
I hope that when you are lying

Under the ground like me your lives will appear
As good and joyful as mine.
No, dears, that’s too much hope: you are not so well cared for
As I have been,

And never have known the passionate undivided
Fidelities that I knew.
Your minds are perhaps too active, too many-sided. …
But to me you were true.

You were never masters, but friends. I was your friend.
I loved you well, and was loved. Deep love endures
To the end and far past the end. If this is my end,
I am not lonely. I am not afraid. I am still yours.

— Robinson Jeffers

Recplay

kp@bit-101.com (Keith Peters) — Sun, 26 Jul 2026 19:42:54 -0400

I’ve been getting heavy into internet radio lately, as mentioned in this post and that post. A lot of what I listen to day to day is just music of a specific genre. You just let it play and it’s the same general stuff all day. But there are also some local stations and college stations, as well as NPR affiliated stations that I listen to that have schedules of different shows at different times of the day and week. Some of these are a DJ playing a specific type of music for an hour or two, or maybe a live show, sometimes interviews, news, talk shows, etc. Since these come on at specific times of the day, you either have to listen to them when they come on, or maybe they have an archive you can listen to some time later. Or… you can record them.

I spent a lot of time thinking about recording hardware. I considered getting a new or vintage cassette deck. I looked into hifi hardware digital recorders. There are surprisingly few of these, and they are outrageously expensive. I’m really surprised there aren’t more devices like this. It would just take an audio capture board and an SBC (single board computer) and some software and buttons. I started looking into making one myself. A raspberry Pi with an audio hat. But goddamn, rPis are so expensive now. rPi 5 with 4GB RAM is over well $100. 16GB is $350. That’s before you get a case and a power supply and a cooling fan and the audio hat. I looked into other boards that are less expensive, but in the end I changed gears a bit.

I decided to do it all in software. ffmpeg can record an live audio stream, like so:

ffmpeg -i https://some-audio-stream -c copy recording.mp4

So simple. But there’s so much more you can do with it. I wanted to be able to schedule future shows - have it start at a specific time. And record for a given amount of time - 30 minutes, an hour, two hours, whatever. And I didn’t want to have to type in streams, just the name of the station.

recplay!

So I built a little command line app in Go. It calls out to ffmpeg to do the actual capture and save, but it takes care of the scheduling, catalog of stations, text ui, etc. It does almost everything I need it to at this point, but I still have ideas for more features.

It’s called recplay because it records and plays streams, but also because on the old cassette recorders I had as a kid, you had to press the record button AND the play button at the same time in order to record. See the image at the top of the page.

Here’s the help to give you an idea of how it works:

In general, you specify a stream/station, and a date, time and duration to record, and an output file name. But you can skip most of those.

Or, you can choose a station and play that station without recording.

Here’s the list of stations, so you don’t have to remember long stream URLs:

If you just specify a station, it will start recording immediately for a default 30 minutes.

Also notice that if you don’t specify an output name, it makes one up.

If you want a longer or shorter recording, change the length.

If you want to record at a specific time, you can do that. Without a date, it assumes today.

Or specify another date.

Of course, on *nix systems, you can throw a & on the end of the command to background it. It’ll still record and exit when it’s done.

I’ve been using this all weekend, scheduling and recording all kinds of streams, and listening to them later. One of the biggest pain points was making mistakes in choosing the right date and time. Especially remembering that it has to be in 24 hour time. I’d type 08:00:00 thinking it was 8 pm, not am. So I fixed that up a bit…

Still, sometimes I mess up and wind up with a date/time that’s in the past. In that case, the app will start recording immediately, confusing me. So I patched that one…

The Station List

Initially of course, I hardcoded the station list, but eventually I knew it’d have to be externally editable. Now, when you start the app for the first time it looks for a config file in the system user config location. If it’s not there, it creates it with a few sample stations. This is just a json file that looks like this:

{
  "WERS": "https://15313.live.streamtheworld.com:443/WERSFM_SC",
  "WMBR": "https://wmbr.org:8002/hi",
  "WVEW": "https://audio-edge-n9hx8.yul.o.radiomast.io/20b6f77f-2177-46a2-b461-437a84273d5f",
  "WBUR": "https://fm909.wbur.org/wbur_www",
  "WNYC": "https://fm939.wnyc.org/wnycfm",
  "BBC4": "http://as-hls-ww-live.akamaized.net/pool_55057080/live/ww/bbc_radio_fourfm/bbc_radio_fourfm.isml/bbc_radio_fourfm-audio%3d96000.norewind.m3u8"
}

Once it’s there, you can add whatever you want to it, delete, change the URLs if you find a better one, etc.

Hardware

Initially I was just running this on my laptop, but for scheduling recording, that meant I had to keep my laptop from sleeping. Eventually I got a bit of hardware to run it on. A tiny NUC - a mini computer. The one I got cost only $169 US. It’s about 6 inches square and a couple inches high. It’s very low powered. Came with Windows 11 which ran like a dog. But I wiped that and put Ubuntu Server on it and it’s great. I spent a bunch of time setting up that enviornment and getting it so I can securely ssh into it. I put tmux on it, so I can just open a tmux pane, schedule a few recordings, then detach from that session and it stays running on the server. The recordings then get SCP’ed over to my NAS where they appear in my Jellyfin server. Quite happy with this setup.

TODO

I’m pretty happy where this has wound up, but I have a few other things in mind.

Some shows might start a few seconds early or go a couple minutes late. I’ve been manually starting them to record at times such as 8:59 and setting the length to be a couple minutes longer. I’m thinking of making a setting that does that automatically.
I started looking into some overseas stations like the BBC and realized that the UK is not in the same time zone as the eastern US. Who knew!? Doing time-based math is not fun, so I may have to put a time zone in the recording parameter.

Dependencies

As mentioned in the start, this is all just a fancy wrapper around ffmpeg, so you need to have that installed. The play functionality uses ffplay, which should come with your ffmpeg install I think.

Check it out!

https://codeberg.org/bit101/recplay

More on Drawing Crisp Lines

kp@bit-101.com (Keith Peters) — Wed, 08 Jul 2026 13:02:39 -0400

Following up on yesterday’s post about drawing crisp lines, I realized it would be nice to add some details on why putting lines on a 0.5 pixel makes them crisp and not blurry.

In many common drawing apis, when you draw lines, the pixel values you give mark the center of the line. If you are drawing a one-pixel thick line, it will draw it half a pixel on either side of the measurment you supplied. Because it crosses pixel boundaries, automatic antialiasing kicks in and the line is blurred to make it look like it is in between two pixels.

Here’s an simulated image that might help:

Here, each of the squares in the grid represents one pixel. Each of those pixels is measured not by its center, but by its top left corner. So if you set pixel 0, 0 to black, that whole top left pixel would turn black.

But when you draw a vertical line at a certain whole number pixel value, it puts half that line on the previous pixel column, and half on the given pixel column. So you get a blur.

In the second case, this is a one-pixel wide rectangle drawn at whole pixel boundaries. As you can see it only affects whole pixels. And it’s sharp and crisp.

The third image respresents a line drawn on a 0.5 pixel value. It draws the line half a pixel to the right and half a pixel to the left, which exactly fills that column of pixels, so it is crisp, just like the rectangle. No antialiasing is applied.

In the image below, I’ve drawn a series of one-pixel rectangles on the top half. Below that is a series of one-pixel lines, each with a bit of a horizontal offset. The one of the far left has no offset and is on an exact pixel, so if very blurry and looks thick. Moving left to right, each pixel has a bit more of an offset until the last one has a 0.5 pixel offset. This puts it within a full pixel and there’s no antialiasing.

This also applies to drawing rectangles. But in the opposite way. Rectangles should be on whole pixel values if you wan them to appear crisp.

Below are two squares, and two one-pixel wide rectangles. In both cases the first example is on a 0.5 pixel boundary, and the next one is on an exact pixel. The first ones look blurry and the next ones are crisp.

Finally, since we’re talking about pixels, let’s look at some honest-to-god PIXELS!

Using my USB microscope, here’s the left part of that second picture showing the whole pixel rectangle and whole pixel line. The top half with the rectangle only affects the single pixel. Below that it affects some to the left of that column, and overall it’s less dark.

And we can zoom in even more and really see it clearly.

Here, in the rectangle portion, the entire set of red, green and blue elements is blacked out completely. And the ones on either side of it are fully on.

But on the blurred version, a wider area is just dimmed down. That’s the antialiasing at work.

Further to the right in that same image, the line is at more of an offset and the antialiasing is less prominent, but it’s not quite the same as the rectangle.

And on the far right, where the line is on a 0.5 pixel value, there is no difference between the line and the rectangle.

If there’s any real takeaway from this post, it’s: Get a USB microscope! They are awesome!

Drawing Crisp Lines

kp@bit-101.com (Keith Peters) — Tue, 07 Jul 2026 16:54:28 -0400

Today I was working on some code where I was drawing a bunch of thin lines for various types of grids. After I had the basic code working, I went back and worked on cleaning things up. One thing I’d noticed is that some of the lines were… fuzzy. They were absolutely straight vertical and horizontal lines, but some were clean and crisp, others looked kind of blurry. And they all seemed to have slightly different widths.

Fortunately, I knew just what was going on. Tips I’d picked up on in the early 2000s working with Flash. I knew just what to do to make them clean and crisp and uniform. This worked in Flash, it works in JavaScript, it works in my library which is based on CairoGraphics. Native Processing doesn’t have this issue, but p5.js which is in the browser, does.

What’s happening is that even though the lines are completely vertical, they wind up at various locations on the y-axis. I purposely set up the code to make sure this was the case. I’m a bit rusty on my JavaScript, but put together this snippet:

const width = 773;
const height = 400;
const canvas = document.getElementById("canvas");
const context = canvas.getContext("2d");
canvas.width = width;
canvas.height = height;
context.fillStyle = "white";
context.fillRect(0, 0, width, height);
context.lineWidth = 0.5;

const count = 57;
for (let i = 1; i < count; i++) {
  context.beginPath()
  context.moveTo(i / count * width, 25);
  context.lineTo(i / count * width, 375);
  context.stroke()
}

Because I used random odd numbers for the width and count, they won’t divide evenly and most lines will be at some fraction of a pixel on the x-axis.

So part of what you want to do is round those x values off. We’ll start there. Just showing the loop this time…

for (let i = 1; i < count; i++) {
  context.beginPath()
  context.moveTo(Math.round(i / count * width), 25);
  context.lineTo(Math.round(i / count * width), 375);
  context.stroke()
}

That gets us here:

Not great. They’re all uniform now, which is nice… but they’re all uniformly blurry. Well, I said that rounding was part of the solution. The next part is to move the line so it’s exactly on a half pixel. We can do that by just adding 0.5 to each value…

for (let i = 1; i < count; i++) {
  context.beginPath()
  context.moveTo(Math.round(i / count * width) + 0.5, 25);
  context.lineTo(Math.round(i / count * width) + 0.5, 375);
  context.stroke()
}

And now, voila!

These look very clean and crisp.

But we can do better. With the above code, lines might move a bit further than you really want them to in some cases.

For example, say your x value was 103.57.

We round that to get 104.0 and then add 0.5 for 104.5.

We’ve moved it almost a full pixel to the right. A better spot for it would be 103.5, just 0.07 pixels to the left.

To handle that, we can make a function like so:

function roundHalfPixel(value) {
  return Math.round(value + 0.5) - 0.5;
}

We add 0.5, round and subtract 0.5.

So our 103.57 becomes 104.07. Rounded to 104.0 then subtract 0.5 to get 103.5. Just what we want.

And if we’re at something like 103.21, we add 0.5 to get 103.71, round to 104.0 and subtract 0.5 to get 103.5, which again is the closest half pixel.

Fixing up the code to use this function…

for (let i = 1; i < count; i++) {
  context.beginPath()
  let x = roundHalfPixel(i / count * width);
  context.moveTo(x, 25);
  context.lineTo(x, 375);
  context.stroke();
}

And that gives you the final image:

Honestly you probably won’t see much of a difference here in most cases, but it’s more correct and just might make something look better someday.

This will work on lines of any width. But sometimes the result is more striking than others. I’ve been using 0.5 pixel lines and the effect is subtle, but there. Here’s a comparison of the original code and the final code, with 1-pixel lines.

I only showed vertical lines, but it applies to horizontal lines too. You really only need this for straight vertical and horizontal lines mostly. But give it a shot on angled lines and see if it makes a difference. It’s also good on any rectangular fills (but use whole values in this case, as covered in the next post), gives them crisp edges. But don’t take my word for it. See what it does on your system.

That’s all. I just thought of this trick today and realized that I learned this more than two decades ago. And I’m not sure if I’ve ever heard anyone mention this trick or read anything about it any time recently. Wondered if it’s a lost bit of knowledge among newer creative coders. Maybe not, but figured it wouldn’t hurt to toss it out there in the wild again.

Part of the Problem

kp@bit-101.com (Keith Peters) — Fri, 03 Jul 2026 13:07:00 -0400

Anyone who follows me on line knows that I’m not the biggest fan of AI and where it is taking the world. I won’t make a huge rant about it, but I didn’t get in to programming to just describe what I want to code, have something code it, then point out the mistakes it made. Beyond that, there’s all the impacts on the world, the environment, the economy, workers, artists, other coders, and the kids who will grow up in this new world. That’s it. That’s my rant.

But recently, I have been wondering about my part in all this. I’ve been coding as a hobby for around 40 years and professionally for almost 30. I’ve always been excited about the next big thing. The new OS, new hardware, new devices. I remember a world WAY before everyone had a computer at home, much less an even more powerful computer in their pocket, or on their wrist. I remember a world before any normal person had access to the internet or even knew what it was. And I rejoiced in every new capability. I can have my own computer? Amazing! Doom? Real time 3D? Holy crap! I can render fractals on my own? Wow! I can download an album or any number of songs without having to buy physical media? Now I can stream that music? I can have a computer that I can carry around with me? Now I can have one that fits in my pocket? And I can use these things to talk to anyone anywhere in the world? WTF? I can create a page on this internet that anyone can see? I can write code and put it on my website, and have it to real time interactive physics simulations that anyone in the world can see as soon as I put it there? I can stream entire movies to any device? I can make my own movies and let other people stream them? Goddamn!

Every step, every new big thing I met with enthusiasm and excitement. I never really questioned where it was all leading. I guess I thought about it sometimes, but with rose colored glasses. It was just going to keep getting better and faster and higher resolution. I actually remember thinking that now that anyone in the world could talk to anyone else in the world instantaneously any time day or night, that war would inevitably die off. So naive.

Anyway, we finally got to something that I do not meet with enthusiasm and excitement. I meet it with concern and anxiety.

But I think… I think that this was always where it was leading. Faster, better, easier, more access to data, faster processing of that data. Less friction, more automation. There’s an app for that, and that, and yeah, that too. There’s no earning anything, no waiting, no saving up, not even any thinking about how to do anything. Just pay your monthly subscription and you have it all literally without lifting a finger. How is AI not the end game in this progression?

And I think I am complicit in this. I bought into it at every step along the way. I wrote about it, talked about it, shared the excitement about each little turn in the path. And now we are here and I don’t like where we’ve come. That doesn’t feel great.

In some ways, personally, I have an out. I’m in my 60s now. I can retire in the next few years. And leave this whole mess behind. But if I were still in my 30s or 40s now, I’d be having a serious crisis.

So for anyone younger, who is going to have to live with this for the next few decades, I’m sorry. We were so preoccupied with whether or not we could… etc.

Please do better than those who came before you.

Internet Radio II

kp@bit-101.com (Keith Peters) — Fri, 26 Jun 2026 17:40:05 -0400

The other day I posted about my internet radio explorations. A few people weighed in with their experiences and questions. You can see those in the comments of that post. Mostly it was me talking about adding streams, so I figured I’d more permanently document that in another post. This is mostly about finding, adding, and managing new stations.

Skytune

As mentioned, my receiver, and many others use the Skytune service. This is a huge database of streams, with metadata, images, locations, genres, etc. Probably at least 30,000. And they maintain the list pretty well. I’ve used other databases of streams that were pretty hit and miss. But in my experience it’s pretty rare for a Skytune stream to not work. I imagine they license the service to all these receiver makers and have the resources to keep their lists up to date. And if they slacked off, the manufacturers would look for other providers.

While you can search for and manage Skytune stations on the device, it’s not a great experience. Better is to open up https://skytune.net in your browser, on the same network your receiver is on. You’ll get to a search form like this:

You can filter by genre or location, and enter a search term. Here’s what I got for “SomaFM”:

How you are supposed to use this page confused me at first, because clicking on the thumbnails does nothing. Finally I figured out you have to click on the little heart icon on one of the stations. This brings up a modal like this:

There, you enter your receiver’s IP address (which is displayed on its screen) and whether you want to save the station or play it now. When you do that, it goes to the page where you can manage all your saved stations. This is a page served directly from the device itself. You can get there any time by entering your device’s IP address in the browser.

As you can see, I saved SomaFM’s “Drone Zone” station. But I already had another version of it saved. This happens now and then. There can be different versions. Sometimes different official versions of a station with different bandwidths, sometimes, I don’t know, unoffical versions too? You can try different ones and see if any are better than others.

Here you can also play, edit, delete or re-arrange your stations. The UI is pretty minimal, but is workable and stable from what I’ve seen.

You can also see what’s currently playing, change the volume or mute. Oddly, I don’t see a way to stop anything from playing.

Non Skytune Stations

Although Skytune has thousands of stations, eventually you’ll find a station you want to save that is not in their database.

So, you can add a new station. You’ll need a working audio stream URL and some info about the station - name, location, genre. Plug all that in in the bottom of that same management page, and the station will be added to your favorites.

There are some differences between Skytune and non-Skytune stations. Skytune does add some value for the stations in its database. It gets a thumbnail image, and generally displays more info on the screen than a manually added stream. For example, here’s a manually added station I put on today.

Just a default image. Some stream stats when it starts up, but then it’s just blank.

But other manual streams will show more info, such as the currently playing song, like this:

It just depends on what the stream sends over for metadata. I haven’t run across any streams that display anything but the default image though.

Compare that to an official Skytune station for SomaFM Groove Salad, with the official thumbnail, tag line of “Ambient Chill”, and what’s playing.

Where to find stream URLs

So where do you find the URLs to input? A few answers. Some sites will just give you the URL. Like SomaFM…

If your station doesn’t have a published stream URL, there are various sites that document these, with thousands of streams listed. Here are a few:

There are lots of sites that let you play streams from the site, but these ones offer the actual URLs to the streams to use in other devices.

And if you still can’t find a stream URL, there’s the hacker method.

Go to the page that lets you play a stream. Open the developer tools in your browser and go to the network tab. Filter for media files and play the stream. You might need to refresh if you didn’t get the timing right. Eventually you’ll get at least one listing there. Sometimes more. There are many possible configurations for streaming audio, so you might have to find the right listing by trial and error. But one of them should work.

Submitting a station to Skytune

Say you find an official stream URL for a station that’s not in Skytune. You can submit it by going to “Suggest Station” on the main site. They ask for a bunch of info, and I have no idea what their turnaround is, but it’s worth a shot.

Other

I should add that it’s not just music. There are news stations, talk stations, even podcasts stored on Skytune. I have several news stations and a bunch of BBC stations.

Oh, and Skytune lets you save your stations to disk and restore them from a saved file. Sadly, they are saved in some kind of binary text file, rather than something like JSON which might be usable elsewhere. Open source, this is not. :( But it’s a good idea to make a backup now and then.

Internet Radio

kp@bit-101.com (Keith Peters) — Tue, 23 Jun 2026 19:22:23 -0400

Background

I started buying music in the 70s. Vinyl. Not because it was cool, but because that’s how you bought music. It was either that or 8-track tapes. I never had an 8-track player. I had a record player. We didn’t call them “turntables”. We called them record players. Again, it wasn’t trendy. That’s just how it was. I wound up with a respectable vinyl library.

Cassettes were a thing too, but nobody bought music on cassette. You bought vinyl and recorded it to cassettes if you wanted to play it in your car. Walkmans (Walkmen???) weren’t a thing just yet, but they were around the corner. When I got a car, I invested in a decent car stereo with a cassette player and recorded many of my favorite albums and those cassettes lived in my car.

I left home early and at some point I wound up selling all my vinyl to eat. Sad. I still had my cassettes though. And I gave up my car and bought a Walkman. Not a real Sony Walkman, but a series of Walkman clones. That’s when I started buying new cassettes. I built up a respectable cassette library.

Then came CDs. I didn’t buy into that format though. I’d borrow my friends’ CDs and record them to cassettes.

Then came MP3s. I bought the first commercially available MP3 Player, the Diamond Rio PMP300.

Over the next many years I got rid of my cassettes (I don’t actually remember what I did with them) and built up a decent library of MP3s. And in the last bunch of years I’ve been upgrading those MP3s to lossless FLACs. I have a Jellyfin server at home, with TailScale, so I can access that music anywhere. And I have a few high res portable digital audio players.

So I’m in pretty good shape with my own personal music library.

Streaming

But sometimes I want to just turn something on and let it choose the music for me. Music that I may not have in my collection. Music that I may never heard of.

When I was growing up, this was called FM radio. It’s still an option, but it’s limited by the stations in your area. Even in a decent city like Boston, those options are very limited. And oh god, the ads. There have been times I’ve gotten in my car and driven somewhere 15 minutes away and only heard one song and a bunch of ads. And don’t get me started on those “morning shows”.

So there are streaming services. Spotify I never really loved, but I did subscribe to Tidal for a while off and on. But those are all about building playlists. You can listen to someone else’s playlist, which is kind of like radio, but not really. And, I don’t want to pay a monthly fee. I want radio, but not just local radio.

Internet Radio

Of course I’m no stranger to internet radio. I’ve dabbled over the years. Shoutcast, Ice, Pandora, others.

And I’ve been a big fan of SomaFM for ages and regularly throw them a donation to do my bit to keep them going.

A month or two I decided to go all in on Internet Radio though. I started with an Android app, that I used on an old Pixel phone hooked up via Bluetooth to my stereo. The app I got is called Radio Mobi.

I started with the free version and loved it so much I upgraded immediately. I use it in the car and I use it during work and usually go to sleep listening to it. It kind of took over my music listening world. There are many thousands of stations with every language, location, genre, time period, etc. that you can imagine. I have dozens of favorite stations saved, and bounce between them as the mood hits.

Hardware

The app on the phone via Bluetooth into my stereo was decent, but I was ready to commit to something more permanent. I wanted a dedicated Internet Radio receiver. I spent a good week or more looking over listings and reviews and Youtube videos. And finally settled on the Ocean Digital WR-800.

This device sits on top of my stereo receiver/amplifier. It has its own speakers which are so-so, but it has line out which goes into my stereo system, so I don’t use the built-in speakers. It’s on my home WIFI and I’ve got all the same stations saved as I do on Radio Mobi (give or take).

I turn it on when I start working in the morning and mute it when I’m in meetings. The stations change with my moods.

On June 17 (6/17), a local college radio station had 617 Day. 617 is the telephone area code for the Boston area. So they played all local music. It was fantastic and I learned of a couple new bands and found a new favorite, Slothrust (Slo Thrust? Sloth Rust? You choose) and bought one of their albums.

Many of the hardware internet receivers you’ll buy use a service called Skytune. A nice thing about this service is you can discover stations at that site, and send them to your receiver. When the radio is on, it sets up a local web server on your network where you can manage your station library, add new streams that aren’t in the Skytune network, arrange and manage them, remove stations, start or stop a station, change the volume, etc.

You can do all that on the device itself, but doing it on a computer or phone is great too.

The device also has:

FM Radio. I don’t use it because A. my stereo system has a better FM receiver. B. I now listen to local stations via their stream anyway - perfect reception.
SD card playback. I don’t use it because I have other solutions.
Media Center, aka DLNA/UPNP. Nah.

I just use the internet radio function and it’s fantastic.

Just remembered…

I just recalled that some years ago I tried this experiment with a different device. The CCrane CC Wifi Internet Radio.

This thing is crap. Horrible UX, many unworking stations, horrible sound quality. I used it briefly than didn’t touch it for years. Tried to use it again recently and I couldn’t get a single station on it. Must have changed server configs or something. I trashed it.

Weirdly, the Ocean Digital I just bought was only $10 more than the CC WIFI I bought eight years ago.

Summary

Ditch Spotify! Discover internet radio. You won’t go back. Use the money to buy music.

Actually, I don’t like telling people what they should or shouldn’t do. But I recommend trying it out. You might love it.

XT Eink X4

kp@bit-101.com (Keith Peters) — Wed, 08 Apr 2026 19:04:46 -0400

I have been a big fan of eInk book readers for just about 17 years now. In April 2009 I got my first Kindle and had a number of follow ups over the next many years. Ten years later I moved over to the Onyx Boox line of ereaders, with a Nova Pro 7.8" device, then a Note Air 10.3" device, a Poke 3 6" device, which I sat on and broke about a year ago, so got a Go 6 6" reader to replace it. I also have a Kobo Clara color eInk device and a Remarkable eInk tablet.

A month or so ago I caught wind of the XT Eink devices, in particular the X4. It’s a tiny little 4.3" device! I watched a bunch of videos about it and considered getting one but I didn’t really need a new ereader. (See paragraph one.) But then a week ago I saw some more posts about it and got interested all over again and went for it. It goes for $69 USD, so worth a shot, right?

XT Eink X4

It arrived over the past weekend and I’m in love with it!

The Bad

OK, the thing cost 70 bucks. It can’t be amazing, right? Here’s what it lacks:

No light
No touch screen
No dictionary
No search
No audio
No apps
Only supports epub and txt
Minimal formatting options

If this were a general purpose reader, many of those would be big problems. But none of them bother me. This is not my every day reader for when I’m on the sofa or reading before bed. This is the thing that’s always in my pocket and if I find myself waiting in the car, or in line somewhere or whatever, rather than doom scrolling on my phone, I read a couple pages of the book I’m on.

The Good

Just about everything else on this is great, especially for the price.

Magsafe

It’s got a Magsafe compatible magnet on the back. The marketing says it’s for sticking on the back of your phone. But I think it doesn’t really work that well with a lot of phones. Not with my Pixel 10. Just because of the camera bump - it works fine if I put it on sideways, but that’s kind of useless. But you can get a magnetic case for it and it’ll work with any Magsafe accessories.

Note, it does not charge via Magsafe - only via USB-C.

The Battery

I don’t know, it came charged, I’ve had it for 5 days, used it a bunch, it’s at 93%. Very promising. As I said, charges with USB-C.

The OS and Reader apps

I can’t say much about the factory OS and software. I only tried it for a short while. And then…

Crosspoint-Reader

Everyone suggests Crosspoint-Reader which is a 3rd party, community-developed firmware and reader app. It’s great. It doesn’t have all the bells and whistles of my favorite ebook reader app, KOReader, but it does a lot. Fonts, font size, a fair amount of formatting options, cover display on sleep, status bar customization, syncing progress with KOReader (though it’s very buggy in my testing), wifi book transfer, etc.

The Controls

As mentioned, it’s not touch screen, but has a power button on the left and then back/forward buttons on the side, that look like volume buttons on a phone. Then on the bottom front, there are what looks like two buttons but are really four. The two on the left are back and select, and on the right are additional back/forward buttons. Overall, pretty easy to get around with, but because it’s so tiny, and you’ll have to change pages so often, the button placement is not perfect. Using a Magsafe ring on the back to hold the device with makes this a lot better.

Performance

I’ve been surprised how zippy this is. It does take a few seconds to start up (like under 5 on Crosspoint) but page turning and menus are way faster than I expected on such an inexpensive, tiny device.

Readability

The resolution is pretty low - 220 ppi. But it’s fine for reading a book. The screen is small, but you can adjust the font size to your liking. Of course, if your vision requires you to use a very large font, you’re going to be much better off with a larger screen - the Crosspoint software offers font sizes of small, medium, large, extra large. Extra large will get you a medium size paragraph on the screen. Small will get you a couple of paragraphs. The default medium is about right for me.

Note that there’s no search function on either the default or the Crosspoint software. The default software has bookmarks, but oddly, Crosspoint does not. But both allow you to navigate by chapters.

X3

Just released, or just about to release, is the next version, in reverse numerical order, the X3. It’s even smaller! 3.7"

XT Eink X3

It has pogo pin charging and better button placement, shake to turn pages, NFC. $79 US. I think I’m good with the X4. It’s about as small as I’d want to go.

Photos

I’ll just close out with some photos.

A New Book in the Works!

kp@bit-101.com (Keith Peters) — Sun, 29 Mar 2026 12:10:20 -0400

Hot on the trails of Coding Curves, I’m excited to announce my next writing project…

The Creative Coding Compendium, Volume I

Most of my books up to now have been focused on specific topics - fractals, chaos and strange attractors, or 2D curves. Or they’ve been targeted to specific coding platforms - Flash or iOS. For my next round of writing, I want to open up the flood gates and just write about creative coding. There are so many techniques I’ve learned over the years. And I’m excited to share them. I’m not trying to force any theme on them. Each chapter is standalone and has no solid connection the previous or next chapter.

Furthermore, I’m going all in on the pseudocode bandwagon. I wrote about so many cool coding topics in the 2010s, but all of those books are out of print. And while the principles in them are just as useful now as when I wrote them, the languages and platforms I wrote those books for are dead. Some of the early stuff involved putting code on animation frames in the Flash timeline and looping the timeline back to the frame with the code on it repeatedly in order to achieve animation - a technique that would leave most modern programmers scratching their heads. So going forward, I wanted to write code in a language that could not die - because it doesn’t exist.

A big part of the inspiration for this kind of book is Clifford Pickover’s early books, Computers and the Imagination, Computers, Pattern, Chaos and Beauty and Mazes for the Mind. These are wonderful books filled with crazy ideas and images, with source code written in BASIC. While the code is diligently commented and described in detail in the text, it’s not always the easiest stuff for a programmer in 2026 to parse.

But it’s not impossible and it’s a lot of fun to just browse through one of his books, see an interesting image, convert the code over to some other language and start messing with it.

I’m hoping that this new book I’m working on, and any following it, can continue to have a long useful life ahead of them, no matter what programming looks like 20 years from now (assuming that humans are still writing code in 20 years… but I digress and make myself sad.)

Anyway, perhaps you’d like to see the table of contents.

Three hundred pages of projects, lots of code and graphics, concepts, suggestions of where to take things. And then three useful appendixes covering details related to other code in the book.

Where we at?

As of a week ago, all the chapters had completed first drafts with images and working code. I gave myself a week off and today started on first round editing. Cleaning up typos and weasel words and things that I read and say to myself, “what the hell was I trying to say here?”

When that’s all done, I’ll go through all the code and reproduce it in 2-3 different languages/frameworks. Likely vanilla JavaScript/Canvas, Processing, and Python with PyCairo. Just to make sure that I’m not telling people to do things that will be impossible in any of the more common platforms. Or at least to be able to say, “hey watch out if you’re using platform X, you’ll need to do this…”.

That may result in some more copy changes. And after that I’ll go through the whole book from top to bottom at least one more time making sure it’s as good as it can be.

So… many more weeks of work on this. But I’m very excited about it.

And after that?

As you may have noticed, this one has “Volume I” in the title. My initial brainstorm came up with enough subjects for at least two books, so a second volume is my intention. Maybe more? Who knows. Let me get through this one at least.

Without getting too philosophical about things, as I get older I do think about what I will leave behind. Especially in a world where I suspect people won’t really be learning to code much longer. Not in the same sense that we think of “learning to code”. They’ll tell a computer what they want, and some code will be generated, which may or may not be viewed, and only partially understood if anyone does view it. So if I can leave some artifacts behind about what it was like in the old days, that will make me a little less sad about where things are headed.

Anyway, stay tuned for more updates.

Here are a few sample images from the book:

Coding Curves Is Live!

kp@bit-101.com (Keith Peters) — Mon, 23 Feb 2026 07:00:00 -0500

I did it! It’s live! It’s for sale! My seventeenth book!!!

You can buy it on Gumroad: https://bit101.gumroad.com/l/coding_curves

It’s along side my last self-published book, https://bit101.gumroad.com/l/playingwithchaos that was published nearly 13 years ago!

It’s only $9.99 US, which is pretty cheap for a book these days. And if you buy either one, you can get the other for 40% off. Look at me being a marketer!

When you buy it, you’ll get four versions of the book:

EPUB - you can read this on just about anything, and even add it to your Kindle library and Amazon will convert it to whatever format it needs to be in to read on a Kindle.
PDF - also readable just about anywhere on anything.
KEPUB - this is targeted towards Kobo readers. It’s actually the exact same EPUB file, but with a different extension, which causes the Kobo device to use a different renderer which is much better than their usual EPUB renderer.
AZW3 - one of the native Kindle formats. Really only useful if you want to transfer the file to your Kindle device via USB.

All four of these are free from DRM. The EPUB was validated with EPUBCheck which is a fantastic tool if you are creating EPUBs. It found a bunch of issues and I was able to fix all of them. And all the other formats were generated from the EPUB.

If you want to know more about the kind of curves you’ll be creating with this book, I created a mini site to show them off:

https://bit-101.com/codingcurves/

What’s next?

Getting back into serious writing was a whole lot of fun. I realized how much I’ve missed writing books. The new workflow I’ve developed makes writing and self-publishing a full book about as easy as it can be. With that said…

I’ve already started the work on my next book, Coding Color! This is a book about writing code to create, modify, manage, and effectively use color, color schemes, and palettes. And understanding and being able to convert between different color spaces and models. As you can see by my posts over the past few months, this is an area I’ve been doing a very deep dive into recently. I’ve learned a lot. This isn’t going to be a theoretical or philisophical book about color. It will be very hands on, writing functions that do things with color. I’m very excited about being able to immerse myself in this project now that Coding Curves is complete. I’m sure I’ll be posting more about it soon!

Coding Curves: The Book

kp@bit-101.com (Keith Peters) — Mon, 16 Feb 2026 12:00:13 -0500

Back in late 2022, through the start of 2023, I wrote fourteen blog posts in a series called “Coding Curves”. That’s still up on the 2017-2023 version of this blog:

https://bit-101.com/2017/2022/11/coding-curves/

My intention was always to compile these into a book. I’d wanted to write this book for a LONG time, and started it multiple times. Only by writing it post by post and publishing it piecemeal was I able to get it out. But three years went by and the book form never emerged. Part of it was plain inertia and procrastination, but there were also technical hurdles.

One issue was that the original posts were written in WordPress. Converting that to EPUB/PDF meant probably converting it to markdown first, then running it through pandoc to create the final book. Or, I could grab the HTML that WordPress generated and try to use pandoc on that, but There was SO MUCH excess HTML and CSS added, that I didn’t feel great about that.

But since then I’d converted that whole site to markdown and used Hugo to publish it as a static site. So now I had the markdown. It still needed a lot of cleanup, but it was doable.

The second big issue was that, while I’d previously self-published Playing With Chaos using pandoc, my build process was really cumbersome, and 13 years out of date. I hesitate to admit this, but the book build was written in Ant! You remember Ant? It’s basically “make” or “just” but the build file is written in XML. Yikes!

It was also very tailored to creating books for publishing on Amazon Kindle. It used a proprietary kindlegen.exe command line app (yes, Windows) and a handful of custom Python scripts to massage the table of contents into shape. I really didn’t want to look at this.

But when I finally got up the courage, I found that I could strip it way, way down. In fact, I could do the entire build with just pandoc. It was just a matter of passing all the markdown files to pandoc and outputting the result to an EPUB. There were a few command line switches for things like generating a table of contents, adding a cover image, embedded font, and CSS file. But no other scripts or tools in the toolchain. Great. (Needless to say, I am no longer using Ant, either. Moved it over to just.)

Cleaning up the markdown was a pain. All the images carried through the original WordPress image tags that had multiple sizes of the same image listed. And a ton of punctuation characters - quotes, dashes, greater than, less than, etc. - were expanded to HTML entities. I became somewhat of an expert with sed which allowed me to clean all of that up across all of the chapter files. But even after that, it took a lot of manual clean up to get it into shape. And then I had to grab all the right images from the site and put them in the book folder and make sure they were all correctly linked.

All that was just clean up. After that began the editing.

First, spell check, grammar check, my own custom script to find “weasel words” - cliches that don’t add any value to the text.

And checking all the pseudocode to make sure it was correct. To do this, I made a little app that allowed me to paste in the pseudocode from the book and with relatively few adjustments, let me run it in Go using my own existing libraries. I’m not talking about transcoding or transpiling. I made function definitions that matched the pseudocode functions. These were wrappers for the real functionality in my library. For example, here are a few:

func canvas(w, h float64) {
	width = w
	height = h
	context = cairo.CreateContext(w, h)
	context.ClearWhite()
	context.SetSourceBlack()
	context.SetLineWidth(0.5)
}

func lineTo(x, y float64) {
	context.LineTo(x, y)
}

func moveTo(x, y float64) {
	context.MoveTo(x, y)
}

func stroke() {
	context.Stroke()
}

I also made a bunch of aliases to things, and “global” vars.

var (
	cos           = math.Cos
	sin           = math.Sin
	tan           = math.Tan
	sqrt          = math.Sqrt
	atan2         = math.Atan2
	acos          = math.Acos
	width, height float64
	context       *cairo.Context
)

Then, I could paste in a pseudocode example straight from the book, like this:

canvas(800, 800)
moveTo(100, 100)
lineTo(700, 700)
stroke()

And that would just work!

All this saved me from having to re-write each snippet in some other language in order to test it. I could actually run most of the pseudocode as-is.

After all that, I just had a lot of re-writing to do. Making it read more like a book, not a bunch of blog posts. Removed extraneous passages and awkward wording, added better examples and explanations where needed, generated new code samples and images where called for. In some cases, I rewrote almost entire chapters, with all new code and images.

I also removed all the animation examples, sadly. I imagine there are ways to embed animations in some ebook formats, but I didn’t want to go there.

All of this took several weeks. Whole weekends would fly by when that’s almost all I did. But I’m at the end of it. It’s in great shape, but I’m going through one more pass of the whole book cover to cover and making sure everything is clear and sounds right. I hope to get through that in another week or so.

I considered adding a few more chapters. There are some topics I think would fit in well - stuff I discovered in the past few years. But I decided to release it as-is. As an ebook, I can easily add more chapters later and release an updated version.

The plan is to release it on Gumroad, alongside Playing With Chaos. DRM-free, EPUB, PDF and AZW3. Watch this space.

Isometric Engine 2026 - Part I

kp@bit-101.com (Keith Peters) — Sun, 25 Jan 2026 10:32:18 -0500

Isometric drawing is always a fun thing to work on. I say always because I’ve created many isometric frameworks over the past 20+ years. I think I wrote a book chapter or two about it, and published a couple of Coding Math Videos about it … 10 years ago (OMG):

In those videos I discussed the difference between true isometric, where everything is “one angle” and dimetric, where the width and height of a tile are generaly in a 2:1 ratio. I drew some tiles manually, and loaded in prerendered tile sets and drew those all in various locations. I used the dimetric version, as for that kind of pixel art, it works out better (as described in the videos).

All that’s good but in recent years I’ve been taking a very different approach to doing isometric. Rather than just drawing flat faces, or pre-rendered isometric tiles that someone has hand-drawn - usually at a fixed size - I wanted to draw isometric shapes of varying sizes, with dynamic, high resolution textures, with (somewhat) dynamic shading. I’ve made a few attempts at this, and while it generally works pretty well, I generally got hung up on the API or interface to creating an isometric box, sizing it, positioning it, assigning textures for each of its three faces. For a single box, that’s not too bad, but when you have a bunch of boxes, and you want each one to have a different color and texture… it gets crazy fast. And the last few iterations I did on this were just so frustrating to use that I never used them.

But my most recent attempt is promising and I’ve been having a ton of fun cranking out random images with it over the past few days.

And my recent color palette work has really paid off in making those images look better than they would have.

Drawing a box

Let’s start drawing a single box in a fixed location with a fixed size and fixed coloring. I’m going to be doing true isometric rendering here, meaning that our angles and sizes for each face are going to be the same. Here’s a simple box:

It’s composed of three identical diamond shapes. The obtuse angles are all 120 degrees and the acute angles are all 60 degrees. Every line drawn there is exactly the same length. Hence, isometric - “one measure”. Also, if you removed the internal lines, you have a perfect regular hexagon. The world needs more hexagons.

Now, I want to draw a “texture” on each one of these faces. Don’t get caught up in the word “texture”. I just want to draw something there, a flat color, a pattern, random noise, whatever. But I want whatever I draw there to look like it was actually sitting on that face of the box. For example, I don’t want to draw a circle like this:

Yuck! No! I want it to look like there’s a circle on that face of the box and it’s rotated and skewed like the face itself. Consider that the face of that box is represented as a square - the way you would see it if you were to walk round to the right and crouch down so you were looking at it head on.

So, how do we distort that square, and whatever is in it, to look like the isometric box face we see?

The answer: with 2D affine transforms. Enter the Matrix!

First, I’ll just draw the untransformed face. A 300x300 pixel square filled with gray and a black, 100-pixel radius circle inside of it. Its origin point is the center of the box. I know it’s 300x300 because I made the isometric box with sides of 300 pixels.

Now I need to kind of skew the box up and a bit to the left to make it fit in that face. Through a bit of analytical reasoning, working out the math, and a WHOLE LOT of trial and error, I came up with a transformation matrix that I can apply to the canvas before drawing that puts that face right into place. Here it is:

[sin(pi/3) -0.5]
[    0       1 ]
[    0       0 ]

Most drawing APIs have a way to set the transformation matrix of a canvas or context object. In JavaScript for example, you’d do it like this:

context.transform(Math.sin(Math.PI/3), -0.5, 0, 1, 0, 0);

I think in Processing, it looks like:

applyMatrix(sin(PI/3), -0.5, 0, 1, 0, 0)

I’m using a Go wrapper for CairoGraphics with a whole ton of custom functions and currently it looks like this:

context.TransformValues(math.Sin(pi/3), -0.5, 0, 1, 0, 0)

For the rest of this series I’ll pretent I’m using JavaScript.

Anyway, if you apply that matrix just before drawing the square/circle shape in the last image, you get:

Voila! Perfect! To be fair, this was the simplest one, but let’s do the rest!

Hot tip: make sure you save the state of your context before you do each set of transforms, and restore it before doing the next set. Otherwise, you’ll have multiple transforms layerd on top of each other and you’ll get very confused.

Starting with the left face, I’ll draw another slightly different “texture” and then transform it. First, the untransformed square:

Now, I could flip that box around that center line and then skew it upwards, but say my texture had some text in it, or something else I did not want to appear backwards. So what I’ll do is first slide it up to the left by 300 pixels using a translate, something like:

context.translate(-300, 0);

Now we are here:

Now we just need to skew it up. Here would be the code for that:

context.transform(Math.sin(Math.PI/3), 0.5, 0, 1, 0, 0));

That’s essentially the same transform we used for the right side, but with a positive 0.5 instead of a negative one this time. This causes it to skew from the opposite side.

Now if we add that transform after the translate, we get:

That’s obviously wrong. It has to do with the order of matrix operations. This is something that it takes wrapping your head around. Often you have to apply them in what seems to the brain to be the inverse order of what you want to do. If we do the transform and then the translate, like so…

context.transform(Math.sin(Math.PI/3), 0.5, 0, 1, 0, 0);
context.translate(-300, 0);

Then we are golden!

Side note, there are ways to combine multiple transforms into a single matrix, but I’m fine with making two or three calls. In any case, you’d have to make two or three calls to assemble the final matrix anyway, so I’m leaving it as is.

Other side note: earlier I had referenced the setTransform method, but in this case you want to use the transform method. The difference here is that transform will alter the existing transform, adding a new transform to it, whereas setTransform will remove any previous transform that had been applied and then transform the context as specified. Make sure you know the difference between these two and use the right one.

Onto the final side, the top! I’ll make another face, untransformed at first.

Here, I labeled one corner, “top left” to make a point. With the right face, it was a simple skew upwards. With the left face it’s pretty obvious that we wanted to slide the face over and skew it up, to keep the same orientation we had. But on the top, you have a choice. How should that top face be aligned? With the top left corner on the left side, or at the top? I think putting it on the right or the bottom would make it look upside down, but the first two choices seem equally arguable. I’m going to put it at the left side.

I won’t go through it step by step, but here are the transforms I worked out:

context.rotate(-Math.PI / 3);
context.transform(Math.sin(Math.PI/3), 0.5, 0, 1, 0, 0);
context.translate(0, -300);

There are probably other ways to do it, but these, in this order, give us this:

At this point, I’ve removed the original line drawing, so all you’re seeing is the three faces.

If you want the top left to be up in the top, you can use this set of transforms:

context.rotate(Math.PI / 3);
context.transform(math.Sin(Math.PI/3), -0.5, 0, 1, 0, 0);
context.translate(-300, -300);

Now, for every box, for every face, I can just draw to it like I was drawing to a regular 300x300 pixel canvas. Here, I just drew three different textures, that I already had sitting around in my library:

I can even load in images to apply to each side…

End of Part I

OK, that’s pretty good for a first post. In future posts, I’ll cover lighting/shading, sizing, positioning and a few other things.

We Have Comments!

kp@bit-101.com (Keith Peters) — Thu, 22 Jan 2026 20:06:00 -0500

After getting search working here (and on my other blogs), I was pretty excited. I started thinking, “what else can I do here?”

The one last thing I missed from leaving Wordpress was comments. I looked at a whole bunch of commenting systems. All the ones here: https://gohugo.io/content-management/comments/

I discovered that Hugo even has a Disqus comment template built in. I got that one working as a test, but didn’t like it visually at all, and honestly, just kind of had a bad feeling about Disqus in general. It didn’t help that Disqus wouldn’t even load if you have tracking prevention turned on.

I tried a few others, not much success. The one other one I got working also didn’t look good.

The rest were either paid services, which was a path I wasn’t going to go down, required extensive server side set up, or were tied to services I don’t use - Matrix, Discourse, github.

Then someone on Mastodon suggested I use Mastodon/Fediverse to handle comments. This was kind of in the back of my mind, but I didn’t really have a good idea how that would work. This post was given as a reference:

https://beej.us/blog/data/mastodon-comments/

This really clicked. Great post! The author built his implementation for his own private static site generator, which is impressive. I built mine for Hugo. The main thing I needed was the fact that given a Mastodon post ID, you can load all the info about that post as JSON. He was doing it with env vars and tokens in his source files that get replaced with the vars during the site build. I had some ideas on how I could do that a bit differently in Hugo. Anyway, with the knowledge from that post, I was off to the races.

My implementation

Hugo has a heirarchy of templates and partial templates and markdown files with headers and params. I needed to figure out how to get all that in there.

1. Build the comment UI.

This is straight up JS and HTML and CSS. I created a div and some JS to load and parse the JSON and turn it into styled HTML elements. I did all this at first in a simple HTML page, totally unconnected from my site.

<!DOCTYPE html>
<html>
  <head>
    <title>Comments</title>
    <link rel="stylesheet" href="styles/main.css">
  </head>
  <body>
    <h2>Comments!</h2>
    <div id="comments_div"></div>
    <script type="text/javascript" src="src/main.js"></script>
  </body>
</html>

And the JS that this loaded:

const id = "115939969899573885";

(async function load_comments() {
  const url = `https://mstdn.social/api/v1/statuses/${id}/context`;

  try {
    const response = await fetch(url);
    if (!response.ok) {
      throw new Error(`Response status: ${response.status}`);
    }
    const json = await response.json();
    populate_comments(json);
  } catch (error) {
    alert(`Error loading comments: ${error.message}`);
    console.error(error.message);
  }
})()

function populate_comments(json) {
  const commentsDiv = document.querySelectorAll("#comments_div")[0];
  const posts = json["descendants"];
  for (let i = 0; i < posts.length; i++) {
    const post = posts[i];
    const postDiv = document.createElement("div");
    postDiv.innerHTML = `<h3>${post.account.acct}</h3>${post.content}`;
    commentsDiv.appendChild(postDiv);
  }
}

The load_comments function was straight from the post above. The populate_comments part I just threw together. With a bit of styling copied over from this blog, I got this:

I was pretty happy with this as a proof of concept. Tested different post IDs and it was all good.

The one thing to note here is that it’s just displaying the responses to the original post I’m using. It doesn’t display that original post. But I fixed that later.

Onto the next step.

2. Figure out how to pass the Mastodon ID to the code.

I couldn’t just hard code the Mastodon ID like that. It had to be tied to each individual blog post. In Hugo, each post or page has metadata in a header. You can add your own parameters there. Here’s the header of this very post:

---
title: "We Have Comments!"
date: 2026-01-22T20:06:00-05:00
draft: false
postImage: ""
summary: "OMG we have comments, via Mastodon!"
tags: ["info"]
mastodonId: "115941984813394822"
---

As you can see, I added a mastodonId parameter there. Now we have a bit of a catch-22 here. I need to create a Mastodon post about the blog post in order to get an ID to put in the blog post. But the Mastodon post needs to link to the blog post too!

~~So what I did was:~~

~~1. Created a private post on Mastodon, grabbed the ID and put it in the post.~~

~~2. Then finished the post, published it, and got the URL.~~

~~3. Finally, edited the Mastodon post, added the URL and made it public.~~

In all transparency, I’m writing this in past tense, but as I’m writing it I’ve only done the first step and I’m finishing the post now and hoping it all works out in the long run! But when you read it, it will all be in the correct tense. Messing with my head a little bit. And I’m hoping it works as expected.

Update:

That didn’t work. I guess you can’t turn a private Mastodon post into a public post. So for now there’s going to have to be a bit of back and forth.

Publish the blog post with no mastodonId
Create a Mastodon post with the link to the blog post
Copy the mastodonId into the post and republish.

If anyone has an idea for a workaround for that, let me know.

3. Create the UI for Hugo.

This is done as a “partial”, which is just a partial HTML template that can be added to other HTML templates. Here’s the comments.html partial:

{{ if .Params.mastodonId }}
<h2>Comments!</h2>
<div id="comments_div">
  <a href="https://mstdn.social/@bit101/{{ .Params.mastodonId }}">Comment or read on Mastodon 
    <img alt="mastodon icon" src="/blog/icons/mastodon.png" align="top" class="social-icon"/>
  </a>
</div>
<script type="text/javascript" src="/blog/js/comments.js"></script>
<script type="text/javascript">
  load_comments({{ .Params.mastodonId }});
</script>
{{ end }}

As you can see, it’s all in an if block, so if I don’t add a mastdonId to a post, no comment UI will show up at all.

This partial will be passed a context object that contains the metadata params I described above. So .Params.mastodonId will contain the ID I added. This gets passed to the load_comments function.

4. Update the JavaScript.

I updated the script that loads the comments. As mentioned earlier, loading the post context with https://<mastodon.server>/api/v1/statuses/<ID>/context gives you the data around a particular post - any posts it is a child of (ancestors) and any child posts of this post (descendants). But it doesn’t give you any info about the post itself. So I had to make two calls - one to load the initial post that I made, and another to load the descendents.

Here’s the final JS:

async function load_comments(id) {
  const contextURL = `https://mstdn.social/api/v1/statuses/${id}/context`;
  const postURL = `https://mstdn.social/api/v1/statuses/${id}`;
  try {
    const postJSON = await getJSON(postURL);
    const contextJSON = await getJSON(contextURL);

    populate_comments(postJSON, contextJSON, id);
  } catch (error) {
    console.error(error.message);
  }
}

async function getJSON(url) {
  const response = await fetch(url);
  if (!response.ok) {
    throw new Error(`Response status: ${response.status}`);
  }
  return await response.json();
}

function populate_comments(postJSON, contextJSON, id) {
  const commentsDiv = document.querySelectorAll("#comments_div")[0];

  // original post
  const postDiv = document.createElement("div");
  postDiv.innerHTML = `<h3>${postJSON.account.acct}</h3>${postJSON.content}`;
  commentsDiv.appendChild(postDiv);

  // replies
  const posts = contextJSON.descendants;
  for (let i = 0; i < posts.length; i++) {
    const post = posts[i];
    const postDiv = document.createElement("div");
    postDiv.innerHTML = `<h3>${post.account.acct}</h3>${post.content}`;
    commentsDiv.appendChild(postDiv);
  }
}

If anything goes wrong, I just log it to the console and carry on as if nothing had happened. The end user won’t be able to fix it anyway.

5. Pulling it all together.

I wanted the comment section to go on any post that has a mastdonId. These are rendered with a template called single.html. Here’s the tail end of that:

    ...
    <a href="{{ .Page.Next.Permalink }}">Next Post »</a>
    {{ end }}
  </div>
</div>
{{ end }}

<hr/>

{{ partial "comments.html" . }}

{{ end }}

As you can see, at the end of the single page, I add a horizontal rule and then load the comments.html partial. The dot there is to pass the post context, which is how we get .Params.mastdonId passed to the partial and eventually the JavaScript.

I’m pretty excited about this. Again, I’m writing this like it’s already done, but as I wrap up this post, then I’ll be able to test it all live. Crossing fingers…

2003 Blog in Hugo

kp@bit-101.com (Keith Peters) — Tue, 20 Jan 2026 23:02:41 -0500

My first version of this blog was put up in 2005. I think it was on the “Blogger” platform. And I think I moved to something else, then finally to Wordpress.

In 2017, I had over 1,200 posts and the database was getting creaky. Most of the content was Flash-based and I hadn’t touched Flash in years, so I started a new Wordpress site.

In late 2023, I was so fed up with Wordpress, I started this current version of the blog, as a static site, using Hugo.

Some time in 2024 or 2025 I took the 2017 blog and converted it into a Hugo site. That had about 140 posts in it. Not too bad.

I always intended to do the same to the 2003 blog. But as I said, it had over 1,200 posts. Sounded scary. But I sat down and got it done tonight. And it’s now live here:

https://bit-101.com/2003

Not much of the content is relevant anymore, and most of the links are broken. But I’ve had fun going through it again.

I also implemented search in all three versions of the blog, so it’s easy to find old content. That’s all.

We have search!

kp@bit-101.com (Keith Peters) — Mon, 19 Jan 2026 21:35:51 -0500

I love Hugo. I have negative regrets on switching over from Wordpress some years ago. But there are a couple things I’ve missed.

One is comments. Well, mixed feelings about that. I miss the interactivity. I do not miss the constant battle with spam.

The other is search. Hugo is a static site generator so there is no search functionality built in. Today I started looking around for how I could add it. There are a number of solutions. The first two I tried just didn’t work. Several required multiple JavaScript libraries and other files in order to do all the search on the client side. And you have to manually place all these files in the right place and then edit to them to configure things. And many of these seemed like hobby projects.

But eventually I found Pagefind. It seems pretty solid.

Lowdown on how it works:

After building your site, you run the pagefind executable, which builds an index of your site and adds it to your public site directory. It also automatically pushes the needed front end files into your site.

You need a small snippet somewhere on your site that will create the search UI, including the results. I just created a new page for that, with a link in the main menu to the search page.

It took a bit of back and forth to get it working, mainly due to my rustiness with Hugo itself. I think Pagefind itself worked fine right away.

Then you can configure various options on what parts of your site and what parts of each page to search, and what parts to ignore, adding some metadata to pages so it knows where to get the page title, etc. All of this was more just tweaking it to get all the right results and get them to look right.

For example, it was indexing my list pages - lists of posts and tags. I had to figure out how to exclude them. Wasn’t too tough. And it was showing a Mastdon logo on each post, which I guess was just the first image it found, somehow. So I turned off showing images in the results, which is fine. And it was showing “BIT-101” as the title for each page result, because it looks for the first H1 tag, which is the title of the whole site. So I had to tag the post title as being the title to index.

All seems to be working pretty well now. Anyway, no need to say more. It’s right up there in the menu. Try it out. I’m really happy to have this functionality back!

Palette Visualization

kp@bit-101.com (Keith Peters) — Sun, 18 Jan 2026 10:14:03 -0500

I’ve been doubling down on exploring color over the last couple of weeks. In my last post, I described a new app I created to extract a color palette from an image. One thing I mentioned there was that RGB is probably not the best color space to represent colors in when trying to cluster similar colors, as it was not designed with human perception in mind. In this post, I manage to show this visually and come up with a better solution that makes extracting palettes even better. And even to a fancy trick with visualizing palettes. This might be a long one, so strap in.

The Question

My palette extraction code was working pretty well. To recap, I was taking each pixel’s RGB values and converting them to 3D coordinates between -0.5 and 0.5 on each axis. The red channel was mapped to the x-axis, green to the y-axis and blue to z. With all the pixels plotted in that 3D RGB space, I applied k-means clustering to find however many clusters I wanted the palette to be. As I said, it was working pretty well, actually miles ahead of everything else I’d tried, but all too often it was missing colors I thought should be in there.

And I didn’t know exactly why. I had some theories:

That RGB was not a good color space to match human color perception.
That the math of clustering RGB colors tended to group colors that the human eye might see as much different.
That nearby contrasting colors would make us see certain colors differently or more strongly - an observation that wasn’t supported by the numbers.

In short though, I just didn’t know exactly what the algorithm was seeing when it looked at these points. But, hey, I’ve been rendering points in 3D for nearly three decades now. So why don’t I whip something up and take a look?

Visualizing RGB Space

As I said, I’ve been around the block on projecting 3D points so it wasn’t hard to put this together. I just had to take the RGB coordinates of each color (which ranged from -0.5 to 0.5 on each axis), scale them, apply some perspective, and then plot a point in that color. Once I had that down, I’d be able to rotate, translate, or scale them as much as I wanted, and create an animation. Though technically I considered this pretty simple, I was blown away by what it showed me. I’ll show off a few examples of some differently colored images to see how it visualizes things.

Some of the images I used a lot for testing were pictures of paintings done by two of my good friends, Todd Anderson (https://mstdn.social/@toddanderson) and Jeremy Osborn (https://mastodon.art/@jeremyosborn). They are both talented watercolor artists who post often on Mastodon, each with their own distinctive style and color choices, so it was really useful to see how the algorithm handled their artistic styles. I also grabbed a few images using my getimage tool, and generated some algorithmically as well. First, a painting by Todd.

Here, Todd uses palette with lots of browns and reds as well as some contrasting slate blue and grays. Taking all the pixels in this image and plotting them in a 3D RGB space gives us this:

Your browser does not support the video tag.

As described earlier, one corner of the cube is black, with the red, green and blue channels moving off on each side. The corner opposite to the black one would be full RGB, or white. I’ve discovered that this kind of distribution is common for visualizing images in the RGB space, a shape that starts at or near the black corner and sweeps up to the white corner, with the stronger colors forming arcs off to one or more of the sides.

Here, I’m also using the RGB color space to run the k-means clustering algorithm to extract an eight-color palette. In addition to showing the palette on the bottom of the image, I mapped the palette’s colors as circles in the RGB space as well. It does OK, but really misses that slate blue altogether, averaging it in with the brownish grays.

Let’s look at a painting by Jeremy with a very different palette.

Here we have lots of greens/yellow-greens, some grays and very light blue and a touch of red.

Your browser does not support the video tag.

Again we have that sweep from black to white. We pick up a lot of green variations, but just barely catch a hint of blue, and miss the reds altogether, even with an expanded palette of ten colors.

The next example is one I grabbed with getimage.

Anuja Tilj on Pexels

This has strong areas of green, dark reddish-purple, orange and yellow, with a lot of off-white/tan. How does this look in RGB space?

Your browser does not support the video tag.

Pretty big fail here. It focuses on finding a whole bunch of shades of brown and tan. The only bright color it really finds is a bit of that yellow. One of the problems here is the way that similar colors sweep along in an arc from dark to light. Because of this, the colors on one end of the arc are quite distant from colors on the other end, and will be averaged in with other closer colors. This is similar to the “two moons” data set, which is a known problem for k-means.

https://uva-applied-ml.readthedocs.io/en/latest/notebooks/6_kmeans_clust.html

You’d expect each arc to be a related cluster, but k-means works strictly on distance. There are other clustering algorithms that deal with this kind of problem better. But we still have something up our sleeves here.

But before we move on, let’s just see a random distribution of points in RGB space. Here I just filled every point in the image with a random RGB color.

And the visualization…

Your browser does not support the video tag.

Because everything is very smoothly distributed here, with just about every color represented, the algorithm does a pretty good job at picking out all the different colors and shades.

But what do we do about the failures in the more realistic images? Well, I keep saying that RGB isn’t a great space for this. Let’s try another space.

Visualizing in HSV

HSV is another way of describing colors. Think of the space as a cylinder. Going around the outside perimeter, from 0 to 360 degrees we have different hues, starting at red, going through orange and yellow, green, cyan, blue, violet and back to red at 360.

Saturation is how much of the color is there. A saturation of 0 gives you grayscale, where a full saturation of 1.0 (or 100%) gives you the brightest shade of the color possible.

And value is how dark it is. Value 0 will be black and value 1.0 (or 100%) will be bright.

In the HSV space, we can think of saturation as going from the center of the cylinder as saturation 0, to the outside edge with saturation 1. And value will be 0 at the bottom of the wheel and 1 at the top.

By HSV_color_solid_cylinder.png: SharkDderivative work: SharkD Talk - HSV_color_solid_cylinder.png, CC BY-SA 3.0,

Here’s a bunch of random color points in the HSV color space.

Your browser does not support the video tag.

You can see that colors near the center are grayscale, and the colors out at the edge and top are the brightest and most vibrant. The benefit of this color space over RGB is that similar colors are more likely to be close together. Let’s see how it looks.

First with Todd’s painting…

Your browser does not support the video tag.

This one kind of blew my mind. The pixel cloud forms a nearly flat plane slicing right through the middle of the HSV cylinder. To me this shows a nicely executed complementary color scheme, with the slate blue directly across from the reddish-oranges, but quite understated, just enough for contrast. It also shows a good range of values from dark to light.

In addition to visualizing the pixels in HSV space, I also changed the palette extraction code (the k-means clustering) to use the HSV space to find the clusters.

The new algorithm did a somewhat better job of getting to that slate blue. But I think the visualization shows that, numerically, there is not as much blue there as the eye sees in the painting. The contrast with the brown/orange makes the blue tint really stand out. But seen in the visualization, there’s really not a whole lot of blue there, and it’s not very saturated.

Let’s see Jeremy’s now…

Your browser does not support the video tag.

Much better performance here. We got the blue, and some of the dark red this time. I’m happy with this.

Also, it’s purely coincidental, but I find it interesting that the visualization almost looks like the scene it is representing. A bit more on that concept later!

And finally the eggs…

Your browser does not support the video tag.

A big win here! We got the red/purple, green, orange, and yellow, as well as a range of browns/tans.

All in all, I was very excited about the visualizer itself, and it helped me to see what was going on and improve the palette extraction method from RGB to HSV, which itself is now improved. As I said in the last post, this is never going to be “perfect”, but it’s better now than it was a week ago. I do still need to update the getpal tool to use the HSV method. I’ll do that soon, but wanted to get this post out.

Bonus Fancy Trick!

As I was working through the first versions of this tool, I posted some of the videos on Mastodon. Matilda responded with this post:

This concept hit me like a bolt of lightning. Within a fraction of a second I knew exactly how I would do this, and had already started writing the code in my head. Sadly, I was away at a company off site and didn’t have time to get to it for a while. I also wanted to clean up all the visualization and HSV palette code before embarking on a new project. But I finally got around to it a few days later. Here’s the walk-through.

Get a 3D point cloud model.

I was all set for this. In 2024 I created a library I called Wire. This did 3D wire-frame rendering and could load and render point clouds in xyz format. While working on that, I’d gathered a few point cloud models. One I used often was a human skull. It was composed of 593,934 3D points. The xyz format is very simple. Here’s a sample from the skull.xyz file I have:

443.25656128 439.81356812 1715.9786377
442.25656128 448.81356812 1715.9786377
439.25656128 459.81356812 1715.9786377
386.25656128 460.95645142 1716.12145996
438.25656128 461.81356812 1715.9786377
420.39944458 463.95645142 1716.12145996
437.25656128 464.81356812 1715.9786377
436.25656128 466.81356812 1715.9786377
374.39944458 469.81356812 1715.9786377
435.25656128 469.81356812 1715.9786377
375.39944458 471.81356812 1715.9786377
434.25656128 471.81356812 1715.9786377

It’s just one line per point, x, y and z coordinates separated by spaces.

Parse the point cloud into color values.

There are a couple of problems right off the bat. First there are way too many points in this model. Second, the values are not in a usable range. So I wrote a short program to open the file, read every 50th line and map the coordinates to be between -0.5 and 0.5 on each axis. Then, considering those were HSV color space coordinates, I was able to back port them to hue, saturation and value. Then convert that HSV color to an RGB color and save them all to a file. Here’s a small portion of that file, skull.pal:

...
#67d551,#64d2a0,#f1acb1,#8ad149,#f3a2f2,#5ad75b,#f4a2f1,#68d552,#b3c83d,#70d45b,
#67ceca,#bbc637,#7dd34a,#f3a8c9,#afc941,#cbc149,#98ce4d,#bdc63d,#6bd550,#deb97c,
#8cc5eb,#8bd152,#60d575,#aacb46,#e8b2a2,#beb7f4,#e7b2a1,#dfb87e,#eba6f7,#dfb97a,
#bfc639,#61d3a1,#76d458,#61d57a,#97cf42,#90d143,#76d45b,#79d44e,#e2b781,#60d580,
#b4bafa,#67d39a,#77d459,#a1cd52,#e7b39d,#c4c53a,#e8b39f,#f6a2f2,#5ad858,#f3a4f0,
#c3b6fa,#96cf54,#61d76c,#72d557,#ecb29a,#e7b49a,#8dd243,#a0ce40,#67cfd2,#e8b49b,
#d1c152,#a4cd53,#f4a9cf,#cdb3fc,#aacc49,#a6cd48,#dabd65,#70d568,#f2a8de,#68d764,
#e7b596,#b8bbf6,#77d551,#debb75,#64d85a,#76d55f,#e7b692,#b9bbf9,#bfb9f4,#eeafbf,
#ecb0bc,#84d44a,#cec256,#8bc9e2,#f2a6f7,#6cd76e,#e0adfa,#6ad58b,#c2c741,#9ec3f0,
#cbc44f,#d8bf6a,#7eccdf,#7ad566,#5ed965,#6ad4a4,#64d872,#85d549,#95d24f,#67d4aa,
#cfb5f4,#64d4b0,#61d2cf,#66d0d9,#7bcce4,#c5b9f3,#edb1bf,#96d251,#9bc5f1,#76d762,
#62d4b6,#9dd154,#67d78d,#5dd3cb,#aace4c,#cab7fc,#7ccde0,#a0d145,#83d55c,#70d865,
#61d4bc,#eab790,#eab6a2,#83d560,#e2bb88,#75cfd8,#70d86a, ...

This is a list of with 11,645 color values.

Generate an image with those colors.

Now those colors were generated from 3D points in the HSV color space, so if I plotted them all in the visualization tool, they would convert back into the original x, y, z points and form a 3D image of a skull!

So I created a new image and for every pixel in that image I randomly chose one of those 11,645 values to color that pixel with. This resulted in an image like this:

Not very interesting though. So I sorted all the pixels by hue, saturation and value and got this:

Better, but still a bit bland. What if I warped it a bit with some Simplex noise? In this case I let the noise wrap, so that if a pixel is pushed off one edge, it returns to the opposite side of the image. Now we have:

Now that’s nice and funky and should still have roughly the same random pixels from the skull palette. Let’s feed it into the visualizer and…

Your browser does not support the video tag.

How crazy is that?

For one more test, I generated 10,000 3D points using a bit of math to make a 3D shape. I converted those points to a palette, sorted and warped it just like the point cloud skull. It looks like this:

Now, if you look at that, it looks pretty damn similar to the image generated with the skull palette. But the colors are slightly different. When you run this through the visualizer, you get:

Your browser does not support the video tag.

I could go on, and may post some more examples later, but you get the point.

Is this important or useful? Probably not. But I think it’s cool as hell and I’ll always play with stuff like this.

Thanks

Shout out for Jeremy and Todd for being nice enough to let me mess around with their paintings. And for Matilda for the spectacular idea. All three are worthy follows on Mastodon.

Getpal - extract a color palette from an image

kp@bit-101.com (Keith Peters) — Sun, 11 Jan 2026 09:16:29 -0500

At the end of my post about color palettes I talked about extracting palettes from images. I said there that I’d gone through a few different strategies but wasn’t really satisfied with any of them. And I said that I wanted to try k-means clustering to see if that would have better results. I implemented k-means a few months back on 2D points and got it working pretty well, but for some reason I thought that doing it with colors would be a lot more complicated. Turns out, it was pretty easy to adapt the 2D clustering code to 3D - where the 3 dimensions were red, green and blue.

Light intro to k-means

The k-means algorithm helps you find “clusters” in data sets. For example, say you have a set of coordinates where crimes have happened over a city or state. You can plot them on a map and maybe get a visual idea of where the worst crime locations are. There might be three or four hot spots. k-means is just an algorithmic way of doing that.

Say you have the following points mapped out.

You can visually see that there are clumps or clusters where there are more points, but how would you go about identifying those? If you’re not familiar with k-means, think about how you might do that for a bit.

k-means gives us the following 16 clusters, each color coded:

Here, I’ve drawn a circle at the center of each cluster. Each point is assigned to the cluster where it is closest to the center. That’s where the name comes from. Each cluster is formed by an average or “mean”, and there are a specific number of them - k. So you have k means.

Now, you might not agree with some of those clusters. For example, that yellow cluster on the right edge, you might be thinking that should be three separate clusters. The thing is, k-means requires you to set a value for how many clusters you want to sort your data into. I chose 16 and it did its best with that. If I specify 28 clusters, we get this:

Now, it got three different clusters for those ones on the right, but maybe there are others where you feel that two or more clusters could be joined. The takeaway is that the algorithm is very dependent on the initial number of clusters you specify and you may not always get the results you expect.

There are more advanced features that people have added to the algorithm over the years, but this is just the original basic algorithm.

How it works

To start with, you create k means, i.e. you create a specific number of “centroids” (in this case 2D points), which will eventually become the centers of each cluster. So again, you have to choose how many means you want. For this discussion, let’s say we choose five. So we create five points to serve as centroids.

Then, we have to decide how to create these centroids. Where do we put them? Well, there are various strategies for that. The simplest is to just create random points that will fit within the range of the data. That generally works surprisingly well, though there can be some differences in the outcomes on different runs. It can also have some other problems we’ll explore later. But often it’s a good enough strategy. Other strategies involve some randomness but making sure that your centroids are spread around the data set and not randomly clumped together.

Now that you have your five centroids, you loop through all the data points and assign each one to one of five clusters based on the five centroids. You find out which centroid each point is closest to and assign it to the cluster associated with that centroid.

Now you have five clusters, but they are probably not optimal, especially if you chose random centroids. So, you go through each cluster and find the mean location of each one of them - the average location of all the points. In most cases, this new mean will be different than the existing centroid. So you take those five new means and make them the new centroids for five new clusters. And you loop through the points again and assign them. You’ll likely have different assignments.

And you repeat that over and over - assign the points, get the new means, assign the points, etc. - until the new means are the same as the previous means. This means things have stabilized and you now have your final clusters.

Applying this to color

As I said, this was much simpler than I expected.

First I gather all the pixels in the image. Each one has red, green and blue values from 0.0 to 1.0. Maybe you want to use 0 to 255. But these three values let you think of a color as a point in 3D space.

Then you decide how many colors you want in your palette. Again, lets go with five. You create five centroids, which themselves are color values.

Now you loop through all your pixels and find out which centroid they are closest to. This again is Euclidian difference. You take each pixel and get its difference to the centroid on the red, green and blue “axes”, call them dr, dg, db. Then it’s just sqrt(dr * dr + dg * dg + db * db). (See note on this later.)

Then you get the average color of all the pixels in each cluster. Again, this is adding up all their reds, greens and blues and dividing each by the number of pixels in that cluster. Those are your new means and you repeat until the means don’t change.

Now, each of those means/centroids are an average of the pixels in that cluster, meaning that list of centroids is your sampled palette!

Say we start with this image:

Photo by Ella Olsson: https://www.pexels.com/photo/top-view-of-food-1640772/

Running through this algorithm, we get the palette shown on the bottom of the image:

I think this has done a pretty good job. It’s got the dark brown, then the mid brown with a hint of green, the reddish orange, a lighter brown and an off-white. It didn’t really grab the green in this case, but if we up the palette to a count of six, we get it.

Interestingly, that second mid-range brown now seems to have less of a green tint. The green pixels got pulled into their own cluster, leaving that one just plain brown.

The tool and all the pieces

I built a standalone tool that you can run as a command line app. You can find it here: https://codeberg.org/bit101/getpal. There are binaries for MacOS and Linux, though I think you’ll need to install the cairographics library for your platform as well.

But before I built the tool, I built all the code to let me do this within anything I happen to be coding at the time. Here are the pieces that go into it:

A function to get a list of all the pixels in an image. I can do that by loading an external image or grabbing the pixels from the drawing context of an image I am working on. This just returns an array with an RGB value for each pixel. For an 800x800 picture, this is an array with 160,000 elements. For a much larger image, this could get out of hand. So the function lets me set a sampling resolution. If I set that to 4, it will sample every fourth row and column, in other words, one pixel for every sixteen. This speeds things up a lot but still usually gives you a good representation of the color values. The code for all of this is here: https://codeberg.org/bit101/blcairo/src/branch/main/context_histogram.go
The k-means function itself. This takes the array of pixels and a k value - how many colors you want in the palette. This returns the palette as an array of color values, to use as you will. If you want to gaze at the code, it’s here: https://codeberg.org/bit101/bitlib/src/branch/main/blcolor/kmeans.go

There are some nuances though…

Choosing the initial centroids

Initially I was, as described earlier, just creating random color values for the initial centroids. This had mixed (and not very stable) results. You could get different palettes each time you ran the tool. Usually pretty close, but there would be differences. Sometimes things would be way off. I ran across an example that showed very clearly how random centroid initialization can get you into trouble, and reproduced it here.

Say you have a random set of values that are naturally clustered together like so:

And say you wanted to extract two clusters from this. You would expect the algorithm to group them as so:

But say your initial two centroids were placed near the center like this:

Then k-means would return the clusters defined like this:

That looks ridiculous, but all the points in the top cluster are closest to the top centroid and all the ones in the bottom are closest to that centroid, so it’s stable and correct! There are other configurations which would cause other similar problems.

So random centroid initialization can give you random problems. And I would see some odd results now and then when using random RGB initial centroids. In fact, it’s surprising how well it worked most of the time with random initialization. But I wanted to see if I could do better.

Ideally you’d want your initial centroids spread throughout the data. Of course, they’ll change and move around as the algorithm iterates, but you tend to get better results if they are spread out to start with.

I came up with a seemingly well-working strategy using HSV. If I want five colors in my palette, I initialize the centroids with HSV colors where the hue ranges from 0 to 360, like 0, 72, 144, 216, 288. Since a hue of 0 is the same as 360, this ensures that each centroid’s hue is 72 degrees from any other one. Of course, this is done mathematically based on the number of palette colors specified. I used a saturation and value of 0.5 for each color as well. That just seemed to work the best.

I’m pretty happy with how this worked out. It seems to capture those less represented colors, like the green in the above photo, better than the random method for the most part.

Choosing the final color for each cluster

You can use the color value of the centroid as final palette color for that cluster, which is what I did at first. It generally works OK, but realize that is not necessarily a color that is actually in the original image. It’s an average of all the colors in that cluster. If you have a bunch of colors around the centroid, but none exactly near it, you could wind up with a color that doesn’t seem to fit at all. So I gave myself three options here:

Take the average as described above. This is deterministic, meaning you should get the same color each time.
Take the pixel from that cluster that is closest to the centroid. This is guaranteed to be an actual color that exists in the image, and again is deterministic.
Take a random pixel from that cluster. This is guaranteed to be a color in the image, but is non-deterministic.

The tool itself

When you run the tool, you pass it the path to an image it will spit out a comma-separated list of color values. You can also specify the number of colors in the palette with the -n flag. This defaults to 5.

You can use the -s or --swatch flag passing it the name of a new png to create, which will visually show you the palette it extracted. Running getpal -n 8 -s palette.png for example, generates this image:

By default, it will sample at a resolution of 4 as described above. You can change this with the -r flag. But pay attention to the fact that setting this to 10 for example would not sample every tenth pixel, but every tenth row and column, so every one-hundredth pixel, which is starting to get into the realm of missing some color values, for a smaller image at least.

Finally, the -m or -method flag lets you specify the method of choosing the final color for each cluster as just described, the values are “average”, “closest” and “random”, with “average” being the default.

Caveats and tips

As analytical as all of this is, it’s not hard science. Some things to realize:

As seen in the photo example above, and as described earlier, the number of colors you specify can have a very big effect on the palette you get. If you’re hoping to get a particular color in the palette, you might have to specify a larger palette and manually cull some of the colors you don’t want.
The method argument can play a big part in fine tuning the palette you want.
RGB is useful way of mathematically specifying color values, but it does not map perfectly onto human color perception. We see some colors as brighter or dimmer than others even when they have the same value. Some colors stand out to us more than others. The contrast between neighboring colors can change our perception of a given area of color. Like in one of my favorite optical illusions.

The brown square in the center of the top face of the cube and the yellowish square in the center of its left side are actually the same color! You might be frustrated trying to find that yellow in the palette, and only seeing the brown emerge.

There are other color spaces that attempt to adjust for these perceptual differences. To keep things simple, I did not implement any of those. But to help a little bit, I use a distance function that calculates the perceptual distance between two colors, rather than the raw RGB distance. I don’t know how much it actually helps in the long run, but I left it in there.

Summary

In no way is this meant to be a definitive “these are the exact five colors that most represent this image” type of tool. I don’t even know if such a thing is possible. But it does pretty damn well, in my estimation, on grabbing a number of colors out of the image that seem to fit. Far, far better than all the other methods I’ve tried. Use it more as an inspirational tool, grab more colors than you need and choose the ones you like the best, play around with it.

Palettes

kp@bit-101.com (Keith Peters) — Sun, 04 Jan 2026 12:35:51 -0500

Color is hard

Most of what I do is in black and white. I love the way it looks. I like building up forms and shapes and patterns using only values.

Also… color is hard.

Or, it seems hard. Working with color reminds me of composing music. There are so many rules, and it’s all based on physics, and also all based on human perception, and also psychology - though that last one is a bit questionable.

But I’m trying to learn new things, so I did a deep dive into colors and palettes recently.

My bitlib coding library already had a pretty rich set of color functions. I’ve coded up color creation in RGB with floats (0-1) or ints (0-255), HSL, HSV, CMYK, all named CSS colors, functions for working with brightness and contrast and mapping new colors to an existing image.

But whenever I did use color I’d most often just use my RandomRGB function to generate random colors. It mostly ended up looking like crap.

Occasionally I’ll spread things out across a spectrum, like this:

Or, if I’m feeling really adventurous, I’ll use HSV with a single hue and random saturations and hues for a monochrome composition

That’s kind of cool, but I’ve always felt like I’m missing something.

I even wrote a color palette module a while back. But never did anything with it. Mostly because it had such limited functionality. You could create a palette and then manually add colors to it. Then… use those colors … somehow, later. Basically it was just an array of color values. The only real useful thing I did with it was let you map a palette to an image. So all the dark colors would map to the first few colors in the palette, and the lighter ones to colors at the end of the list. This was pretty fun because I could make a grayscale image and then map any palette to it.

Like, I could take this picture of my dog…

Blur it, grayscale it, and map a monochrome palette to it and get this:

But creating a palette was a pain. So I still never used it. So I decided to do some work on it.

Creating a useful palette module

I started doing some research on color palettes, or color themes or color schemes. All basically the same thing - a limited set of color values that supposedly work well together. A lot of this I already knew, but for the first time tried to implement some of it in code.

So over the course of a day or two I sat down and wrote a bunch of palette creation methods.

Monochrome

The first one I did was the monochrome palette. This was similar to what I did above. But in the above I was choosing a new random color for each triangle I drew. The interesting thing about a palette is that it’s usually more limited. A palette of two colors is totally valid. Often they only contain a handful of colors. A couple dozen might be a lot, but it depends on what you’re doing I guess.

Anyway, my method takes a hue and the number of colors you want in the palette. Rather than randomly creating color values, it spreads them out across a range of saturation and values. In pseudocode…

function MonoPalette(hue, n) {
	pal := NewPalette()
	for (i = 0; i < n; i++) {
		sat = i / (n - 1)
		val = 1.0 - s
		// avoid 0 saturation or luminance, which is just grayscale
		sat = map(sat, 0, 1, 0.4, 1)
		val = map(val, 0, 1, 0.2, 1)
		pal.Add(HSV(hue, sat, val))
	}
	return pal
}

I create a new palette and loop through n times.

Saturation is i / (n - 1) which gets it to range fully from 0 to 1, and value will spread oppositely from 1 to 0. But when I had that, I found that the zero saturations were just white, and the zero values were just black. So I used a map function to map those values to keep them away from zero. The values I ended up with were through trial and error and what I thought looked good. Most of the other palette methods I made were pretty similar to this: create a palette, calculate a number of colors, add them to the palette, return the palette.

Here’s a monochrome palette with six colors.

Although this uses the same hue as the version above, it feels more monochrome-y to me. But that’s what’s great about coding your own library. You can do whatever feels right.

Achromatic

After that it was easy to create an achromatic palette. In other words, grayscale. In the past I would have just chosen random values of gray for each triangle, like so:

And this looks fine. But here, I just use a 5-color achromatic palette:

I’m not claiming it looks better, but it’s different. Fewer shades, more contrast, cripser. And now I have control. I can make a straight up black and white palette, or one with as many shades as I need for something to look the way I want.

I won’t show pseudocode for all these examples (the actual Go code is linked above). But this was pretty similar to the technique of the monochrome. Iterate n times across gray scale values from 0 to 1 and add each one of those colors to the palette.

Color Theory Cognitive Dissonance

Next I went for the common color-wheel-based palettes you read about in color theory. I might have a bit of a rant here, so be forewarned.

These palettes include complementary, split-complementary, triadic, tetradic, and a few others. They are defined by the angles between the various colors within the palette, as seen on the color wheel.

Complementary colors are those right across from each other on the wheel, or 180 degrees.

Split-complementary means you go across, and then choose the two colors 30 degrees to either side of that. So 150 and 210 degrees.

In in a triadic palette, you choose three colors that are 120 degrees from each other - they form an equilateral triangle. And tetradic contains four colors 90 degrees apart, forming a square.

Now in any text about color theory, you’ll read all about why these particular arrangements are so important. About harmony and tension, contrast and resolution. Some might go into wavelengths, others into psychology, how the various colors, by being in those exact relationships and angles with each other set up particular feelings and emotions in the brain.

But here’s the thing. Most of those texts are referring to the RYB color wheel. That’s the one you learned in elementary school. Red, yellow, blue as primary colors. Mix them to get orange, green, and purple, the secondary colors. Mix again to get the tertiary colors, which I won’t show here.

Each of these colors is 60 degrees apart from its neighbor on the wheel.

So in this color wheel, red and green would be complementary. As would yellow and purple, and blue and orange.

Triadic colors would be red, yellow, blue, or purple orange green. Each one of the colors in each triad is 120 degrees from its neighbors.

Of course there are a huge number of other triadic and complementary palettes. But since my wheel only shows six colors, that’s what’s I’ll mention.

You’d need to have the tertiary colors to get split complementary and tetradic.

All good so far.

But then we get computers. And RGB and HSL/HSV. And we get the RGB color wheel.

With this wheel, the primary colors are red, green, blue, and secondary are cyan, yellow and magenta. These are also the two triadic palettes you can get with just these six colors. Complementary colors are red and cyan, yellow and blue, magenta and green.

Now my problem is not that they are different. That’s fine. Different models, different paradigms, different technologies, additive vs subtractive, etc. I get it. All good.

My problem is that artists have gone on for years about, for example, the importance of using complementary colors and how important they are and the effects they have on the viewer. And they’re talking about, say, red and green. And now we come along with our RGB and people are still talking about complementary palettes and how these two very specific colors are going to affect your brain. But they’re talking about red and cyan instead of red and green.

I actually saw a site that went over all of this, complementary, triadic, tetradic, split, analogous and all the reasons to use each one, showing them on a static RYB color wheel. Then said, click here for an interactive color wheel you can explore these on. And it went to an RGB color wheel!

In short, I agree with the general ideas of these color palettes - it can be good to have two very different colors, or two similar colors and one different, or three or four fairly different colors. But to say that these exact angles on the color wheel magically ignite our psyches into feeling a certain way is a bit of a stretch, especially when those angles give you totally different colors depending on which wheel you are using.

Anyway, end of rant. On with the show!

More color palettes!

For a bunch of these, HSV or HSL is going to be your friend. Especially if the hue argument is set up to take values from 0 to 360. And even better if these will wrap around, so 380 is really 20, and -30 is the same as 330.

For complementary, you take one color, convert it to HSV, add 180 to the H and create your second color.

For split-complementary, you take the original color, H + 150, and H + 210.

For triadic it’s original, H + 120, H + 240.

And tetradic is original, H + 90, H + 180, H + 270.

You can see these all start out with the same orangey color and have other colors added to them. In these cases, I’m just choosing a random palette entry for each triangle, so it gets a bit chaotic. Often the advice is to choose one or two colors as the main colors and use the others sparingly for contrast. So you might get things like…

In each case, I left the orange color as the contrast. The tetradic was the tougher one because all the colors differ significantly. But these are contrived examples anyway. In a real design you might have the background one color, foreground another. Maybe some text in a very contrasting color to pull your attention to it. Whatever.

A few more!

There’s an analogous palette where you take the original, H + 30, and H - 30.

And then there’s a spectral palette, which doesn’t mean ghostly, just that it spreads across a spectrum. In this case, hues of 60 to 120, which kind of does wind up a bit ghostly.

This is pretty similar to an analogous palette, but you can spread it out more than just 30 degrees and add as many colors as you want.

Finally, I wanted to make a pastel palette. This is quite similar to the spectral one, in that it spreads across a range of two hues, in this case 0 to 60, but it give each value a randomly low saturation.

Both of these last two use the same technique as monochromatic and achromatic - interpolating between the two hues n times, and adding a color with that hue to the palette.

Other palettes

Earlier I had a bit of a rant about the over-belief in these exact angles and combinations of colors for palettes, but they’re a decent rule of thumb. However, if you go onto one of the many color palette websites out there they’ll dump thousands of potential palettes on you, with fancy names like “Olive Garden Feast” or “Pastel Dreamland Adventure” (I did not make those up!) I’m sure most of these aren’t strictly related to any of the classic palette patterns above. They’re simply combinations of colors that people thought looked good. And yeah, most of them do look pretty good. Generally there are some similar colors and one or two contrasting ones. pal

So maybe I want to use one of these palettes in my library. Well often you can export a specific palette as a list of values. For example, https://coolors.co lets you copy a given palette as an array like so:

["#cdb4db","#ffc8dd","#ffafcc","#bde0fe","#a2d2ff"]

This, by the way is the infamous “Pastel Dreamland Adventure”!

On https://colorhunt.co you can also select and copy a palette pretty similarly. Other sites make it a bit more complicated but you can generally get a similar list, even if you have to copy and paste each value individually.

So I made a function where you can just pass a list of strings you can just paste in and get back a palette, like so:

palette = PaletteFromCSS("#cdb4db","#ffc8dd","#ffafcc","#bde0fe","#a2d2ff")

And Now I can have my own pastel dreamland adventure!

I also made functions that will save a given palette to disk in that exact format, and read a palette back in. So I have all those bases covered.

Palettes from images?

A oft-used creative way to come up with a palette is to sample colors from an image. So I did that. I made a function that you can pass a path to an image and a number of colors and it gives you back a palette, like so:

palette = SampleImagePalette("image.png", 5)

I grabbed this image (using my getimage app!) from pexels.com

Photo by Sergei Starostin: https://www.pexels.com/photo/vibrant-tropical-poke-bowl-with-green-smoothie-34902493/

And used that palette to create our usual triangle fill…

I’m not entirely satisfied with this function yet. I tried a bunch of different techniques. There are a lot of strategies to do this. Just randomly taking n number of colors from the image - but you might end up with very similar ones. I even did some work on measuring the difference between colors and making sure I didn’t have two that were too close together. Other ideas are color quantization - reducing the number of colors in the image, and creating a histogram and using colors from that. Nothing so far has really been great. So consider this a work in progress.

What I want to try is k-means clustering. You’d basically plot each color in a 3D space and use the k-means algorithm to find n clusters which should give you the main colors of the image. Watch for that some time this year, if I get around to it.

Summary

Well, I think I know a bit more about color now. And I definitely have created more than enough tools for myself to explore this area. So maybe I’ll be creating a few more colorful images now… maybe?

Print Shop Is Live

kp@bit-101.com (Keith Peters) — Thu, 01 Jan 2026 12:56:52 -0500

Well, I’ve gone and done it. I opened a shop to sell prints of some of my digital creations. And here it is:

Keith Peters Art Shop

Yes, I went with Etsy to begin with. It’s at least something I’m familiar with. I’ll be looking at other sites that are more specific to selling prints.

I have to say that creating the art was the easy part so far. Setting up the shop, all the financial linkages, figuring out pricing, shipping, return policies, descriptions, integration with the printer, etc. - that all was a lot of work. But now that all that is done, it should be pretty easily repeatable with future runs.

Speaking of future runs, I’ve got a new line I’m working on now that I’m very excited about. Here’s a taste…

These began with the same technique I used in the Symmetry Web series, but I let the points wander, rather than be completely random. This creates amazing organic forms. Each one also features a ring where the colors are inverted. I look forward to sharing more of these soon.

Getimage

kp@bit-101.com (Keith Peters) — Wed, 31 Dec 2025 13:16:47 -0500

Problem Statement

Sometimes when I am developing a program or technique, I need an image to load as a background or to do some image processing on, etc. It usually doesn’t matter what the image is, I just need it for testing.

My usual workflow up to now:

Do an image search for something like “cats”.
Find an acceptable image that is neither tiny or massive and roughly the same aspect ratio as I need.
Save it, giving it a usable name like “image.png”, instead of “GettyImages-1208220393-e1612811442160.jpg”
Open it up in an image editor, or use imagemagick to:
Resize it to some usable dimensions. I like 800x800 as that’s the size of my canvas in my creating coding template project. But sometimes I want a different size. Scaling in imagemagick is easy, cropping is a bit more complicated, so I usually open up Gimp.
If the image is not a PNG, convert it to PNG, as the graphics framework I use only loads PNGs.

First world problems, I know. But we can do better for this kind of repetitive task.

Some solutions…

Direct Random Image Download

I looked around for random image APIs. The category of tool I wanted was something where I could hit an API endpoint and the result would be a random photo, not a JSON list of images and attributes.

There are a few. Surprisingly few.

The most promising was PicSum - does exactly what I want. Specify a width and height and it serves up a random image. And it’s free. Sadly, they are all stupid AI generated garbage. Not that it matters for my purpose, but it really annoyed me to look at those things.

I then found that if you use the same API endpoints with picsum.photo it will serve up an actual photo. But it does some redirects to another site that took some getting around. I got it working with a simple shell script. If you want to try that, here it is:

#!/bin/sh
TEMP_FILE="$(mktemp)"
if [[ -z $1 ]]
then 
curl -L https://picsum.photos/800 > $TEMP_FILE
elif [[ -z $2 ]]
then 
curl -L https://picsum.photos/$1 > $TEMP_FILE
else
curl -L https://picsum.photos/$1/$2 > $TEMP_FILE
fi

magick $TEMP_FILE img.png
rm $TEMP_FILE

It uses imagemagick to rename and convert the image to a PNG.

There were a few other similar services, but most of what I tried was either a paid service or broken.

PlaceCats seems workable, if you just want cats. :)

Image Search Services

Then there are other image services. Unsplash and Pexels are two that I’ve used in the past extensively - but only through the UI, not API. But both have APIs. And there are plenty of others.

At this point, I had ideas in my head that I wanted to build a command line app in Go to do this, so I dug into Pexel’s API.

https://www.pexels.com/api/

And went about creating an app.

The getimage app

I created an API key, but to be honest, I initially forgot to use it in my request and it worked fine, so… ? Anyway, I fixed the code and am using the API key now.

They do have rate limiting of 200 requests an hour, or 20,000 requests a month. And there’s a header to see how many requests you have left. But this header was reporting -1 when I was not using the API key. The header did not exist at all when I was using the key. Again, not sure what’s going on with that part of it, but I’m not trying to get away with anything and I don’t think I’ll ever hit up against those limits anyway.

You have to do a search first, using some search term. This will give you a list of results. I programmed in a list of likely search terms:

animals
art
beach
buildings
car
cat
china
city
dog
fashion
fire
flower
food
forest
graffiti
holiday
horse
house
japan
landscape
plants
restaurant
shop
street
woman
woods

When you run the program it just randomly chooses one of those.

You get back a page of results that includes 15 images by default. You can request a larger page size, but that works for me. I then randomly choose one of those results.

Each image has a number of different version (sizes). Generally like:

original
large2x
large
medium
small
portrait
landscape
tiny

I found that large2x were generally the right size for me. At least 900-1000px in each dimension. So I pick that one out. The original size was way too large.

I then load that image using the URL supplied in the JSON.

I then shell out to imagemagick to do the resizing and conversion. So you do need to have imagemagick installed. There is an imagemagick library with Go bindings I could use to make this more self-contained. And maybe I’ll look into that eventually.

Resizing was the trickiest part, but I worked it out with a two-pass system. I make one call to scale the original image to fill the entire space of the requested size. This usually leaves the image too large in one dimension. A second imagemagick call crops it down to the right size. I use a “gravity” of “center” when cropping, which usually keeps the main subject of the image, rather than cropping half of it out.

Making it better

Up do now this worked perfectly for my needs. I’d get a random image sized to 800x800, converted to PNG and saved as “image.png” just by calling getimage.

But I can do better than that. What is a command line app without parameters?

The size was the first part. Pretty easy. Width and height were hardcoded as 800x800. I just added command line params -w and -h. They’re optional. Both default to 800.

I also added a -g param for geometry. This is just a shorcut to -w and -h. If you want a 400x400 image, just say getimage -g 400x400. Geometry will override the width and height settings if you use both.

Then image name. I was defaulting to “image.png” but added a -o parameter for the output name. I started adding logic to allow you to specify different file types, such as gif, jpg, etc. But I realized that since I’m running it through imagemagick, the extension on the output name specifies the output format as well. Free functionality!

Next was search term. Added an -s param to allow you to search for, say “tigers” or “airplanes” or whatever you want, rather than the random search I set up. I can’t guarantee you’ll get a result, but it’ll try.

Then at one point I wanted to grab a specific image I saw somewhere and was annoyed that I’d have to download it, name it, resize it, convert it. So I added a -u param. This lets you specify a URL directly to an image. If you specify this, it will bypass the search functionality, attempt to download the given image, then do the resizing, renaming, converting.

To summarize:

Usage of getimage:
  -g string
    	geometry of image as widthxheight
  -h int
    	height of image (default 800)
  -o string
    	the name (and file type) to save the image as (default "image.png")
  -s string
    	term to search for
  -u string
    	a direct URL to an image - will bypass search, downoad and resize image
  -w int
    	width of image (default 800)

I’m pretty damn happy with this. Even in just a few days, I’ve used this regularly.

Final words

If you use this, be responsible. Get an API key and don’t abuse it with too many requests. Also, look at Pexel’s attribution rules. If you wind up using the images for anything other than test fodder, follow those rules.

I think this would be pretty easy to adapt to use other services such as Unsplash. Looking over their API, it would only require very minor changes in the JSON parsing. Maybe I’ll make that an option at some point.

I’ve also thought about making a config file where you can specify your own defaults, search terms, etc. Provider could be part of that.

Source and a bit more info here:

https://codeberg.org/bit101/getimage

Installation

If you have Go installed on your system, it’s as easy as:

go install codeberg.org/bit101/getimage@latest

You’ll also have to set the environmental variable PEXELS_API_KEY with your actual API key value.

And, as mentioned, installing imagemagick is a must.

Tested on Linux, MacOS, and even Windows!

Contours

kp@bit-101.com (Keith Peters) — Wed, 31 Dec 2025 08:36:19 -0500

Background

I always have different ideas in the back of my head - techniques that I am aware of and want to implement someday but haven’t gotten around to. Drawing contour lines on some kind of height field has been on that list of ideas for at least a couple of years now. I’ve made some attempts to figure it out on my own, with some level of success, but never really accomplished anything worth talking about.

Just so we’re on the same page, here’s what I’m talking about:

You have some kind of 2D field data. It could be elevation data from a government survey, or satellite imagery, or a Perlin/Simplex noise field, a grayscale image meant to encode height/distance, or some other kind of algorithmically generated data. Let’s say this is in a 2D array of floats, so you can get any value by saying data[x][y]. If this is noise data, it could be directly rendered like this:

This was rendered with a resolution of 10 pixels, which is why it’s blocky. Good enough for our purposes though. Although this is rendered visually, think of it as a 2D array of values, like this:

In this case, I have it set so that each value in the array is from 0 to 255, but it can be whatever you want.

What we want to do is draw contour lines for all the data points at a certain “height” like you’d see on a topographic map. Like this:

The Algorithm

A couple of weeks ago I was looking over the links on Mary Holstege’s site (a great resource by the way) and ran across this post on contours which led me to Paul Bourke’s CONREC post.

Side note: if you’re not aware of Paul Bourke, it’s worth diving deep into his site. So much great stuff there.

Anyway, after reading up on the technique, which I generally understand, I ported Bourke’s C code to Go. There was actually a sample Go implemention listed there, but I always like to write stuff myself and make it my own rather than just importing libraries when possible. I wouldn’t do that on a job necessarily, but that’s my strategy for my creative coding efforts.

I’m not going to go in depth into the algorithm itself. But in short, it divides the field into triangles and for each triangle checks how many points are above, below, or on the current contour level being drawn. Based on that, it can generate two points that serve as one tiny segment of the contour line. When you draw all of those segments, you get the contour lines as seen above.

And you have to do this for each contour level you want to map. For example, here is the single contour for a value of 128, middle gray in the noise field:

And here it is for the value of 200, which is a much lighter area:

The CONREC formula has you pass an array of contour values that it will iterate over. In the first contour image above, I passed an array of 10 values from 0 to 255. Here it is with 20 contour levels.

So that’s the basic algorithm. Now I’ll go into a bit more of what I added to it.

My Take

My Go implementation of CONREC was mostly a straight port of the C code. I did make some changes to make it more Go idiomatic, and tried to simplify a few aspects.

MakeLevels

One of the first things I did was to make a helper function for creating the levels array. So I can call makeLevels(count, min, max) and it will generate an array of the given size with values in the given range. For example, running makeLevels(10, 0, 255) which is what created the first contour image above, results in the array [0 28.333333333333332 56.666666666666664 85 113.33333333333333 141.66666666666669 170 198.33333333333334 226.66666666666666 255].

Segments to Paths

This all worked fine, but one thing I was not thrilled about was the fact that each of these segments was drawn all by itself. They were rendered in grid order, and by contour level order, so there was no concept of one of those contour loops being a single entity. I wanted to turn this into a list of paths, where each path was a list of 2D points I could then connect, maybe changing the color or thickness of, transforming in some way as needed. That wasn’t possible in the code as given.

So I changed it up so that instead of drawing each segment as soon as it was found, I created segment objects composed of two 2D points. So that for each contour level, there was a list of segment objects relating to that level.

But these were still unordered lists of segments, not continuous paths. The next part was the biggest challenge, and the most rewarding.

For each contour level, I had a list of segments that I wanted to turn into a list of paths. For example, looking again at contour level 128,

… you can see that there are six discrete paths there. Each one starts and ends on one edge of the canvas. For contour level 200,

… there are eight paths. Some of them intersect and edge, some just loop back to themselves, a couple almost join (or maybe there’s three there… or maybe they overlap. I’m not sure.)

What I ended up doing:

Take the first segment off the list and add its two points to a new path.
Loop through the rest of the segments in the list.
For each segment, if one of its points was equal to the last point in the current path, append the OTHER point to the end of the path.
Or, if one of its points was equal to the first point in the current path, prepend the OTHER point to the start of the path.
In either case, remove that segment from the segment list.
Keep doing this until you find no more matches.
Save the current path to list of paths, and repeat starting at 1 above.
When the segment list is empty, you now have a list of sequential paths.

Simplifying Paths

As is, the algorithm generates a whole lot of segments, which means each path contains a whole lot of points. For the above example with 10 contour levels, there were a total of 5,794 points in all the paths. That’s not horrible, but a 10-pixel resolution is kind of chunky. As you increase the resolution, the number of points explodes. And most of those points are very, very close together, or almost directly in the same line, so they are not adding a lot of value.

Fortunately, I had just tackled this problem! See my post from just a few weeks ago, Simplifying Paths. So this was a no brainer. Once I created each path as just described, I ran autosimplify on it. This brought the total number of points in the example down from 5,794 to 1,812. And in some cases it actually looks better, because sometimes when you draw a path through very close points you can get some artifacting.

Levels or Paths

In many cases, all I wanted to do was draw each path as shown above. But I might want to do something different for the set of paths in each level. Maybe color them differently. To accomodate both of these needs, the return value from my contour function returns an object with two properties. One is a flat list of all the paths. I can loop through and draw them all as per above. The other one is an array of levels. Within each level element is the list of paths for that level. So I can now do something different per level now. Here, I colored each one differently:

Here, all the orange lines are at low levels. The dark blues are at higher levels.

Or, I can generate elevation markers like you’d see on an actual topographic map.

At this point, I feel like I should remind the reader that we’ve been working in 10-pixel resolution this whole time, and as mentioned, it’s a bit chunky, but good enough for a demo. But you can change that resolution and make everything much smoother:

Generating Height Data

As described, this method takes in a 2D array of floats representing height data. But how do you generate that data? Obviously, you can generate it algorithmically using Perlin or Simplex noise or some other technique. Or you might be able to download some elevation data somewhere. But a common use case is to pull the height data from an existing bitmap image.

To that end I created a few new functions that can generate a height array from an existing image on disk, or from the current state of the canvas. In fact, that’s what I’ve been doing in the above examples - drawing the noise field and then pulling the height data from that.

To test that, I downloaded this random height map image from the web:

I loaded it in and adjusted the gamma somewhat to make a it a bit brighter.

I used my new height data function go get the levels and used that data to generate the contour lines:

Upping the number of countour levels to 32 really brings out details. Note, this was done with 1-pixel resolution.

I will say that at 1-pixel res and 32 levels, this took a few seconds to render (around 5). So, there’s room for optimization, but all in all not horrible as it is.

Resources

All my code for this is in my bitlib and blcairo repos. Specifically:

In addition to that, there was another article that I’d bookmarked a while ago on the subject but never got around to reading: Meandering Triangles

Just as I was sitting down to write this post, I pulled it up and scanned through it. I was pleased to see that the main algorithm was roughly the same as CONREC, but more pleased to see that the author had converted the points into paths in fashion quite similar to what I had done. I’m going to have to read this one more deeply, because I feel there may be some good things to learn there, and improve my own code.

2025 Retrospective

kp@bit-101.com (Keith Peters) — Thu, 25 Dec 2025 16:00:04 -0500

Well, here we are at the end of another year. Time to reflect on what went well, what didn’t, what I will do different next time.

Creation

Yes, I created some stuff…

Twenty-six posts on this blog.
Forty-two posts on https://www.artfromcode.com/
Twenty-four tracks on SoundCloud: https://soundcloud.com/bit101

Music

Let’s start with the last one, music. For a few years I wanted to get to the points where I could execute a song and release it into the wild for people to listen to. In fact, in my 2022 Retrospective one of my goals was to release a song. I did actually put out a “song” here and there since then. Often just some random tone generation or something like that. In 2025 I published 24 songs on SoundCloud. Actual songs with structure and different instruments. Not saying they were amazing, but they still make me happy when I hear them.

These were mostly created with Renoise and the M8 Tracker. This really kicked off in January when I was bedridden and quarantined with my first bout of Covid. I tried out Renoise and loved it. Loved the tracker paradigm and got an M8, which was really just completely amazing. I was very deeply into creating music for most of the first half of the year and then somehow just burnt out.

Creating music is a lot of fun and really rewarding. I’m extremely happy that I got to the point where I did with it, and that I had enough confidence to release the stuff I did. For my entire life I felt that the inability to do anything musically was a horrible failing. So I’m over that now. At the same time, I realized it’s not my calling. I’m sure I’ll continue to dabble in it now and then and create some tunes once in a while. But visual graphics and animation is my real passion. I hope to combine the two more in the future.

3D Creation

I’m not really a huge 3D guy. I do love creating retro-3D effects in wireframe or point clouds, but full-on 3D rendering has never been my outlet. That said, I did create my own 3D raytracer from scratch in 2022. And that was one of the most rewarding things I’ve done programming-wise. For some reason this year I was reminded of POV-Ray which I played with back in the day. It’s still out there and functions perfectly. So I installed it and had fun for a few weeks. I had a bit of a go trying to create a wrapper for POV in Go. I had some success, but in the end it was over-engineered and I went native and just started coding scenes by hand. Most of the problems I was trying to solve with the wrapper had better solutions built-in. It just took a while to find them. It didn’t go very far, but it was probably good for a month of fun. Here’s some more words on that subject:

https://bit-101.com/blog/posts/2025-06-14/pov-ray/

Daily Code

In October, I was feeling the urge to do some daily coding. I created my own hashtag, #octanimate and posted something every day in October. Here are the results:

When October was over, I wasn’t ready to stop, so just continued on daily for an additional 39 days, using the tag #dailycoding. Those are all over on Art From Code: https://www.artfromcode.com/tags/dailies/

After 70 straight days of posting, I was satisfied for a while.

I also took place in Advent of Code. This year was different. They went from 25 days down to 12 and killed the global leaderboard. This was a good change in my opinion. I still didn’t finish though. I got through day nine and had a blast, but I looked at the instructions for day 10 and said, “I’m done.” Maybe I’ll go back and try to wrap up the last three days at some point, but I’m good for now.

Genuary 26 is coming up in a week or so. I think I’ll abstain this year though. I’ve had good fun with these prompt-based challenges in the past, but in general, I’m not crazy about them. I have a lot of ideas on my own that I haven’t gotten around to. A lot of the prompts are interesting, but things I’ve already done. Some are just things that I have no interest in doing. I’ll probably look at them each day and if something sparks an interest I might give it a go.

Personal Tech

NAS / Home Server

I revamped a lot of my personal technology this past year. Earlier in the year I had a home server consisting of an thin HP client running Ubuntu Server. I had Plex, and then later Jellyfin running on it, some other personal services, using Wireguard ported through an external VPS.

This summer I upgraded all of that. I got a larger used HP box and installed TrueNAS on it, reused the older one as a backup server. Switched over to Tailscale from Wireguard. I spent a lot of time scheduling backups (local and cloud sync remote), S.M.A.R.T. tests, snapshots, scrubs, and set up all kinds of monitoring. More details here:

Since then I’ve also moved my RSS reader over to a self-hosted FreshRSS install and my “read it later” stuff to a self-hosted LinkDing. I’m really, really happy with both of those.

It was my intention to move the whole rig over to a dedicated NAS case with a low power motherboard early in 2026. But since it’s working fine as it is, I may postpone that a bit. At least to see if the price of RAM goes down.

Git

I fully migrated off of github. I was not one of those people who freaked out when Microsoft acquired github, but I don’t like the direction they’re headed right now and did not feel great about having my code there. After looking around, I settled on Codeberg. They seem like an amazing company and I feel really good about that move. I also set up a Forgejo install on my NAS, for hosting private repos. It feels really good to self host this. I did have a bit of concern over being fully responsible for maintaining this. But I have to trust in the work I did on my NAS - double backups, snapshots, scrubs, tests, monitoring, alerting. It’s the most responsible data protection I’ve ever had set up.

More info on that whole git migration here

Personal

Overall, going pretty well.

Work is fine. I think. One never knows these days, right? I think it’s good. I hope it’s good because I’m in my 60s now and the prospects of getting a new tech job at my age feel kind of sketchy. In the decades I’ve been in tech, and the many companies I’ve been in, I’ve never seen anyone my age hired into a technical role. I might have a chance as a manager. But for all the activism around various forms of discrimination (activism which I support), it’s still open season for ageism. I’ve heard age-related comments that might get someone fired if they said those things about any other group, but for age nobody bats an eye.

Health is fine. I wound up with a strangely large number of medical visits this past year. A couple of procedures, nothing serious at all, one dental, one cosmetic. And lots of follow ups and tests for previous concerns. Any time they look too closely, they find something else they want to look at. And when they look at that, they see another thing they want to check. But I worked through that whole cycle got all the tests done and have a clean bill of health at this point.

Next Big Project - Prints!

In recent weeks, I’ve embarked upon a new project, which is creating physical paper-and-ink prints of some of the stuff I create. I’ve repurposed keithpeters.org for that (was previously hosting my music). I’ve done some test prints and I’m really happy how they’ve come out. I’m doing a lot of other groundwork to set this up and some time very early next year I’ll have more to say about this.

Drawing Rotated Shapes

kp@bit-101.com (Keith Peters) — Thu, 04 Dec 2025 09:15:09 -0500

Just a quick one to verbalize a choice I often make but never thought much about.

Say you want to draw a 100-pixel line, starting at point A, and extending at an angle of 30 degrees (PI/6 radians).

A. You can do it with math. Using trig to find the end point of the line. Here’s some pseudocode, but should work similarly in many systems.

context.moveto(pointA.x, pointA.y)
context.lineto(pointA.x + cos(pi / 6) * 100, pointA.y + sin(pi / 6) * 100)
context.stroke()

B. You can transform the canvas and draw a 0 degree line to 100, 0.

context.save()
context.translate(pointA.x, pointA.y)
context.rotate(pi / 6)
context.moveto(0, 0)
context.lineto(100, 0)
context.stroke()
context.restore()

Stategy B has more steps. You have to save the context’s transform and restore it later, translate to the starting point to get things to rotate from that, do the rotation, then draw the line.

Strategy A, while fewer lines of code, requires more complex math.

Doing a bunch of fairly unscientific profiling tests in cairographics with Go bindings, the trig method generally wins out slightly, but not always. It’s pretty close.

So I can’t make a play on performance. You’d have to test your own code in your own platform.

I will say though, that if you’re drawing multiple shapes all rotated at the same angle from the same point, of course you should be transforming the context and then drawing them all.

On the other hand, if the location of that final point is important for some reason - maybe you want to color the line differently if it intersects some other shape - using trig and knowing the untransformed location of that second point would probably be the way to go.

But for the simple one-off cases, sometimes the trig method feels right, and sometimes the transform feels like the more obvious choice.

Simplifying Paths

kp@bit-101.com (Keith Peters) — Fri, 28 Nov 2025 14:30:48 -0500

Background

About a year ago I made a post about Evenly Placed Points on Bézier Curves. I’ve found that code to be enormously useful. In fact, recently, I expanded on this concept, so I can go from a list of points that create a path like this…

to a path made of piecewise quadratic Bézier curves like this…

to a path of points that recreates the curve directly like this…

The second two images may look the same, but one is drawn using quadratic Bézier curves and the other is drawn with a series of points with straight line segments between them. In fact, here’s that image drawn again, with only points, not lines. You’ll have to look pretty closely to see the points.

As described in the above-mentioned post, the latter is useful for interpolating along the curve, or placing items on the curve, or finding a point on the curve that it closest to some external point, and other things. The functionality I have now lets me go from the list of points in image 1 the the expanded list of points forming the curves in image 3, in one shot.

An example is this piece I did a couple of days ago.

Your browser does not support the video tag.

The circles traveling on the edge of the curve are uniformly distributed along that curve and they need to know the exact position of that curve at any point. This would be pretty tough with regular Bézier curves, but with uniformly interpolated points along the curve it’s much easier.

The Problem

Since my function takes an unknown number of points in unknown positions and forms an unknown number of quadratic Bézier curves of unknown length, it’s impossible to know how many points to assign to each curve individually. That is, how many small segments to break each curve into so it closely follows the shape of the curve. Through trial and error I’ve found that around 50 points virtually always does a completely suitable job. Fewer than that, and in some complex curves you wind up with sharp corners here and there. So I hard-coded 50 into the function.

And in general this has been great but I know in some cases I’m winding up with many, many more points than I really need. If I have just 6 points in my original list, that winds up with 6 curves and 300 points in the final path. Again, look at the previous image with the curve rendered only in points. Many of the points are so tightly packed you can barely see them individually.

So rather than trying to get the magic number of points up front, I started thinking about how I could simplify the path after the fact. Remove the extraneous points, keep the necessary ones.

The Solution

In transparency, I’ve done this kind of thing before in Flash, probably close to a couple of decades ago. I remembered what I did then and tried to recreate it. To pretty good success in fact.

The concept is pretty simple. I loop through my point list, looking at groups of three consecutive points, points[0], points[1] and points[2] for example. Call them pA, pB and pC. Then I test if these three points are collinear. In other words, do they fall on the same line? If so, then pB isn’t adding anything to the curve at all. I could just draw a line from pA to pC, ignoring pB altogether and you wouldn’t know the difference. So I remove pB from the list and carry on through the rest of the list.

Something like this:

function simplify(points) {
    for (i = 0; i < points.length - 2; i++) {
        pA = points[i]
        pB = points[i+1]
        pC = points[i+2]
        if (colinear(pA, pB, pC)) {
            points.remove(pB)
        }
    }
}

Obviously, that’ pseudocode, and altering a list while iterating through it can get you in trouble. It all worked out for me the way I set it up in Go, but I’ll leave it to you to figure out the best way to cull out those middle collinear points in your language and framework.

Now of course, it’s impossibly rare that you’ll ever find three dynamically generated points that are exactly collinear, so we’ll allow a little leeway. We’ll actually be saying, are these points close enough to being collinear?

So how do you tell if three points are collinear? Well, there are different ways to go about this. The most obvious and direct is to compare the slope of the line from pA to pB to the slope between pB and pC. That would look something like:

function colinear(pA, pB, pC, threshold) {
    m0 = (pB.y - pA.y) / (pB.x - pA.x)
    m1 = (pC.y - pB.y) / (pC.x - pB.x)
    diff = abs(m1-m0)
    return diff < threshold
}

Now you can adjust your threshold value so you’re removing the most points possible, but your curve isn’t starting to get chunky. I find that a threshold of around 0.1 gives pretty good results. This means that if the two slopes differ by less than 0.1 you can remove the middle point.

Visually, this image might help:

In the top image, the slop from the first to second point is quite different than the slope from the second to the third. These are very far from collinear.

In the bottom image, the slopes are pretty similar. This is just a visual example, but you might be able to say this is close enough to collinear.

But I found I got a lot better results with a different method.

Alternate method

In my library I have a Segment object, which is formed by two points. And I have a Segment.distanceTo(point) function that you can pass a point into and it gives you back the distance from that point to the segment. I make a segment with pA and pC and get the distance from pB to that segment. It’s something like this:

function colinear(pA, pB, pC, threshold) {
    segment = new Segment(pA, pC)
    dist = segment.distanceTo(pB)
    return distance < threshold
}

Here, I find a threshold of about 0.25 gives good results. In this case it means pB needs to be less than a quarter of a pixel from the segment formed by pA and pB in order to be removed. That’s pretty close to collinear.

Here you can see the first and third points joined by a segment (in black) and the distance from the middle point to that segment in red. In this case, it’s pretty close to the segment and might be considered collinear.

Btw, you can find my Segment object here: https://codeberg.org/bit101/bitlib/src/branch/main/geom/segment.go

For whatever reason, with this method I can remove a lot more points and still have a much smoother, more accurate curve than I can using the slope method.

My theory is that with the slope method, you can have a sequence of points that are very close together. The slopes might differ more than your threshold, so the middle point doesn’t get removed. But because the distances are so small, that middle point is visually collinear - maybe only a small fraction of a pixel away. It should be removed.

Again a visual might help.

Consider this a very zoomed in view of three consecutive points. The slopes between the two pairs differs by a good amount, but if you consider that each circle is one pixel in diameter, it’s only off by a small fraction of a single pixel. If you zoomed out, you’d barely see this.

So you wind up with not as many points being removed. But if you crank up the threshold to catch those close-up points, you wind up removing too many points in areas where the points are further away and things get chunky there.

The distance-to-segment method is much more aggressive on closely spaced points, where you’re safe to remove more, but more lenient on larger curves with more space, where you want to keep points for accuracy.

That’s my theory anyway.

Iteration

If you do well here, you could end up removing a good number of points. Maybe even half, depending on the state of your list. But you might be able to do better. Run it again and you might be able to remove a few more. And again, and maybe again a couple more times. Rather than manually calling it a bunch of times, you can add an iteration parameter to your simplify function. I’ll also throw in a threshold parameter.

function simplify(points, threshold, iter) {
    for (j = 0; j < iter; j++ ) {
        for (i = 0; i < points.length - 2; i++) {
            pA = points[i]
            pB = points[i+1]
            pC = points[i+2]
            if (colinear(pA, pB, pC, threshold)) {
                points.remove(pB)
            }
        }
    }
}

I’ve found that on a fairly complex curve, with the distance-to-segment method and a threshold of 0.1, five iterations gets down to the minimal number of points. Anyway, this method gives you full control. But maybe you want something simpler that just works pretty good most of the time.

Auto-simplify

I created another method called autosimplify. This hard-codes the 0.25 and intelligently iterates as much as it needs to.

It does this by keeping track of the number of points before and after each iteration. If it’s the same, then it’s stopped removing any points. You’re done. If it removed any, then it will try again and see if it can removed more.

function autosimplify() {
    count = points.length
    while true {
        for (i = 0; i < points.length - 2; i++) {
            pA = points[i]
            pB = points[i+1]
            pC = points[i+2]
            if (colinear(pA, pB, pC, 0.25)) { // hard-coded threshold
                points.remove(pB)
            }
        }
        if (points.length == count) {
            // we didn't remove anything this time. done.
            return
        } else {
            // we removed something. keep track for next time.
            count = points.length
        }
    }
}

Stress Test

To show off the results - both in quality and in point removal, I created this stress test. It starts off with 50 moving randomized points. This creates 50 internal curves with 50 points, for a total of 2,500 points in the final curve. (2,501 actually, since I add an extra one in there to close the loop for… reasons.) This original path is drawn in blue. Then I run the autosimplify function as described above, and shift the result over 10 pixels. In the center, I print the number of original and simplified points.

Your browser does not support the video tag.

To my eyes, there is little or no difference. And the numbers don’t lie. Average reduction is 80%.

Summary

The concept here is, start with more points than you probably need. Then, intelligently cull out the ones you don’t need.

I’m sure there are improvements that can be made here. Maybe a more dynamic colinear method that takes into account not only the slope but the distance. I’m sure this will see some improvements over time.

Runways

kp@bit-101.com (Keith Peters) — Mon, 24 Nov 2025 06:00:34 -0500

I’ve been posting daily coded animations for the past 55 days as of today. You can see my “Octanimate” series in these posts:

And what I’ve been doing in November here: Dailies

The latest one of the dailies is here:

Your browser does not support the video tag.

I really enjoyed the process of making this and wanted to share some of the process.

For a while now I’ve had the urge to do some kind of mapping visualization. I’d been looking around for open elevation map data sets but didn’t find anything easy to get a hold of and work with. The other day I thought, “what about just getting the coordinates of every ____ in the country?”. It would have to be something that was important enough to be documented and worth someone actually compiling the latitude and longitude of each point.

Airports was the first one that came to mind. So I did a search. The first hit I got not only had the location of each airport in the country, but the coordinates of each runway! Eight thousand, six hundred and twenty four of them to be exact. Here’s that link. https://geodata.bts.gov/datasets/110af7b8a9424a59a3fb1d8fc69a2172_0/explore?location=42.363561%2C-71.006261%2C14.39&showTable=true

There are multiple formats available for the data. I first went for the GeoJSON format. But this was a bit too much. It actually contained coordinates for the polygon shapes of each runway! I pulled down the CSV version and that was much simpler - the coordinates for the start and end of each runway, basically just a line segment. Here’s a single line of that file, representing a single runway at LAX.

635,2025/10/30,01818.,A,CA,LAX,LOS ANGELES INTL,LOS ANGELES,US,PU,PU,06L/24R,33.94911247,-118.43115986,33.95210391,-118.40194891,8926,150,ASPH-CONC,GOOD,GRVD,,,,,,HIGH,3RD PARTY SURVEY,2010/03/23,120,203,383,880,181220.57421875,6663.48966096801

The coords for that runway are: (33.94911247,-118.43115986) to (33.95210391,-118.40194891), found in the 13th through the 16th fields there.

This was super easy to parse in Go, grab the two points for each runway, do a bit of translation and scaling to center the whole thing and draw each one of those 8,624 runways. And voila!

Since a few degrees of lattitude or longitude cover a very wide area, mapping this directly to pixels gives us a tiny little map. But fi I zoom in by say 15x, we’re seeing the whole country.

Now if I pan over to Boston Harbor and zoom in 10,000x, we have the glorious chaos that is Logan International Airport.

And it’s easy enough to do that for any location. Here’s San Francisco.

Next I wanted to animate it. This took a bit of trial and error. On the one hand, you want to see what some actual runways look like. On the other hand, you want to see the whole mass of them. After trying a whole bunch of things I settled on starting zoomed in on Boston at 4000x, slowly zooming out to 25x then panning over to SFO, and finally zooming back in. Stuck some labels on there and a bit of fading in and out and it’s a wrap.

This was a really fun project and really just took a couple hours one night and a bit of touch up the next morning. Getting, parsing and drawing the data was easy. The zooming into a specific location and the labels were a bit tricky, and getting the animation right consumed the most of the time. I’m looking forward to finding other data sets like this that I can play with.

Daily Coding

kp@bit-101.com (Keith Peters) — Sat, 15 Nov 2025 16:13:06 -0500

I had such a fun time doing Octanimate last month that I just kept going.

I didn’t come up with a catchy tag for November, so I’ve just been using #dailycoding on mastodon (others use the tag too). I’m not going to post those on this site, because I’ve also been keeping up with them over here: https://www.artfromcode.com/tags/dailies/. I don’t know how long I’ll keep thig soing, but it’s fun so far.

In October, my theme was noise. Not intentionally, I just wound up doing a lot of Simplex noise based stuff. This month, I’m working a lot with particles. I used to be obsessed with particles systems in my early Flash days. I haven’t done much in a while, so I started building out a particle system module that will eventually wind up in my own library. But you’ll see some of the pieces there use it already.

Octanimate Part III

kp@bit-101.com (Keith Peters) — Fri, 31 Oct 2025 15:55:00 -0500

Day 21: Your browser does not support the video tag.

Day 22: Your browser does not support the video tag.

Day 23: Your browser does not support the video tag.

Day 24: Your browser does not support the video tag.

Day 25: Your browser does not support the video tag.

Day 26: Your browser does not support the video tag.

Day 27a: Your browser does not support the video tag.

Day 27b: Your browser does not support the video tag.

Day 27c: Your browser does not support the video tag.

Day 28: Your browser does not support the video tag.

Day 29: Your browser does not support the video tag.

Day 30: Your browser does not support the video tag.

Day 31: Your browser does not support the video tag.

Octanimate 2025 Part II

kp@bit-101.com (Keith Peters) — Thu, 23 Oct 2025 18:48:49 -0400

To recap, #octanimate is just my personal hashtag for posting an animation every day in October. You can follow me on mastodon if you’re there (you should be there, btw). Or you can just view the entries in a few groups here. A few days ago I posted the first ten entries. Here are the next ten.

Day 11: Your browser does not support the video tag.

Day 12: Your browser does not support the video tag.

Day 13: Your browser does not support the video tag.

Day 14: Your browser does not support the video tag.

Day 15: Your browser does not support the video tag.

Day 16: Your browser does not support the video tag.

Day 17: Your browser does not support the video tag.

Day 18: Your browser does not support the video tag.

Day 19: Your browser does not support the video tag.

Day 20: Your browser does not support the video tag.

See you in another week and a half or so for the final group!

Proximity Webs Across Point Sets

kp@bit-101.com (Keith Peters) — Tue, 21 Oct 2025 16:43:41 -0400

Background

I posted this animation on Mastodon today, as day 21 of Octanimate.

Your browser does not support the video tag.

I didn’t think too much about it, but it got more likes than I expected, and quite a few comments or questions about what exactly was going on there. I threw out a few quick one line answers there but thought it might be worth more of an explanation.

It’s actually a pretty neat technique that I came up with just on a usual “what if…?”

Specifically, what if I did a proximity web not just on a single set of points, but on two sets of points. Even more specifically, I wouldn’t compare each point in a set to all the other points in the same set, but I’d compare each point in set A with all the points in set B. If they were within a certain distance, draw a line between the two points. And then move the points in each set around in some way.

Step by Step

First we take a grid of points. Pretty basic. These are 20 pixels apart on each axis.

Now we make another grid of points. Spoiler: it looks exactly the same as the first grid.
Now we rotate each grid, one clockwise, the other counter-clockwise (or anti-clockwise if that’s how you roll).

Your browser does not support the video tag.

Already this is creating some interesting patterns. But let’s keep going.

Then, as described above, we’ll take each point in grid A and compare it to each point in grid B. If they are less than 28 pixels apart, we draw a line between them. The line’s width is inversely proportional to its length. Why 28? That’s roughly the square root of two times 20 (distance between points in each grid). So the connecting lines will almost but not quite reach across the diagonals when the grids are aligned. I figured more than that was a bit too busy, less was too sparse. You may have to zoom in to see the lines if you’re on a phone or other small device.

Your browser does not support the video tag.

Again, this is creating an interesting pattern. But a bit too rigid for my taste.

Randomize each grid. In other words, move each point some distance in a random direction from its normal location. But not permanently. We’ll start with a regular grid, move each point out a ways, and then back.

Your browser does not support the video tag.

And again, make two of these and have them rotate in opposite directions.

Your browser does not support the video tag.

Finally, we go back to drawing the lines, and remove the points altogether, with a bit of blur to smooth it all out. And we’re back to the original

Your browser does not support the video tag.

A Different Example

Interestingly, the previous day I’d posted a similar piece using the same basic technique. In this case the two point sets were quite different. One was a grid exactly as I showed here, the other was just a set of random points, being moved around with 4D Simplex noise. It looks like this on its own:

Your browser does not support the video tag.

But when we apply the same technique to these two sets of points (with a max distance of 20), we get this:

Your browser does not support the video tag.

To me this looks like some kind of alien writing. The caption I gave it when I posted it was “The alien encyclopedia contains all the knowledge of the universe. If only we could read it.”

I actually like this one a lot better than the dual grid one, but the dual grid piece got more interaction and questions. I guess you never know.

Source

The source for both of these is at https://codeberg.org/bit101/octanimate25 if you care to dive in a bit more.

Octanimate 2025 Part I

kp@bit-101.com (Keith Peters) — Sun, 19 Oct 2025 12:01:02 -0400

Over the years I’ve taken part in a number of community projects. Like “7 Days of Code”, “Jamuary”, “Genuary”, etc. And a few I’ve started ad-hoc and wound up getting a decent following on - like “MayContainGifs” and “AuGIF” and something like “JulyIsForGIFs” (or was it “GulyIsForJIFS”?), where people created a gif or animation of some kind every day for a month.

I was feeling the need to do this again a few weeks ago, so created “Octanimate” where I commited to posting an animation every day in October. I didn’t really do anything to promote it other than just start using the hashtag on my daily posts. I think so far one other person has used the tag. But that’s fine. It’s driven me to create and post 19 animations so far this month and I’m confident I’ll get through the next dozen days. Most of these have been on Mastodon, and you can search the tag here:

https://mstdn.social/deck/tags/octanimate

If there’s any kind of theme for this one it’s noise. Mostly just warping things smoothly with 4D Simplex noise. (See my previous post.)

Also, the source for all these is up at https://codeberg.org/bit101/octanimate25. It’s all in Go using my own bitlib and blcairo libraries, which are also open source, though I don’t think anyone else in the whole universe uses them. But again, fine. I love them. And anyway, if you want to figure out a technique I used on any of these, it shouldn’t be too hard. Just note the license.

Anyway, I figured I’d post the first ten days worth here. and do it again for 11-20 in the next few days, and then wrap it up at the end of the month. No comments, no explanations, just animations. Here we go:

Day 1: Your browser does not support the video tag.

Day 2: Your browser does not support the video tag.

Day 3: Your browser does not support the video tag.

Day 4: Your browser does not support the video tag.

Day 5: Your browser does not support the video tag.

Day 6: Your browser does not support the video tag.

Day 7: Your browser does not support the video tag.

Day 8: Your browser does not support the video tag.

Day 9: Your browser does not support the video tag.

Day 10: Your browser does not support the video tag.

Looping Noise

kp@bit-101.com (Keith Peters) — Fri, 17 Oct 2025 09:00:00 -0400

This is a story. A story of laziness and stupidity. But also a story of redemption and victory.

It’s about noise. Like Perlin noise or Simplex noise. And making it move. But making it move in a specific way. It’s about the problem, the crappy solution I lived with for too long, the eventual hack I figured out, and finally, the real solution which I knew was there all along but for some reason never implemented. And some other roads worth following in the future.

Setting the stage

So you want to animate some noise. It doesn’t really matter what you’re doing with that noise or how you’re rendering it. For something to work with, let’s just render some noise on a square surface. We’ll get the value of the noise at each pixel location and convert that to a grayscale value and color the pixel using that. I’m gonna go with some pseudocode in this post because I don’t want to get caught up in language details.

for x = 0 to width {
  for y = 0 to height {
    n = noise2(x * scale, y * scale)
    n = map(n, -1, 1, 0, 1)
    setPixelGray(x, y, n)
  }
}

Generally you want to scale your pixel values down a lot lower before feeding them into your noise function. Very low values give smoothly changing values, while high values look more chaotic. The actual scaling values you need will differ based on the noise algorithm you are using, but you’ll eventually get a feel for the ranges that work.

The pseudocode noise2 function is 2d noise - generally Perlin or Simplex.

I’m assuming your noise functions return values from -1 to 1, and you have a function that will set a pixel to a grayscale value from 0 to 1, so the map function converts between the two. If you want to learn more about map, read this.

Again, your language and platform and libraries may do something different, but I trust you can work it out. A possible result from the above:

Animating it

OK, now we want to make it move!

We could move it on the x-axis or the y-axis like so…

n = noise2((x + offset) * scale, y * scale)
// or
n = noise2(x * scale, (y + offset) * scale)

Here, I’m considering that offset is some value that is changing over the course of the animation. Maybe the first frame it’s 0 and on the last frame it’s 400. This is going to give you something like this:

Your browser does not support the video tag.

Not particularly exciting. Yes, we are animating the noise, but we’re not really changing what the noise looks like, we’re just panning across a noise field. Also, when it gets to the end it jumps back to the beginning. Nothing more I hate than a non-looping short animation. Well, there a few things I hate more, but non-looping animations are up there.

Into higher dimensions!

OK, not higher higher dimensions, but higher than two. Three for now.

If we use three-dimensional noise, rather than a flat plane of noise, we have a solid space of noise. It’s infinite in size, but you can imagine it as a cube. We can move through this cube on the x, y or z axes. At any given point, we’ll be looking at one slice of the noise at a given z value. It’s a bit like doing a CT scan or MRI or ultrasound on that noise cube. And if you change the z value over time, you can animate those slices and the noise morphs around into different shapes.

So we’re looking at something like this:

n = noise3(x * scale, y * scale, z)

We could scale z here too, but in this case we’re not stuck with pixel coordinates, so there’s no real need to do that. We can just have z interpolate between whatever values we want. For example, z might be 0.0 on the first frame, and 1.0 on the last frame.

Here’s what this looks like:

Your browser does not support the video tag.

OK, that’s pretty cool, but it certainly doesn’t loop. It goes from one z value to another and when the animation ends it jumps back to the start and plays again. So we need a smooth transition back around to the starting point. (If you’re doing a real-time animation where you can just continuously increase z then you’re all set. But if you’re doing a rendered animation, you’ll want to figure out how to end where you started.)

So here’s the crappy solution I’ve been using for years. [shame…]

Diligence overcomes difficulties; sloth makes them - Ben Franklin

I animate the z value from value A to value B (say 0 to 1) for the first half of the animation, and then back to 0 again for the second half. If we do this linearly, it looks like this:

Your browser does not support the video tag.

A bit better. It loops, but not very smoothly. It moves along at a linear pace then hits the end and bounces back the other way. So linear is not good - and not what I actually used. Just showing the process step by step.

Imagining you have a normalized value t that is 0.0 on the first frame and 1.0 on the last one (technically just shy of 1.0 but that’s a story for another day). You create your noise like so:

n = noise3(x * scale, y * scale, sin(t * tau) * offset)

Multipling t by tau (2PI) gives us an angle. Take the sine of that and multiply it by an offset value - how far we want it to move back and forth in the z-axis. 1.0 will do fine for starters.

Your browser does not support the video tag.

This is pretty nice… way better than the linear version. No jarring jumps at least. It kind of coasts slowly to a stop before reversing. I did things like this for years. And at some point I started hating it.

What I want is something that just continues to morph the noise’s shape, but magically ends up exactly back where it started, without reversing.

The problem is we’re just using three dimensions, taking 2D slices, sliding down through the cube and back up. So the reverse path is exactly the same as the forward path. We need another dimension to get more movement. Then you could loop through those two dimensions in a circle or ellipse and have a unique path that arrives back where it started.

I’d like to borrow a dimension please

At first I thought, “Oh, I can just borrow the x or y dimension and make it do double duty. And you kind of can. The code would be something like this…

x = x * scale
y = y * scale
n := noise3(x, y + cos(t * tau) * offset, sin(t * tau) * offset)

Here we pre-multiply x and y by scale only to make the longer line a bit shorter.

Again, we have t * tau to get an angle, but now we take take the cosine and sine of that, times an offset. That gives us a unique y and z value that will follow a circular path and end up back at the start. Sounds good, but here’s what it looks like:

Your browser does not support the video tag.

It’s not super satisfying. It’s a bit of the original panning combined with the 3D slice scanning. It’s still got that move through to a certain point, then reverse feel to it. Not as bad, but not great. So borrowing a dimension is not going to work.

So a minute ago I said we need another dimension. That right there is the solution. We just need 4D noise. It exists. But I didn’t have it in my library. And I didn’t want to dig up a version of it and port it to my language of choice (Go). So rather than do that…

The Hack

Learning nothing, I steadfastly tried another way to get around 4D noise. And I’m kind of ashamed to say, it worked out pretty damn well.

My thoughts: What if I take the z-slices in one area of the noise field using sine, and then take z-slices from another area using cosine? Then average the two of them. Since the movement is out of phase, one would be moving down quickly while the other was slowing down, and the first would be still moving down while the other was moving back up.

Or, another way of saying it is: rather than adding a dimension or borrowing one, we’re just using a different part of the noise field and pretending it’s another dimension. But this gives us two different slices, which we just average together.

n1 = noise3(x * scale, y * scale, sin(t * tau) * offset)
n2 = noise3(x * scale + 100, y * scale + 100, cos(t * tau) * offset)
n = (n1 + n2) / 2

And here’s what that looks like:

Your browser does not support the video tag.

Honestly, that’s not too bad. Animation-wise, it’s exactly what I wanted. I did find that the averaging wound up kind of washing things out and reducing contrast. In other words, averaging is going to result in somewhat of a bell curve. Fewer darks, fewer lights, more grays. So I used another small hack after the above to increase the contrast:

n = map(n, -0.8, 0.8, 0.0, 1.0)

Due to the averaging, more values will wind up in between -0.8 and 0.8. So we just map value in that range to 0, 1 in the map function we were already using. It makes the darks darker and the lights lighter. You can play around with that value to get different levels of contrast. But beware, too much and your darks and whites get overblown.

Here’s a corrected version:

Your browser does not support the video tag.

I was pretty proud of this. And also rather ashamed at the same time. If you’re on a platform where you just can’t get a 4D noise function, I think this might be a viable solution. But it was time to put on my big boy boots and do it right.

Just do it

Yup. I ported a Simplex 4D function to Go.

It wasn’t all that hard. I should have done it years ago. I actually re-ported all four dimensions from this C code into Go. All told it took about an hour. And I’ve been loving 4D noise for weeks since.

To make smoothly looping 2D noise using 4D noise, you use two of the dimensions for your rendering and the other two to loop in a “circle”. Something like this:

n = noise4(x * scale, y * scale, cos(t * tau) * offset, sin(t * tau) * offset)

I’m using the x and y dimesions for my x and y values and doing the looping with z and w.

Here’s an example of rendering that:

Your browser does not support the video tag.

Resources

I did a lot of research on this while I was trying to figure out a way to do this without using 4D noise. There may still be a valid way to do it, but I didn’t find one using regular noise functions. A lot of Reddit and StackOverflow answers merely talk about smoothly animating 1D noise using 2D noise. But they rarely express it in those terms, just like, “Yeah, it’s easy to have looping noise, just use 2D noise with polar coordinates and that will smoothly loop the resulting value.” Yes, the 1D value.

Daniel Shiffman has a nice video on this topic, where he explains more in depth about, again, looping 1D noise using 2D. He gets into animating 2D noise using 3D noise, but never gets to the point of looping the 2D noise. Still, it’s a very well explained video, which might clarify some of the points I brushed over here.

Polar Perlin Noise Loops

He also mentions Etienne Jacobs’ post on looping noise:

Drawing from noise…

This goes into many of the same things and gets into 4D noise, but in my mind doesn’t go too much into the “why” of needing four dimensions.

And finally, I just found this Shiffman video well after I wrote this post and as I was just tidying up the resources section:

4D Open Simplex Noise Loop

One last thing…

There is another way around this problem. It has to do with tiling noise. There are ways to make Perlin and Simplex noise tileable in different dimensions. This is usually used to make textures that … well, tile. In other words, for a 2D noise tile, the left edge of the tile would perfectly match up and smoothly blend to the right edge. Same with top and bottom. But if you imagine sampling values going from left to right in a stright line, you’d wind up with the same values over and over. Essentially the same as if you had looped it. (Actually the same thing would happen if you went in a straight line in any direction!)

If you extend that to 3D tiled noise, you could create looping noise by just moving through the 3D slices of that noise field, without any need for the fourth dimension.

However, you need a noise algorithm that creates tiling noise in order to do this. Here’s a paper on that, if you want to explore this route. Tiling Simplex Noise and Flow Noise in Two and Three Dimens

Summary

In short:

I knew that 4D noise was the solution very early on, but was lazy about porting 4D noise into my library.
I assumed there was some other trick people were using to get around using 4D noise.
I tried a bunch of things that were not great.
Many of the Reddit and Stack Overflow threads on this seem to say it’s easy, but they are describing an easier problem, not the probem I was trying to solve.
I eventually did hack together a workable solution!
Porting 4D Simplex noise to Go was pretty easy.

Proximity Webs

kp@bit-101.com (Keith Peters) — Sun, 12 Oct 2025 14:30:08 -0400

I don’t think I ever actually wrote about this before, but I decided I probably should. It’s about a technique I call a “proximity web”. It’s something I first did about 23 years ago. I don’t recall seeing anything quite like it before, but I’ve seen it hundreds of times since then. I’ve seen it on posters, billboards, tv shows, movies, many web sites, music videos, etc.

Now I’m not going to be so bold as to claim that I invented this. But… I might have??? There is an old friend of mine who has since become pretty big in the generative art world. They once told me that as far as they were concerned, I invented it.

I can say that I came up with the technique on my own. But whether people copied me and eventually it became a thing, or whether someone else did it independently before or after me and others copied that person, I have no idea. All I know is that whenever I see it somewhere, I grin a little bit and internally take a bit of ownership.

For the record, here is a recording of a Flash movie I published on September 20, 2002:

Your browser does not support the video tag.

This was the first time I used the technique. I did a bunch more similar things over the next week or so. Here are just a couple of examples:

Your browser does not support the video tag. Your browser does not support the video tag.

I did a lot more of this over the next few years, but that’s the basic idea.

Here’s a quick example I just put together now while writing this. This is the kind of thing you’re most likely to see out in the wild. Sorry it doesn’t loop. I got lazy. But this is very much what I was posting in 2002 and 2003 and beyond.

Your browser does not support the video tag.

Ignore the mechanics going on in those examples in terms of how things are moving around. The proximity web concept is just how they are connected. In short:

There is a maximum connection distance between any two nodes.
If the distance between two nodes is more than that, there is no connection.
If the distance is at or below that threshold, the nodes will be connected with a line.
The thickness of that line is inversely proportional to the distance between the nodes.

In other words, when the nodes are very close together, the line is very thick/bright/whatever. As the distance increases towards the maximum, the line gets thinner and lighter so that when it reaches the maximum it becomes zero width.

In pseudocode…

dist = distance(nodeA, nodeB)
if dist < maxDist {
  setLineWidth(1 - dist / maxDist)
  strokeLine(nodeA, nodeB)
}

In this case the thickness goes from 1.0 when the nodes are right on top of each other, to 0.0 when they are at maxDist away. But it’s simple math to change that to whatever range you want, or use it to define brightness instead of width or whatever else.

This is a really cool effect when animated because it’s like the connection is stretching as the nodes move apart, and finally breaks when it gets too thin.

Note well, I’m NOT saying I thought up the idea of connecting dots with lines. But this particular rendering effect where the nodes are floating around and the lines fade in and out based on proximity is rather unique.

Jump forward a few years to October 7, 2008. I posted something on my Art from Code site, called Lissajous Webs.

This used a Lissajous curve to create a large number of static points and connected them all with a proximity web. The huge number of connections made some really interesting textures.

Here are a few examples:

The next day I threw some randomness into the Lissajous function and came up with Random Lissajous Webs

Again some examples.

These evoked a really mysterious look, which I loved to death. And others did too. In fact, if you search the term “Random Lissajous Web” you’ll find a few examples of this exact thing with, sadly, rarely any attribution. Even when the authors are using a license that requires others to attribute their work, they don’t attribute the technique they used - or even the name! Sigh. Some examples from as late as 2023.

https://openprocessing.org/sketch/1836813

https://openprocessing.org/sketch/1793237

On the plus side, I did get some attribution in the massive generative art book, Generative Design. When I say “massive” I mean it’s about two and a half inches thick and chock full of amazing work, examples and code. (Note, this was from an era before the word “generative” had been stolen by AI “creators”.)

There’s a whole chapter with webs based on my work. In fact they got my permission for this beforehand and I even got a free copy of the book, which was nice.

Summary

I’m not complaining or looking for any big pats on the back. I’m not suing anyone even if I could. I’m not accusing anyone of ripping me off maliciously. I put my stuff out there as open source, often without a lot of thoughts on licensing. And a lot of people liked it and used it for a lot of things. I’m honored. All I’m asking is that if you see something like this out there in the world… give me a virtual fist bump, at least in your own mind!

And of course, if you do have some examples of this that predate 2002, I’d love to see them. It would crush my ego, but there’s not much of that left anyway.

Dictionaries

kp@bit-101.com (Keith Peters) — Sat, 27 Sep 2025 08:11:47 -0400

Growing up, and even into my 20s and 30s I was an avid dictionary user. I was taught how to use them and I always had one or more on hand. Nice thick volumes that had the spelling, the pronunciation, all the different definitions broken into parts of speech, including transitive and intransitive verbs, example usages, and etymology. Maybe even some synonyms or special usage notes if there was something tricky about it. I especially always dug into the etymology. Knowing the history of a word really got that word to sink in for me. And having done that a lot made it easy to understand other words with similar backgrounds.

I haven’t had an actual dictionary for years. But I still try to look up words using the web. It’s just not the same. I didn’t think about it too much, but it finally just got frustrating enough where I snapped. Usually I just type the word, plus “definition”. That usually gives me some kind of search engine definition and links to random dictionary sites. I generally try to go into one of the dictionary sites.

Let me tell you… dictionary sites are horrible.

Dictionary Sites

They’re even generally run by companies that are well known for publishing quality dictionaries. But once they get on the web, it’s no longer a dictionary. It’s a web site. It’s an experience. It’s a social network. It’s a marketing opportunity. You get ads and games and quizzes and trending words, slang guides to the new words kids are using. Word of the day. How-tos, interesting word histories, videos, podcasts, shops, merch, mailing lists. I’m not making this up, I’m just cataloging what I see on the Merriam Webster front page.

See that tiny red bar at the top? That’s the part I’m interested in - where you can enter a word to find its definition.

See all the rest of that crap? Not interested in any of it.

It should just look like the Google home page. Logo, search field. End of story.

Now, that’s Merriam Webster, which is a solid dictionary. But OMG, the site… makes me cry. A computer and a web site should be the best thing that ever happened to a dictionary. No flipping pages back and forth. Just type the word and you’re there. Don’t understand a word in the definition? Click to go to that, then click back. Why did they have to ruin it by doing all this other stuff?

But OK, what happens when you look up a word? Let’s look up “canonical” - for no reason. It just came to mind. No, it’s not a reference to the company that puts out Ubuntu. Just a random word.

Alright. We have a nav menu with games, word of the day, grammar, word finder, slang, newsletters, wordplay, and more. And a sidebar nav of what’s on this page. And a couple of call to actions for games and stuff.

But we do have definitions. And each definition has an example.

And we have more examples, but these are “automatically compiled from online sources to show current usage”. I don’t really like this idea. Real dictionaries hand curate useful examples, not do automated web searches. At least it’s not obviously “AI”.

Then we have etymology and some other stuff…

And more stuff…

I don’t need rhymes and nearby words or citations. This isn’t bad stuff, but it’s just trying to cram everything possible in here.

Keep scrolling and you’ll be back into games and merch and stuff.

The etymology entry links to the root word, “canon”, which is fine because it’s a click away. An actual dictionary would do the same thing.

So honestly, this is decent content. It’s just surrounded by a deafening amount of random crap.

I’m not going to compare other sites. Because this is one of the better ones - at least in terms of content.

What about apps?

My next thought was to try an app. Honestly, I was about to go buy a physical dictionary, but hoped an app might be better than the sites I’d seen.

The two I tried were the Merriam Webster app and the Oxford Dictionary app. Both on Android. Both have free versions with ads and subscription models. Ugh. But I’ll keep an open mind.

The Oxford Dictionary app

The Oxford Dictionary app contains something like fifteen different dictionaries - English, American and a bunch of other languages. It also has full page ads that you have to wait like 10-15 seconds before you can see the content. Every time you look up a word. Like on those really, really crappy free games. So, so, so bad. But, you can get rid of those for just $2.99 per month. $36 a year for a dictionary? OK, fifteen dictionaries, thirteen of which I’ll never touch. Meh.

While the content was decent, the format, layout and overall UX just wasn’t great.

Interestingly, I also checked out the American Heritage Dictionary app. It’s by the same publisher and looks exactly the same as the Oxford app. Only has the one dictionary and it’s $11.99 per year. Or you can buy it for $49.99.

The Merriam Webster app

Alright, we’re getting somewhere. Again, ad supported free app, but it’s a small banner at the bottom of the page. Not awful. And it’s $2.99 per year to go ad free. Also, $5.99 per year for some kind of pro version, which also gives you a kids dictionary, and medical and legal dictionary. Not sure if I needed those, but I went for the $5.99 version. I think it might have some other features. But that’s a decent price no matter what. The physical version of this same dictionary would be around $21. And would not fit in my pocket.

So what’s it look like? Any better than the web site?

Yes! I was very pleasantly surprised!

When you first open it, you get this:

Not TOO horrible. Only two random distractions.

Once you’re in the word though, it’s great.

It’s got the same defintions and embedded examples as the site and some additional info, including etymology. But that’s it. Nicely formatted and readable.

Extra info, but not too much.

Other nice features

Audible pronunciation
Thesaurus too! That’s always useful, practical, applicable, functional.
Click on any word on the page to look up that word.
Scroll left/right to go through your history. I actually missed that at first and was trying to back navigate, which just took me to the home page, which was very annoying.

For words that are in the kids, medical, and/or legal dictionaries, you get links there as well.

To be honest, the kids dictionary is pretty good if you just want a quick idea of what something means.

The app menu also lets you see a list of recent words you looked up, as well as any words you’ve saved. You can also share definitions… oki doki, whatever.

Overall, this is a winner. Just what I wanted.

Other options

Desktop app

I thought of trying Stardict. This is an open source dictionary app, which you can add different dictionaries to (there are a lot). On my ebook readers (Onyx Boox devices) I use KOReader which integrates with Stardict compatible dictionaries. It’s a great concept, but the content varies widely, depending on which dictionaries you add to it. Many of them are outdated and oddly formatted. But it’s great to have it integrated in the ebook reader. Just tap a word and get a definition. But all too often, there’s no result found in any of the three dictionaries I have installed there.

At any rate, a couple of attempts to install Stardict on my computer just led to segfaults. I’m sure I could debug it, but I’m pretty happy with the Merriam Webster app for now.

Wiktionary

It was also suggested trying Wiktionary. This is a free dictionary operated by Wikimedia, who also operate Wikipedia, etc. This is also very good, but goes a bit deep. Oddly, etymology is up front, followed by usually several different pronunciation guides before getting to an extensive list of definitions, often very specialized and obscure ones - politics, engineering, medical, economics, finance, programming, philosophy, etc. Plus derived terms, related terms, translations, references, etc.

Definitely errs on the side of being complete, but a bit overwhelming sometimes.

Summary

Just my thoughts on dictionaries and how good they can be and how bad some have gotten by going on line, which should make them even better. Find whatever works for you.

Migrating away from Github

kp@bit-101.com (Keith Peters) — Sun, 07 Sep 2025 06:00:00 -0400

I am no longer relying on github for code hosting, version control, pages, anything else. I’ve fully moved over to Codeberg.

I’m not going to go into the reasons why I moved. Not here at least. If you are interested, I did a write up in my last remaining repository at github: https://github.com/bit101/farewell.

TLDR, it’s the whole push on AI there. Multiple angles, and I’m just not in alignment with where they are headed. Codeberg seems nice.

This post is more to let people know where my stuff is, and to give a bit of a story about how I made the move.

The Moving Process

When I decided to move, I had something like 123 repos on github. So, no, it was not a decision I made lightly. It took me weeks to move everything over one by one. Codeberg is great though. they have a migration function that lets you pull in your code from a bunch of different places.

You choose github (or whatever service you are coming from) and fill in the details.

You’ll need to generate an access token on github and use it here. Tip: make sure you specify read access on just about everything when making the token so that Codeberg will be able to grab what it needs.

BTW, don’t forget to set up your ssh key on Codeberg. I’m going to assume you know how to do that. No different than github or anywhere else. Just figured I’d say it so nobody accuses me of skipping such an important step.

Another tip: get the http address of the github repo, not the ssh address. If you grab the wrong one, it will tell you though. I did it a bunch of times.

And you can just migrate the data (all the commits) or stuff like pull requests, releases, labels, issues, etc. If all goes well, which it generally does, your repo is now on Codeberg.

Then you can go into github and archive or delete it.

Updating your local clone

Now, assuming you had the repo cloned on a local machine, you’ll need to update that so it pushes and pulls to Codeberg now.

Go into the root folder of the repo and type:

git remote set-url origin <codeberg repo address>

You get that url from Codeberg, the same as if you were going to clone it. Something like ssh://git@codeberg.org/bit101/mymigratedcode.git. Here, you’ll almost certainly want the ssh address, unless it’s just read only.

Now you should be able to type git remote show origin and see that the url is updated. And of course pull, push, etc.

Stuff that didn’t make it

There were a lot of repos that were just old code that I know I’ll never touch again, projects I started that I never continued, libraries that probably won’t work with any modern code, early versions of projects that were more fully fleshed out in a later attempt. Most of that just got deleted. There were also a bunch of JavaScript libraries that probably had some interesting things in them, even though I’m not actively using them. I combined all of them into one repo and migrated that. In the end I wound up with 60-something repos. So things got cut almost in half. No regrets at all on any of the stuff that went away.

Go Code

You may not run into this, but a lot of my work over the past eight years has been in Go, which has it’s own dependency managment system that’s based on where the code lives - generally a git repo. So, for example, if I wanted to pull in my bitlib library into a project, I’d type go get github.com/bit101/bitlib. Within that library is a go.mod file that will look like this:

module github.com/bit101/bitlib

go 1.22.1

And there will be another go.sum file that is generated when you actually meet the dependencies by pulling in a specific version of a library.

github.com/bit101/bitlib v0.4.7 h1:+H/UNQmEMPkNk2CIih0hVWnL1GP4lEH9BnZHupZ+Pns=
github.com/bit101/bitlib v0.4.7/go.mod h1:d0L7f0E8a21VD7cDuVuIsNyD4yqqxZsvgKxh5Nx433E=

Finally, each file that imports anything from bitlib will have stuff like this:

import (
	"github.com/bit101/bitlib/geom"
	"github.com/bit101/bitlib/random"
)

So… for each library I ported over, I had to go through all the files and replace github.com/bit101/... with codeberg.org/bit101/.... Then, for each project I ported over I had to do the same. Actually, I generally just deleted the go.sum file and ran go mod tidy which pulls in all the dependencies and recreates go.sum.

After all that, I did a bit of testing to make sure it all worked. OK, to be honest, there are still a bunch of older projects I haven’t cleaned up yet, and some little used libraries that will need some updating. But I can create a new project and it works well from scratch, and it just takes a few minutes (usually) to update an old project.

Actually, in the middle of doing all of this migration I learned that it might be possible to set up a sort of “vanity url” which points at the actual git repo where your code is. So if you change repos, you just need to change where that vanity url points. I assume that would live on a domain I own, so I could do something like go get bit-101.com/bitlib and that would pull the code from wherever it actually lived. I still need to look into this, but no rush at this point. As long as Codeberg doesn’t abandon its mission any time soon.

Vim plugins

In recent years, I made a few vim/neovim plugins. I’m just going to call it “vim” though I’m referring to both. There are several plugin managers for vim. Generally these also operate on a github repo. I happen to use vim-plug where you specify the plugin with your github username and repo name like so:

Plug 'bit101/bufkill'

Most of the other plugin managers do something similar. The manager assumes github and looks for github.com/bit101/bufkill - which works fine if your code is in github. I don’t think it recognizes any other git sites by default. Luckily, you can just put a full url there as well, like so:

Plug 'https://codeberg.org/bit101/bufkill'

You might run into other situations with your languages or platforms that depend on github-specific paths. They’ll certainly be fixable.

Open Source

One thing I wasn’t fully aware of when I started moving things over was that Codeberg fully embraces free/libre/open source software, whatever you want to name it. If you’re creating a startup and have a bunch of private, proprietary code that’s copyrighted and trademarked, move on.

Codeberg is run by a non-profit association with an explicit mission of advancing the creation and development of free content and free and open-source software.

This mission is shared by Codeberg’s contributors, donors and association members. We, and our servers, work tirelessly to provide you with our free service. However, there is one thing that we expect from you in return: To contribute back to the ecosystem by attaching a suitable license to the works that you put out in public, so as to let others reuse and adapt your works.

This was, in general, not a big problem for me. In fact, I’m wholeheartedly on board with that! Most of my repos were already open. But, I wound up with about eight private repos that I didn’t want to open up. Things like my dot files, a few client projects, some game projects that I might want to do something with, the source for a few of my sites and my personal wiki. Codeberg does say that if you are contributing to free software, they’ll allow you 100MB space for private repos. All told my private repos added up to about 65MB so I was OK, but didn’t want to worry about it and constantly have to check file sizes.

By the way, Codeberg doesn’t get super heavy about this. It’s more like, “here’s how we look at things, please abide.” I didn’t see any threats about taking stuff down. In fact, they say if you really need a bit more space, get in touch and they’ll hear you out. This is about as heavy as I see them get:

If you do not contribute to free/libre software (or if it is limited to your personal homepage), and we feel like you only abuse Codeberg for storing your commercial projects or media backups, we might get unhappy about that.

Forgejo

Anyway, what I decided to do with my private repos is to create a private Forgejo install on my NAS. Codeberg itself is a fork of Forgejo, which is a fork of Gitea. I got this installed and moved my eight private repos over there. This is pretty neat to be honest. In fact, this very site is a Hugo static site, that is being stored in that very Forgejo install.

I will admit that I’m a bit worried about self-hosting important files like this. But my NAS has:

A ZRAID1 configuration with one drive of redundancy
Regular snapshots that are saved for two weeks
A daily incremental backup to another device using Restic, with weekly pruning, keeping 7 daily, 5 weekly, 12 monthly and several yearly snapshots
A daily sync to a cloud provider
Regular SMART tests on all disks
Regular scrubs of the datasets
All of the above fully automated with monitoring and alerting if anything goes wrong

I’ve never had such serious data protection in my life. And I have some old Flash source files that are 27 years old and have bounced around a dozen or more computers and a Dropbox account. I think I should be OK. But yeah, I still worry.

Licenses

I took heed of Codeberg’s statement mentioned above:

…there is one thing that we expect from you in return: To contribute back to the ecosystem by attaching a suitable license to the works that you put out in public, so as to let others reuse and adapt your works.

A lot of my repos had licenses on them, but… it was probably less than half. With no license, they’re not really open source or free software. So they all have licenses now. I went with AGPL 3.0 for all the new ones. It’s a bit on the strict side. I used to use MIT most often, but never felt great about it. MIT’s license is basically, do whatever you want, just don’t come crying to me. After seeing so much abuse in recent years where billion dollar companies are using all this free and open source software and not supporting the authors at all, it seemed right to be a more strict. I don’t even think much of my stuff is in actual use anyway. I imagine people peek in to see how some particular effect was done, learn and adapt their own code to that. Which is fine with me in general.

Oh, and for the record, all the Codeberg quotes above are from their FAQ page.

Sizes

One final thing I wanted to check on was the sizes of my repos. Since I’m not dealing with a huge billion-dollar corporation, I didn’t want to be needlessly wasting space on their servers. The vast majority of my repos were under a megabyte. Most just a few kilobyes. But there were a few that were… massive. Mostly where I had inadvertently checked in a bunch of images or even videos, or other large files. I’ve cleaned up the worst offenders. There are still a half dozen or so that are a few (less than five) megabytes. I’ll get to those next.

Gitlab

OMG, after all that, I remembered I had a gitlab account. I checked and there are 35 more repos in there!

But most of them are just junk. Ideas I started and for some reason threw them up there rather than github, but never did much with. I think I can probably trash most of them. I’m not as worried about gitlab as I was with github, but they are also very much aboard the AI train. It’s billing itself as an “AI powered devsecops platform”. There’s just nothing in that description that I care about. I just want somewhere to put my code, keep track of changes and make it available to anyone else who might benefit from it. So I might as well clean up my tracks there too.

Remarkable 2

kp@bit-101.com (Keith Peters) — Thu, 28 Aug 2025 18:20:48 -0400

I think my last “review” of the Remarkable was a bit too high level and focused more on strategy decisions the company made that I didn’t agree with. This one will focus more on features.

I made a bunch of notes (on the Remarkable) about all the different features I could think of, and whether I thought they were good (and why), OK, or needed some improvement. Here we go!

The Pen Tools

Overall the drawing / writing tools are very nice. The calligraphy / fountain pen one is pretty amazing, though I don’t really use it all that much. I mostly use the fineliner pen on the thinnest width in black for writing. It’s not pressure sensitive, so you get a consistent line. The ballpoint pen is very similar but can fade or get thicker depending on your pressure.

I would like more variety in line widths. Most tools only give you three widths, thin, medium, thick. I’d like to get somewhere in between thin and medium on the fineliner.

The highlighter is the other one I uses a lot. It only has a default width, and only one shade of gray. There are other colors but on a non-color screen, they all look pretty much the same for the most part.

If you draw a line, rectangle or other quadrilateral, triangle, circle or ellipse, and then hold at the end, it will create a perfect representation of that shape, replacing the sketchy version you made. If you continue holding, you can scale and rotate it. I use that all the time for drawing borders around text or even making “check boxes”.

The eraser is good. You can tap on the eraser tool and erase exactly what the eraser touches, or you can switch to a selection eraser which erases what you select. Or, if you have the Marker Pro, you can flip the pen and erase with the opposite end. That’s good for large, quick erasing, but not as accurate. One thing I miss from the Onyx Boox drawing tools is a vector eraser - where any line you touch at all with the eraser will be removed.

The select tool is very nice, though it did take a while to realize you have to fully select around whatever you want to select, not just intersect with it. This can get tricky if you want to select a certain amount of text and move it. I’ll often miss a few random strokes.

When you have a selection, you can scale and rotate, copy, cut and paste, even across documents. Scaling always keeps the original aspect ratio. It would be nice to stretch in one axis or the other, but no big deal.

Typed text

You can enter typed text via the on-screen keyboard or the official keyboard cover or if you want to hack a bit, with an external usb keyboard.

I’m not really interested in this, but tried it out, and I’m glad I’m not interested in it because it’s kind of bad.

I expected to be able to place a text box somewhere in the page and type into it. And then select and move that text around or rotate and scale it, like in every other drawing program you can add text to.

But it’s not like that. It’s really more like a wordprocesser where you just type on the whole page. You can then add drawings or handwritten text, but the two don’t really mix well. If you draw and then type text, or vice versa, and then type or delete lines, it moves the handwritten material around.

If you delete a bunch of text, the handwritten material may scroll up and right off the top of the page. I lost a paragraph or so of handwritten notes that way.

I’d suggest if you do one or the other on a given page. Don’t mix.

Personally, I just avoid typing text. It wasn’t something I planned to use anyway.

You can also select handwritten text and convert it to typed text. This seems to work really well, but again, not a big use case for me, at least right now. Maybe I’ll find a reason to use this in the future.

Gestures

There are various gestures you can use with your fingers.

Two finger tap - undo
Three finger tap - redo
Left/right one finger swipe, change pages or create new page
Single finger swipe down, extends the current page
Two finger movement pans the document
Two finger swipe left/right on pdf creates a margin area to take notes
Swiping up or down from the top or bottom edge with one or two fingers navigates you to different parts of the app - file manager, favorites/recents, page overview
Standard pinch to zoom in/out

Most of these are very good. But get a little confusing, and sometimes don’t register 100%, but not too bad. I am not a fan of the continuous page thing and for a while I kept accidentally extending my page by swiping vertically when I wanted to change to the next page.

Layers

Each page can have multiple layers, like many graphics tools. The bottom layer is hard coded to be the background, which can be a template for notes, or the pdf itself if you’re viewing a pdf.

As mentioned, I wish custom templates were supported officially, though working around it isn’t hard if you are technically adept.

Beyond templates, I haven’t used layers at all, but they seem fine.

Page Management

In a single doc you can see an overview of pages, add pages, rearrange them, delete them, etc. All works fine and fairly intuitive.

You can also choose what to show as a thumbnail for the doc - the last page you were looking at, or the first page of the doc. I’ve made custom cover pages for most of my docs that clearly show what they are in the doc overview screens.

The Tool Bar

Pretty straightforward with the tools you mostly use up front. A few more features hidden with a three-dot menu. The whole thing is easily hideable.

It would be nice if there were some customization possible, but what you get is fine.

Search, Tags, File Management, etc.

You can search typed text, tags, document names and that all works fine.

In the latest beta, you can even search handwritten text.

THIS IS MAGICAL!

I can search all my documents for a single word and it can figure out my crappy handwriting and display all the results with amazing precision.

I haven’t used tags at all yet. And haven’t really needed to search much yet, but I imagine after I have a lot more content, this will all be super useful.

The whole file management system is very straightforward and fine. You have nested folders with documents in them. What more do you need?

Linux Stuff

You can SSH into the device - via USB at first, but once you’re in you can activate WIFI-based SSH. You can use passwords or ssh keys. Then you have a Linux file system and tools, like vim, nano, all the usual utils. Wonderful. You can copy files to or from it with sftp, scp, rsync. Lovely.

Sadly, though there are mobile apps for Android and iOS and desktop apps for Win and Mac, there’s no Linux desktop app. And there’s an official Chrome extension for sending web pages to the device, but nothing for Firefox. There are unofficial ones that work fine though.

There’s also a MyRemarkable web site that lets you send documents to the device. I haven’t really used it, but it’s nice to have there.

Physical

The Marker Pro is good. The magnet where it sticks to the device is solid. The leather case I overpaid for is nice. Wish it did the auto-sleep/wake thing though. The screen size is the same as my Onyx Boox Note Air, but the Remarkable has much bigger bezels, so the device is a bit larger (though thinner).

Summary

Almost all of these features work great and are well thought out and have decent defaults. My biggest complaint would be that I’d like a bit more choice or customization of things, like line widths and more shades of gray. The typed text set up is awful in my opinion, but I never planned to use it anyway.

Remarkable

kp@bit-101.com (Keith Peters) — Sun, 24 Aug 2025 12:41:03 -0400

Well, I finally did it. I went and got myself a Remarkable 2 eInk tablet.

History

I’m a big fan of eInk. I got my first Kindle in 2009 and have been doing most of my reading on eInk tablets ever since. After a few Kindles I wanted something a bit larger and was really interested in writing, drawing, sketching on eInk. So I got a Onyx Boox Nova tablet. That device was around 8" and included a stylus for drawing. A bit later I wanted something larger and got on Onyx Boox Note Air - a 10.3" tablet with drawing capabilities. I loved that but silly me, I wanted something a bit more portable - pocket sized, so got an Onyx Boox Poke 3. This had the same screen size as the Kindle Paperwhite I had just before that, but much smaller bezels, so utterly pocketable. Sadly, it was so pocketable, that I sat on it and cracked the screen. I replaced it with an Onyx Boox Go 6, which is essentially the same device with a new name.

The Note Air is great for reading PDFs or books with code or lots of diagrams. I’ll still use it often for that, but most of my day to day reading is done on the smaller Go 6. The writing/drawing capabilities of the Note Air are pretty good. I’ve sketched out a ton of ideas, often when I’m doing some creative coding or trying to work out some trig problem. I did try and use it a few times for organization, general note taking etc. But it started to show its weaknesses there. I’d say it’s a great ebook reader with an OK drawing/sketching ui.

Enter the Remarkable

I watched the coming of the Remarkable with some interest but never saw it as compelling enough to take the plunge. Although it can display PDFs and epub books, it doesn’t have many advanced ebook reading functionality. It’s bare bones in that area and doesn’t have a ton of storage, so I was happy with my Onyx devices. It seems like the drawing/writing aspect was really good though. But would the upgrade be worth the price? Did I want an eInk device as my note taking and organization device? It wasn’t a stunning success on the Note Air, so I wasn’t sure.

But then more and more people I knew got Remarkables and raved about them. Finally a good friend / coworker got one a few weeks ago and it really got me looking deeper. I read a lot and watched a lot of videos and decided it wasn’t for me.

But I kept thinking about it.

Then read some more and watched some more and bought one. It arrived earlier this week.

Warming Up

I knew I wanted to go all in on the organization aspect of the Remarkable and after digging around, I purchased a customizable PDF workbook from On:Planners. I got the “Ultimate Organizer for Remarkable”. It wasn’t all that expensive, and the cool thing is that you can customize the hell out of it. Which pages to you want to show for each year, quarter, month, week and day? A whole bunch of other templates and layouts you can add on to the end of it. When you’re done planning your planner, you export the PDF. Don’t like it? Go back to the config page and change it and export again, unlimited times.

I created the planner I want and although it is designed for the Remarkable, it fit well on my Note Air. So I made two copies, one for work and one for personal and started using them there for a few days while waiting for my Remarkable 2 to ship.

Organizing

I hate to sound dramatic, but it’s been kind of life changing. I’ve gotten in the habit of writing everything down and have realized how often I have a thought and don’t write it down and five minutes (OK, one minute) later I’m like, “what was that thought?”

Now I’m trying to catch those as they come up. If my Remarkable is not at hand, I try to capture it some other way. I was using Google assistant voice recognition to make notes, but I told it I wanted to “buy dental floss” and “charge my Garmin (running watch)” and it made notes to “eye dental plus” and “George Gorman”. No lie. I wonder if they’ve dumbed down the original assistant so they could push people to Gemini. I don’t remember it being that bad. Anyway, I started carrying a small paper notebook and Space Pen in my pocket if I’m out and about, just for capturing stuff.

Capturing all of those things I want to do and having them in a good, easily navigable system has been huge in improving my focus and productivity - both at work and personally.

Notes

I started taking a lot of notes. Like in meetings. After doing this all week I realized something - notes are not super useful for the future. How often does anyone really go back and read old meeting notes? But they are super useful in the present, helping me to stay focused and more fully understand what’s going on. If I don’t know what to write about something that was said, I find I usually don’t understand it and I can clarify it then and there. And the act of writing it helps me remember it better even if I never look at the notes.

But… the Remarkable?

Oh yeah, that’s what this is about. So I was doing all this organization and note taking all week - first on the Note Air, and then on the Remarkable.

I have to say… it was night and day. The Note Air is fine, but very klunky in terms of navigation. PDF workbooks (that you write on) are in one part of the device, but notes, which can have layers and a template, live in another part of the device. And the UI for drawing/writing is a quite a bit different in the two apps, so it gets confusing. Palm rejection is not great, so I was always accidentally navigating away from where I was on the Note Air - super annoying.

Pretty much all of those complaints went away on the Remarkable. All docs, whether notes or PDFs just live in the pseudo-file-system. And you get access to recents and favorites as well, so swapping to some other document you want is generally pretty easy.Writing and palm rejection is fantastic.

Now every review you will see on the Remarkable gushes about how it “feels just like paper!” and “it even sounds like paper (holds microphone up to the tablet and writes, so you can hear the scratchy sounds)”. I’m surprised nobody is reporting it smells like paper. Whatever. The pen and screen are good and it’s not like you’re writing on glass, so that’s nice. I could care less what it sounds like.

Complaints

Some of these are minor, some more serious.

This is a very premium device.

I mean that from a cost perspective mostly. Yes, it has a nice build and good components, but it’s freaking expensive. Especially considering how much of a single use device it is. I’m not complaining about the single use aspect. I like that, but it costs a LOT for a single use device.

Not just the device is expensive. The cases, OMG. The case with a keyboard in it is like $200. The leather one is like $170. The cloth case (CLOTH!!!) is like $130. WTH? They are nice cases. The device snaps into them solidly with a magnet. But $130 for a piece of cloth wrapped around a piece of cardboard with a magnet in it?

As it is, I bit the bullet and got the leather case and upgraded pen. Spent a lot more than I should have, but YOLO.

Lack of expected basic features.

I’m not talking about fancy bells and whistles, but things that just surprised me.

None of the cases support wake/sleep on open/close
There’s no light on the device (Remarkable 2, that is. The newer color Paper Pro version does have one).
Pretty small storage (8 GB) and no way to upgrade.

None of these are deal breakers, but it really surprised me that a device in such a premium price range would skip all these things.

Lock in.

I have no interest in the keyboard case, but it’s the only supported way to use a keyboard with the device. There’s no Bluetooth and USB keyboards are not recognized.

Since the Remarkable runs on Linux, smart people have written programs to make using USB keyboards possible, but you gotta be a bit tech savvy. Supporting external keyboards is something they easily could have made possible, but they chose not to in order to sell their own add on.

Lock in.

I know I already said that, but there’s more. Templates. The Remarkable comes with a good handful of templates for various grids, lines, planners, etc. If you sign up for the paid Connect service $29 per year, you get access to a bunch more.

But there is no supported way to add your own templates. No supported way. Again, if you have the know how to SSH into the device, navigate a Linux file system on the command line and edit a JSON file, adding your own templates is a piece of cake. And if you do all that, your custom templates will get wiped out when you update the device, so be prepared to set up a way to copy them over again.

I’ve already made three custom templates that I use every day.

Lock in.

Yup, I said it again. This time for the syncing system. If you register your device, it will store all your documents in the cloud. With a caveat. Any backed up document that hasn’t been edited in a certain amount of time (50 days I think) will be removed. It won’t be removed from your device, but it will no longer get backed up. Of course, you can overcome this limitation by signing up for the Connect service ($29 per year).

You can also back up docs to Google Drive, One Note, Dropbox. But I think this just backs them up as PDFs, not the editable native format. I don’t plan to go that route, so I don’t know the details for sure.

I’d kind of rather they just made the backup service paid. You pay, we back up. You don’t, we don’t. But to do that, they’d have to offer some kind of local backup functionality. That would be nice, honestly.

Instead, if you don’t pay for Connect you get a confusing half-assed free backup service where you never know what’s backed up and what isn’t. That is horrible.

Once again, if you have some Linux experience, it’s super easy to SSH in to the device and find out where the docs are. I set up an rsync script that copies them to my laptop. Piece of cake.

Verdict

Again, the complaints about missing basic features (auto sleep/wake, light, storage) are really more surprises. I can live without them.

The lock in stuff is more concerning. The fact that they explicitly made simple things “impossible” (local backup, custom templates, external keyboard) in order to sell exclusive services and addons is very disappointing. For the premium price they are charging for the device, they should make these things possible to someone who is not a complete Linux nerd. It makes me question the company’s attitude towards its customers and gives me concern over what they will lock down next.

Luckily, I am a Linux nerd! So I can overcome all of these, I have my custom templates and local backup. I don’t need a keyboard but if I change my mind, I can hack one in. Honestly, once you dive into the Linux aspect, you realize it’s a great hacker device!

So, in spite of sounding super negative in all of the above, I’m happy I got the the Remarkable. No buyer regret. It’s a keeper.

In short, I love the device itself but I question some of the decisions of the company that made it.

I’ll probably post occasional tips or tricks I discover. Anything that’s not already been mentioned by a hundred other Remarkable users.

NAS Part II

kp@bit-101.com (Keith Peters) — Sat, 23 Aug 2025 16:50:00 -0400

In my last post (I have a NAS!) I talked about the NAS I created with TrueNAS. This has been fantastic. I wanted to write more because:

A. I wrote too much the first time but still didn’t cover everything.

B. I’ve done more stuff with it.

Catching up

In the first post I talked about the initial setup, replacing one of the drives, the apps I installed, monitoring, and syncing to devices.

A couple more things about all that, and things I’ve done since then.

SyncThing errors be gone

I was still having occasional glitches with SyncThing. Not so much conflicts, but not connecting right, not updating, getting into weird states, etc. The thing that finally fixed it all for me was realizing that Android’s file system is not the same as what’s on the NAS or my other computers. While I kept ignoring dynamic manifest files like .sdr for KOReader, what really heped was disabling syncing of file permissions, and making sure that it was also ignoring extended attributes. I went through days of things not syncing right. But once I got rid of permission syncing, it started working perfectly and I don’t think I’ve had a glitch or conflict in weeks.

OMG Lyrics!

One added, unexpected, and kind of magical thing that happened was when I added a music lyrics fetching plugin to Jellyfin. It took a bit, but it scanned my whole library and downloaded lyrics for the vast majority of files that actually have lyrics (non-instrumental songs). That was nice. But magically, those lyrics synced to my two portable digital audio players, putting those lyrics files on those devices too. And suddenly I have full lyrics support for most of my music library everywhere I use it. This is amazing. A while back I wanted to get some lyrics, but doing it by hand… impossible. I paid for a MusixMatch subscription for a bit, but didn’t really like it all that much. Now it’s just there, and backed up forever.

Real backup

There’s the “3-2-1” backup rule: 3 copies of your data, 2 different kinds of media, 1 off site. I have an offsite sync to pCloud. But what concerned me with that was that if I accidentally deleted or corrupted a bunch of files, they’s sync to pCloud and delete/corrupt there too. There’s some level of history there, 30 days or something, but it didn’t feel great. I wanted a true incremental backup.

My old “home lab” HP slim device was sitting unused. It only has space for a 2.5 inch drive though. So I got another 3.5 inch 4 TB drive and put that into the NAS. Did the whole disconnect, replace, resilver thing I did earlier. Worked perfectly. Now that has three 3.5 inch 4 TB drives and I put the 2.5 inch I removed from that and put it in the other HP.

I installed TrueNAS on that too! But very minimal. The only app I put on there was Tailscale, but I’ll probably never use that there anyway. Then I set up restic on my main NAS using the new backup NAS as a target. This is great. It does an incremental backup every day. Once a week it does a prune, keeping 7 daily backups, 5 weekly, 12 monthly and some number of yearly backups.

A cool thing about restic is that you can just mount your backup on any computer that has restic installed. I can do that from the NAS itself or even my laptop. Then I can browse any saved snapshot as if it were a large filesystem. I can mount it from my laptop, go into a snapshot I did months ago and grab a file that has since been deleted. That’s super useful. Of course, there’s full restore capabilities as well.

Frankenstein’s NAS

I mentioned that I now had three 3.5 inch drives in the NAS. But that machine only has space for two 3.5s and one 2.5. After much mental engineering, I got a hard drive bracket that I put on one of the internal drives that elevates outside of the case, giving me a place to install another 3.5 inch drive. Of course this means I can’t put the cover on the case. So be it.

Similarly, I discovered when I put the 2.5 inch drive in the backup box, it’s a thick drive, and the 2.5 inch mount there doesn’t support that. I did a bit of surgery and got it in there, but now that cover doesn’t fit either!

Even with the covers off, as I was monitoring the drives, they were getting fairly warm. They were getting into the high 40s C. Not sure if any made it to 50 C, but some hit 49 for sure. That’s kind of borderline from what I’ve read. 60 C is heading into the danger zone. 50 … probably OK, but not great. This was on both machines. Doesn’t help that we’re in the middle of a two-month heat wave here in the Boston area.

Since these are custom slim form factor cases, there’s no space to mount fans, and no extra headers to power them anyway. So I picked up a couple of USB powered fans. There’s plenty of USB ports! Jockeyed these into rough position so they are blowing on the drives and voila! They all stay in the mid 30s C mostly. Even lower on a cool night. That’s with the fans on low. An extended drive activity session might get them up to the low 40s. I’ll take it.

Am I proud of this mess of computer parts held together with duct tape and paperclips? Damn right I am! But more on that later.

MeTube

One more thing I wanted to do was try to cut back my YouTube doom scrolling. There’s plenty of stuff I want to watch. Some channels I subscribe to, various things I want to learn about, etc. But all too often I’ll go there looking for one thing and an hour later I’ve watched twenty stupid videos about god-knows-what. I don’t even remember, but some of them had to do with cute animals.

I set up another video library in Jellyfin. This is simply a folder on the NAS that it will scan for videos. Then I installed MeTube on the NAS. It’s one of the TrueNAS supported apps. Then I installed a MeTube extension in my browser. Now I can right click on a video thumbnail and there’s a context manu item, “Send to MeTube”. Or if I’m on a video page I can just click the plugin icon and it does the same thing. A few minutes later, the video appears in my Jellyfin library.

This is my Jellyfin setup for media. I’m pretty happy with the styling I did here. :)

And this is what’s in there right now:

Generally I try not to watch anything ON YouTube now. I’ll scroll through and send what looks interesting to MeTube and then I have a nice playlist to go through later. For the most part I delete the videos after I watch them unless it’s something I’d really want to keep around.

Photos

I also have a ton of photos going decades back on the NAS. I’ve previously used Immich which is a supported app on TrueNAS, but I could not get it to work, and to be honest, while it’s great, it’s pretty heavy, with AI image recognition and categorization, etc. I just wanted to have my pictures available. Jellyfin supports photos too, so there they are. I put a ton of work into cleaning up duplicates and stuff I didn’t need to save, and arranging them all by year and month. They’re in the best shape they’ve ever been in at this point, and all backed up in multiple spots. This is good.

Maybe someday I’ll give Immich another shot. Would be nice to have some search capabilities.

Future

As I said in the beginning, this whole project was a learning experience. I didn’t want to invest in an expensive prebuilt Synology or something, not knowing if that’s what I needed or wanted. Boy, have I learned a lot!

I’ve put enough time and energy (and not a tiny amount of money, though not all that bad) into this, so I’m going to run it as is for a while. But I think next year some time I’ll definitely be investing in a NAS case, maybe the Fractal Design Node 804

And a low-power/efficient ITX motherboard/CPU.

Happy to hear any suggestions.

Command Line Tricks

kp@bit-101.com (Keith Peters) — Mon, 28 Jul 2025 18:57:21 -0400

Just a bunch of useful command line scripts to do various things. These are all prime candidates for aliases.

For example, I alias du -d 1 | sort -hr | column -t -R 1 -N Size,Folder -l 2 to dui. The i doesn’t stand for anything, but it’s easy to remember!

Find examples

Find files by name.

Can add -type f or -type d to only look for files or directories.

Simple glob patterns, not regex.

find . -name "*foo*"

Case insensitive

find . -iname "*foo*"

Regex

find . -regex ".*foo.*"

Matching multiple names with AND.

find . -name "*foo*" -and -name "*bar*"

Shorter…

find . -name "*foo*" -a -name "*bar*"

And is implied.

find . -name "*foo*" -name "*bar*"

Matching multiple names with OR.

find . -name "*foo*" -or -name "*bar*"

Shorter…

find . -name "*foo*" -o -name "*bar*"

Reject file name matches. Can use -and or -a here too.

find . -name "*foo*" -not -name "*foo-bar*"

Find files by extension.

find . -name "*.ext"

Count of files only.

find . -type f | wc -l

Count of folders only.

find . -type d | wc -l

List all file extensions in a folder.

Simple

find . -type f | sed -e 's/.*\.//' | sed -e 's/.*\///' | sort -u

With counts, and sorted

find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -gr

Find all empty directories.

find . type -d -empty

Find all empty files.

find . type -f -empty

Find all directories that don’t have nested directories (leaf directories).

-links = 1 for current dir, 1 for parent dir, 1 for each nested dir.

find . -type d -links 2

Find all hidden files.

Hidden

find . -type f -name ".*"

NOT hidden

find . -type f -not -name ".*"

Find all hidden directories.

Hidden, ignoring current dir “.”

find . -type d -name ".*" -not -name "."

NOT hidden

find . -type d -not -name ".*"

Delete matches.

Be careful here!!!

find . -name "*foo*" -delete

List 10 largest files.

find . -type f -exec ls -lhS {} + | head -n 10

Other

Simple list of directory sizes.

dui -d 1 -h

Folder sizes sorted and tabulated. Change `-d` to control depth.

du -d 1 | sort -hr | column -t -R 1 -N Size,Folder -l 2

Run new command with arguments from the last command.

new_command $_
# or
new_command !$

Recall a specific argument from the last command.

# single
new_command !:n

# series
new_command !:n-m

# arbitrary
new_command !:n !:p

I Have a NAS!

kp@bit-101.com (Keith Peters) — Sun, 20 Jul 2025 11:06:27 -0400

Background

I’ve had some kind of home server / home lab for many years. Probably going back to 2008-ish. Generally this has been just a desktop computer running some Linux distro or another and connected to the internet. For many years, this was just an extra computer I could connect to, try things out, store files I didn’t need on my laptop (it usually had a decent amount of storage). I also often used it as a desktop computer for doing things if I happened to be sitting at the desk where it was.

I used these desktops for somewhat of a “backup” but it was really just an extra copy of files in case my laptop fell off a cliff or something. A few years ago I started doing more structured, scheduled backups with restic. This worked out pretty well, but at some point it failed and I didn’t get around to fixing it for a while.

As for external connectivity, at first I was just opening up a port on my router. Then one day I got a glimpse of the traffic that was trying to access that port and shut it down quick. I dodged a bullet there. Then I set up Wireguard. I had an vps in the cloud that was the main connection point. That could route traffic/services to the home server, or any other device on the Wireguard network.

This all worked well for years and I wound up putting up various self-hosted services there. Music servers (Subsonic or compatible) and video servers (Plex, Jellyfin) mainly, but some things I created myself, and a lot of different self-hosted apps that I tested but didn’t end up keeping around very long.

A few years ago I rebuilt my home desktop with a pretty powerful setup, somewhat of a gaming rig. I decided this was overkill to run as an always-on home server. So I got a thin client HP ProDesk device for cheap.

This is a tiny device. A bit bigger than an external DVD drive. It has room for an NVME SSD and a 2.5" drive. I plugged in a couple of external USB drives and I’ve been running that as my home server for over a year. It was running Ubuntu server, so my first headless server. But the USB connections were a bit funky.

This is when I started eyeing NASes.

Synology? QNAP? DIY? What to do?

I wasn’t sure what the NAS experience was. So many questions. Is this something I needed? Was a headless server with storage fine? Did I really need a RAID? What apps could I install on which types of NAS?

Prebuilt NASes are expensive, even before adding drives. Even a 2-bay NAS is not cheap. And not expandable at all. Want to add a drive or a couple more? Nope. Get a new NAS with more bays. I didn’t want to sink many hundreds of dollars into this before I even knew whether it was something I needed.

So I started looking at the DIY route. After reading many articles and watching many videos, I settled on a used HP EliteDesk 800 G3 SFF for under $90 US on Amazon.

The SFF is for “small form factor”, but it’s a lot bigger than the ProDesk. Specs:

Processor: Intel Core 7th Gen i5 7500 3.4Ghz
Memory: 16GB DDR4 RAM (2x8GB, leaving 2 slots open for more)
Storage: 128GB NVME SSD
Bays: 2x 3.5" drives, 1x 2.5" drive

It came with Windows 10 installed, but I never even booted into that. I was looking at Unraid, TrueNAS Core and TrueNAS Scale and went for the last one. Installed Scale and it booted up. I had two 4TB external HDDs that I broke open and extracted the 2.5" drives and put them in the HP. It recognized everything and I created a pool with the two drives in a mirrored VDEV, synced some files to it, messed around with the system and got a feel for it.

But I wanted to eventually use all three storage slots and you can’t expand a mirrored VDEV, so I figured I’d do it right from the start and ordered another 3.5" 4TB drive. When that got in, I set up a RAIDZ1 VDEV. This gave me 8TB (7.14TiB) storage, with one drive used for parity.

Most of the files I wanted were on DropBox. I set up a cloud sync to pull all those files down. It was almost 1.4TiB of data, so it took a good 48 hours (DropBox does a lot of limiting). But with all that, I’m only using 18% of my storage.

Errors! Oh no!

As the data was syncing, I was running into occasional read/write errors on one of my drives. It was one of the 2.5" drives, which was a couple of years old, yanked out of that external HDD case. Not really shocking. This probably explained some of the errors I’d been seeing on my previous setup. I ran some S.M.A.R.T. tests on the drives and that same drive was consistently failing. But… the errors were not happening very often. And hey… parity disk. I’m already feeling the comfort of having a NAS with a RAID. So I let the download finish and planned on replacing the drive afterwards.

Apps!

The main functionality I wanted was:

Music server
Movie/TV server
External connectivity (VPN)
Syncing media to devices

I know what you’re thinking… “But that drive has errors! You need to fix it!” Yes, I ordered a drive, and started setting up my apps while I waited for that to arrive.

Jellyfin

For music and video I just went with Jellyfin. I switched over to Jellyfin from Plex earlier this year, when Plex more than doubled its prices while simultaneously reducing its features. I’ve been really impressed, especially with its music service. I’d been using PlexAmp for music, which is a great app on both desktop and mobile devices. Jellyfin is not quite as full featured, but does everything I needed.

Tailscale

For external connectivity I switched over to Tailscale. I’d been wanting to do that for a while, but procrastinated for months just because Wireguard was working and I didn’t want to mess with something new. It turns out that Tailscale was super easy to set up - much easier than Wireguard. And is so much better. Wireguard sets up a server/client configuration, which was why I had the cloud vps as the server. But Tailscale is 100% peer to peer. Any device can access any other device. I think you might be able to do that with Wireguard, but I never figured out how. With Tailscale, that’s just the way it works out of the box. Once this was working I realized I had no need at all for the vps anymore. All my devices could talk to the NAS directly. Come to think of it, I probably could have just set up my home server as the Wireguard server and done the same thing. I think.

My Tailscale devices - Linux laptop, phone, tablet, NAS. Still need to set up my Macbook Air.

SyncThing

There are a few devices I need to sync files to.

Two eInk e-readers (Onyx Boox Go 6 and Onyx Boox Note Air). All my books are de-DRMed and saved as epubs or pdfs on the NAS.
A high res Digital Audio Player (DAP) (Hiby R4). Thousands of flacs and mp3s on the NAS.
An Android tablet (Lenovo Tab). I also wany my books synced there, though I don’t use this much.
My phone (Pixel 7 Pro). I like my books there too.

For this I used SyncThing. It syncs the books to the readers and tablets and the music files to the DAP. It was a bit fiddly to set this up, but I got there. I used to use an app called DropSync for this - an Android app that is kind of like rsync for DropBox. It works well, but I’m no longer relying on DropBox, so needed to change this.

One consideration for this was that SyncThing wants to constantly monitor changes and sync constantly. Since all these devices are battery powered, I don’t want this happening 24-7. Also, it’s not like I’m adding books and music at all hours of every day. Maybe 1-2x a week, max. I set it up so that they’ll only connect if they are on AC power. This is great. If I add a book or album to the NAS, I just plug in the device and it will connect and sync.

Here you can see that all my devices are disconnected. By design! I plug one in to AC power, and it will connect.

And here are all my apps in TrueNAS as of this writing.

I’ll talk about Uptime Kuma a bit later.

Resilvering

OK, OK, the new drive arrived. Nothing crashed in the meantime. Time to replace it. btw, I ordered some other hardware at the same time. HP does some funky mounting and cable routing in their SFF devices. It’s cramped and you need special screws for the drives to slot in correctly. Up to now, they’ve just been kind of balanced in there. I got the screws and some 90-degree connectors, which made everything a lot better.

Following multiple forms of documentation, I first took the problem drive offline. You just do that in the TrueNAS UI. Then I went in and pulled the drive and plugged in the new one - while the system was running! WTF. I didn’t know this was possible. I know dedicated NASes do this, but figured it was some special magic hardware sauce. No, you can just hotswap a SATA drive - assuming it’s offline or unmounted I guess.

As soon as the drive was offline, my pool warned me that it was in a degraded state. This means that everything will still function, but you should be careful about doing a lot of stuff with files. You now have no parity disk and if something goes wrong, that’s that.

Next, I chose the offline drive and asked to replace it. I selected the new drive as the replacement and that took about a minute. But things were still degraded because this new drive was blank. It had to be “resilvered”. I didn’t really know what that meant, but I have somewhat of an idea now. With the RAIDZ1 array, two drive’s worth of data is spread across three drives. It keeps checksums so that if you lose one whole drive, it can use what it has, plus the checksums to reconstruct any missing data. It does that to reconstruct that new drive. It’s not just copying data, it’s calculating what needs to be there based on the other two drives and checksums. So this takes a LONG time. For my 1.34TiB of data, it took over 12 hours. But when it was done, it was back to full functionality.

Although this was a bit of a pain in the neck to do right in the first week of setting up the NAS, I’m pretty happy it happened when it did. I learned a lot and when that other 2.5" drive goes (which I’m sure it will before long), I’ll know just what’s coming.

Backups

I’ve been kind of using DropBox as an offsite backup. But I’m kind of done with it. It’s a constant upsell and pushing all kinds of other new features on you all the time. After looking at a bunch of other cloud storage providers, I’m switching over to pCloud. It’s a bit cheaper than DropBox and it’s very focused on just data storage. Lots of good reviews.

I’ve backed up my NAS there with just a straight sync. This is not really ideal because if I accidentally delete some stuff, it’s going to be deleted in pCloud too. I have 30 days to recover deleted files there, but it’s not the best in the long run. Better than DropBox though.

What I’d like to get going is an actual versioned, progressive backup. I think I might be able to use restic to do that on pCloud. Or maybe there’s a way to do it with rclone, which is what TrueNAS uses by default there. That’s the next thing I want to look into. And probably doing a local restic backup on an external drive as well. Future stuff. At least it’s somewhere else for now.

Also worth considering is that if my backup size goes over 2TB, I’ll have to figure something else out. The vast majority of that is music and movies. And I’m way more concerned about preserving my music. If I lost all my movies tomorrow, I’d actually be kind of ok with it. I might just move those to a local backup only at some point.

Scheduling

There’s a bunch of stuff to schedule in TrueNAS.

First the cloud sync / backup. I have one for my general files and another for all the app configuration stuff, which is all set up in a different data set on the NAS. These will both get synced to pCloud daily. Eventually something more robust as described as above.

Then there are S.M.A.R.T. tests for all the disks. I have those scheduled for once a week.

Finally there is a scrub task, which is a ZFS thing that I only partially understand. But I know that it examines all your stored data along with the checksums to see if anything has been corrupted. If it finds anything, it can generally correct it. So that’s nice. The schedule on that is a bit more complex based on what was set up by default. I’m not messing with that until I understand more.

I set up the timing on all that so each of these things should not be happening while another major taks is happening. I think that’s sensible. You don’t want to be syncing data while your checking your disks or data. I guess.

Monitoring

One thing I was proud of in my previous server setups was the monitoring I set up. Every service and app was checked via cronjobs. If it was running and OK, I’d ping a check I set up on https://healthchecks.io/. If it didn’t get a ping in a certain interval it would start emailing me.

There’s already some alerting built in to TrueNAS. If disks go down or have errors, if S.M.A.R.T. tests fail, and a few other things, I set up alerting to notify me by email.

But what if TrueNAS itself goes down? Power or internet goes out? For that I set up HealthChecks again with a very simple cronjob that pings every so often. If it doesn’t ping, I know about it.

I thought about doing the same for my other apps, but wound up installing Uptime Kuma. This is kind of the opposite idea. HealthChecks is like a “dead man’s drop” - it doesn’t get a ping, it takes action.

Uptime Kuma, on the other hand, pings each of my services regularly, and if that ping fails, it notifies me by email. Here’s my dashboard:

Et cetera

Just realizing I could go on for a while more, but that’s enough for one post. There will be more, probably, later.

Simple Image Sequence Viewer with mpv

kp@bit-101.com (Keith Peters) — Sat, 28 Jun 2025 10:54:35 -0400

A while back I created an app to allow myself to play a sequence of images as a video. I called it isv for “Image Sequence Viewer”.

https://bit-101.com/blog/posts/2024-06-28/image-sequence-viewr-isv/

It was decent but it had issues so I kind of stopped using it. But I still have a need for the use case. I’m often rendering an animation as a long sequence of png frames and then creating a video from them using ffmpeg. I don’t want to have to wait until the whole thing is done before seeing how it’s coming along, so I go into the folder where the frames are being created and view the first one and use the arrow keys to scroll through them in a kind of janky animation.

This morning, on a whim, I decided to see if mpv, the video player I use while developing animations, could play image sequences natively. Lo and behold! It can! The syntax is somewhat complex. It does this by merging the files into a video on the fly, so there are a few mf or merge file commands. Found a workable solution on StackOverflow and adapted that into a shell script that you can pass in a directory path to.

#!/bin/sh
mpv mf://$1/*.png -mf-fps 30 --loop 2>/dev/null

Some explanation:

mf:// is the merge file path of the files to merge. There are several formats you can use here, but I’m just using the glob version. $1 is the first argument to the script, which should be the path to the frames. You can also pass a file containing a list of the frames, or a printf type statement like image-%d.png. The glob is simple and works.

-mf-fps is the frame rate and --loop makes the resulting animation play continuously. Quiet things down by sending any output to /dev/null.

I put this in a script file called mpf so I just need to say mpf out/frames and I have a looping video playing with whatever frames have been rendered so far.

It doesn’t have all the bells and whistles that I tried to put in isv, like bouncing forwards and backwards, or watching for new frames and playing them. Here I’d need to stop and run the script again to get new frames. But it works great and is very simple. Maybe I’ll see if I can get some kind of polling going, but I assume that’s going to mean stopping the video and restarting it. Not quite as seemless as isv was on that front. On the other hand, this winds up being an actual video that I can resize, even full screen, which I couldn’t do in isv and I can still stop, start, go back and forth frame by frame, etc.

For a one line script… not bad.

POV-Ray

kp@bit-101.com (Keith Peters) — Sat, 14 Jun 2025 12:03:53 -0400

Let’s do 3D for a while!

In the past few weeks, I’ve taken a break from making music. I’m sure I’ll get back to it, but I was doing it just about every day for five plus months and I needed to step away. I’ve been thinking about dipping my toes in 3D, so why not follow that path for a bit?

I installed Blender and rendered a gray cube, then got overwhelmed by the UI. There’s all kinds of menus and all kinds of panels. And the panels have menus, and the menus have panels. And I don’t know what most of it means. I’m sure I could learn, but it just wasn’t calling to me. I’m not a big UI fan. For coding, I don’t like fancy UIs or graphical text editors. I use neovim. For graphics, I don’t use PhotoShop or any other paint or vector graphics program. I create graphics in code, in neovim. For animations, I don’t use AfterEffects or any other animation package. I create frames in code and then create movies and gifs with ffmpeg - in neovim. For music, I am not a fan of DAWs (Digital Audio Workstations) like Ableton or Reason or whatever. I like Supercollider (which I code in neovim) and trackers, and hardware synths and sequencers.

So for 3D, there was an obvious solution…

POV-Ray

POV-Ray (persistence of vision raytracer) is a 3D raytracing program that has been around for decades. I’m probably just going to call it “povray” or “pov” for the rest of this article. While it does have its own IDE on some platforms, you can also just write a .pov scene file in plain text and render it with the povray command line tool. Guess which editor I’m writing my .pov files in!

POV-Ray goes deep. Deeper than I expected when I started. There’s a lot to learn, but unlike Blender, it’s not all in your face, intimidating you. You can just write a simple scene and render it and keep adding things to it as you learn more. I know, I know, you can do that in Blender too. But when all those features are in my face, it’s distracting.

The purpose of this post isn’t to teach you povray. There are plenty of tutorials and videos going back many years on that. But I just wanted to walk through some modern solutions to rendering things with povray and the path I took. Maybe you’ll find it useful. I’ll also throw in some random videos I’ve created with pov recently. They won’t necessarily be related to the topics I’m discussing in the text at that point, but they should break things up and prevent a huge wall of text and code. Well… here’s one now!

Your browser does not support the video tag.

Here’s a pretty simple scene file that renders a box sitting on a plane, with a sky-like background, some lights and a camera.

#version 3.7;
#include "stdinc.inc"
#include "stones.inc"

global_settings { assumed_gamma 1 }

light_source {
  <-4, 4, -3>
  White
}

light_source {
  <8, 8, 0>
  White 
}

camera {
  right x * image_width / image_height
  location <2, 2, -4>
  look_at  <0, 0, 0>
}

box {
  <-1, -1, -1>
  <1, 1, 1>
  texture { T_Stone10 }
}

plane {
  <0, 1, 0>, -1
  texture { T_Stone40 }
}

sky_sphere {
  pigment {
    gradient y
    color_map {
      [0 color White]
      [1 color Blue]
    }
    scale 2
    translate -1
  }
}

Yeah, it’s not nothing, but each piece is relatively straightforward. The light sources have a position and a color (there are more complex light sources, but these work fine). The camera has a location and a point it’s looking at. It also has that first line which sets up the aspect ratio of the camera. That one is boilerplate. The box is made from two points and a texture. The plane has a vector showing it’s orientation: 1 on the y axis makes it extend out on the x and z axes. And it has a position on the y axis, -1 puts it right under the box. The sky sphere is a bit more complex, but you can see it makes a white/blue gradient on the y axis. And here’s what you get:

It uses some built-in textures that are included at the top of the file.

Can I make it procedural?

When I say procedural here, I don’t mean “generative” or “algorithmic”, aka “random”. I mean the difference between procedural code where you tell the computer what to do step by step, and declarative code, where you just describe what you want to have happen. POV-Ray scene files are declarative. You describe the scene and let pov figure out how to create it. As opposed to saying, for example…

Create a new scene.
Create a 2x2x2 box at 0, 0, 0.
Give the box a stone texture.
Add the box to the scene.
Create a plane on the x-z axes, at -1 y.
Give the plane a different stone texture.
Add the plane to the scene.
Create a light source at … etc.

This might look like in pseudocode:

scene = new Scene()

box = new Box(2, 2, 2)
box.position(0, 0, 0)
box.setTexture(Stone10)
scene.add(box)
...

Random video alert!

Your browser does not support the video tag.

So, yeah, I set out to do something like that using Go. And I got pretty far. You can see my progress here:

https://github.com/bit101/pov

In brief, the scene has an array of objects. You can create various objects and assign them different properties and when you are done, it will serialize the scene and all its objects into a .pov file that you can then render with povray. In fact, the program even took care of running povray for you and displaying the image.

Say you have a sphere object with x, y, z and radius properties. This is a struct in Go. It get serialized as:

sphere {
    <2, 3, 4> 2
}

Of course, it uses the struct properties to fill in the text that’s written to the .pov file. It mostly does that with sprintf statements sort of like sprintf("<%f, %f, %f> %f", sphere.x, sphere.y, sphere.z, sphere.radius). Pretty simple stuff.

Your browser does not support the video tag.

Where it started getting complex

First, transforms.

Transforms can be used on objects or textures, patterns, etc. You can say, for example, translate <1, 2, 3> or scale <2, 3, 4> or rotate <10, 20, 30> to transform on separate axes.

But you can also transform on all axes with one value: scale 5. Or you can transform on only one axis like: scale 2*y - scales by 2 on the y axis only.

So now there are three different transform syntaxes I had to account for. Not impossible. I started down that path, but it was getting a bit messy.

Second, textures, etc.

An object can have a texture, a material, a pigment, a normal, and a finish, which combine to define how the object looks. In fact, it can have more than one of all these things. And you can combine these in different ways. For example, you can add a pigment directly to an object, or you can put it inside a texture. And you can have pigments and textures that are predefined, like T_Stone10 and Blue used above, or you can have colors defined with rgb <r, g, b> and manually create textures on a lower level.

In short, a lot of permutations to account for in code. I did not go very far down that path.

Third, animation.

To animate in povray, you tell it how many frames you want to render. Let’s say you want to render 60 frames. Now in your .pov file you have access to a variable called clock, which will be 0.0 on the first frame and 1.0 on the last frame. So you can rotate an object around the y axis like: rotate 360*y*clock

I was happy to see this because it’s very similar to how I animate in my own 2D rendering library. The render function gets passed a percent value that ranges from 0.0 to 1.0 depending on what frame it’s rendering. I can use that in all kinds of ways to make single or smoothly looping animations.

But I struggled to find a good way to represent this clock variable in my Go code. With all other values, I could use whatever expressions I wanted and distill them down to a final serialized value. With animation, I needed to preserve the expression, 360*y*clock and serialize that.

The solution for the clock and increasingly for many of the other issues I was running into, was just to save everything as strings. And eventually the absurdity of what I was doing hit me. I was using an external program to store a bunch of strings that I wrote to a text file. Why not just write the text file directly???

So that began phase two.

Your browser does not support the video tag.

Going native and creating a build system

So I switched my workflow to just writing .pov files. This was great. When I learned a new object or property I didn’t have to create a new object to represent it and figure out how to serialize it. I could just write it.

At this point I started diving into creating a great build system. I like using make and I have F5 set up in neovim to just run make which will look for a Makefile in the current directory and execute it.

Your browser does not support the video tag. Here’s the Makefile I’m currently using:

FRAMES=60
# ZEROS = number of digits in FRAMES
ZEROS=2
CYCLIC=true
WIDTH=800
HEIGHT=600
VERBOSE=false
CONSOLE=false
ANTIALIAS=true
FPS=30

# image or video
default: image

# if you change a variable above and want to force a re-render
# default: clean image

################
# Single frame
################
image: out/out.png
	@eog ./out/out.png 2&> /dev/null

out/out.png: main.pov
	@echo "building image..."
	@rm -rf out
	@mkdir -p out
	@povray \
		main.pov \
		Antialias=$(ANTIALIAS) \
		Display=false \
		Verbose=$(VERBOSE) \
		All_Console=$(CONSOLE) \
		+W$(WIDTH) +H$(HEIGHT) \
		Output_File_Name=./out/out.png

################
# Animation
################
video: out/out.mp4
	@mpv -loop ./out/out.mp4 2&> /dev/null

out/out.mp4: out/frames
	@echo "generating video..."
	@ffmpeg \
		-y \
		-framerate $(FPS) \
		-i ./out/frames/frame_%0$(ZEROS)d.png \
		-s:v $(WIDTH)x$(HEIGHT) \
		-c:v libx264 \
		-profile:v high \
		-crf 20 \
		-pix_fmt yuv420p \
		./out/out.mp4

out/frames: main.pov 
	@echo "generating frames..."
	@rm -rf out
	@mkdir -p out/frames
	@povray \
		main.pov \
		Antialias=$(ANTIALIAS) \
		Display=false \
		+W$(WIDTH) +H$(HEIGHT) \
		Final_Frame=$(FRAMES) \
		Cyclic_Animation=$(CYCLIC) \
		Verbose=$(VERBOSE) \
		All_Console=$(CONSOLE) \
		Output_File_Name=./out/frames/frame_.png

################
# Other
################
clean:
	@echo "cleaning..."
	@rm -rf out

I won’t go into this in too much depth, but will point out some things I am proud of.

Up top are all the variables and settings that get incorporated in the rest of the build.

Running make will execute the first target, default. You can set either image or video as a prerequisite here and it will run either of those targets.

The image target will check out/out.png. This will check main.pov. If that main.pov is newer than out/out.png or if the png doesn’t exist, it will run that target and build the image. The image target will the display the image.

Similarly, the video target will check out/out.mp4 which will check the date of the out/frames directory to see if it needs to re-render the video from the frames, and will check main.pov to see if it needs to regenerate those frames.

This means that if you just spent 10 minutes rendering a video and run make again when nothing has changed, it’s not going to start from scratch. It knows that everything is up to date and will just display the existing video.

Your browser does not support the video tag.

A little from column A and some from column B

This was all working great until I started doing some more complex renders. I started algorithmically generating lots of objects using various formulas. This requires things like variables and functions. POV-Ray has these things… kind of.

“Variables” are defined with #declare statements (or #local statements). But if you want to change the value of a variable you have to redeclare it. Like so:

#declare X = 0;
...
#declare X = X + 1;

You’re supposed to use capital letters to start variables in most places. And in spite of the fact that most povray code does not need semicolons, #declare and #local statements do.

There are two function-like structures: functions and macros. Functions are also declared and can have parameters (floats only) and can return a single value (float only).

#declare Foo = function(n) {
  n * 2
};

#declare Bar = Foo(7);
// Bar will now equal 14

Functions can take vectors (<x, y, z> values) but can only be used in specific transform type cases. Otherwise, if you want to affect more than one value, you’d have to use a macro and nest #declare statements inside your function.

#macro Multiply2(X, Y, Z)
  #declare X = X * 2;
  #declare Y = Y * 2;
  #declare Z = Z * 2;
#end

#declare X = 1;
#declare Y = 2;
#declare Z = 3;
Multiply(X, Y, Z);
// X, Y, Z will now equal 2, 4, 6

Note that functions are defined with brackets { } while macros use #macro and #end. Also, function parameters should be lower case while macro params should be upper case. Odd syntax stuff like this makes variables, functions and macros rather painful.

Also note that I’m hardly an expert here. This stuff is a bit confusing and old fashioned, so I may have gotten some things wrong or overcomplicated it. I’m happy to be corrected if so.

So I wound up taking a hybrid approach. If I need to create a complex object that requires a lot of math, functions, recursion, etc., I’ve taken to building that in Go and just outputting a file that defines the object itself. I can then import that object into my scene file and instantiate it, transform it, add textures, materials, etc.

I’m not going to deep dive into that here, but if you want to take a look at an example, you can see https://github.com/bit101/pov_experiments/tree/main/250613b

And here’s the result of that:

Your browser does not support the video tag.

I don’t know how the hell I would have done something that complex in povray only.

Summary

I always feel like I have to have a summary rather than just ending off on that last thought. But I’m not always sure what to summarize. Maybe I’m just showing off. Or documenting this so when I come back to it in a few years I can be like, “oh yeah, that’s how I handled that…”. Anyway, maybe you’ll see more content along these lines. Maybe I’ll ricochet off in some other direction next week. One way or the other, I’ll be back.

Robins

kp@bit-101.com (Keith Peters) — Mon, 26 May 2025 10:19:18 -0400

Background

A few years ago I build a wood shed in my back yard. it’s just an open structure to stack logs in, with a roof to keep out the rain. I made a shelf at the top for storing sticks for kindling. I walk past this shed several times a day - whenever I leave the house or come home.

First sighting

In late April, I was coming back from a dog walk and a bird flew out of the top of the shed. When I looked closer, I saw a bird’s nest in amongst the kindling sticks. The bird was an American robin. Note, these are very different birds from what gets called a robin in the UK and other areas (the European robin). I’d often read about how cute and fluffy robins are and it never made sense to me. American robins are stern looking, business-like birds. Not ugly, but they always look very serious and slightly annoyed. Eventually I discovered that these are not the same robins others were talking about. Anyway, cool! A nest right there at eye level. I did some reading to find out what was going on there.

What I read

Some facts about robins:

Robins couples are monogomous for a single mating season. The next year they choose new partners.
They don’t live in nests. They sleep and rest in trees, on branches. The nests are solely for laying and hatching eggs and raising the babies.
The female robin does most of the nest building, though the male often helps gathering materials - grass, string, mud.
Once the nest is built, the mother moves in and stays there for the cycle of one batch of young.
The male stays nearby and keeps an eye on things, defending the nest as much as he is able.
The mother robin will lay anywhere from two to seven eggs, three or four being most common.
The eggs hatch after 12-14 days, at which point, the mother will spend most of her time gathering food (worms, bugs) and feeding them. The father might help get food for them now and then as well.
About two weeks after hatching, the babies will be ready to start flying. Soon after that, they are off on their own.
The parents then abandon that nest and build a new one for the next round of babies. They may have as many as three or four nests in a season.

What I saw

From there on, every day I would see the mother sitting on the nest. She’d generally fly away as I walked by, which I felt bad about. But that was how I left my house and came back. After a while she did start to get somewhat used to us and if we moved slowly and passed by as far from the shed as we could get, she would often stay in the nest and eye us carefully.

After a week or so, eggs!

Three tiny, perfect blue eggs.

Some notes on the photos:

Some of the photos, like the above, are very close up and taken from right above the nest. I never purposely scared the mother away. I only approached if she was not there. I held my phone up, took a photo and was out of there in two seconds. I did not touch the nest, its contents, or its surroundings.
All other photos were taken at high zoom from at least 20 feet away, sometimes from the yard, sometimes from a window.
In some of the photos the inside of the nest looks very red. That’s just a reflection of the red stain on the shed itself, where the natural light did not reach. It’s not blood!

After the eggs, the mother was almost always sitting on the nest. I imagine the did the bare minimum foraging to feed herself, but otherwise she was right there.

Right on schedule, about two weeks after I saw the first eggs, they hatched. At first we just noticed that the mother was not always sitting down in the nest, but sitting higher or on the edge of the nest.

Eventually I discovered three very tiny, ugly, pink, fuzzy creatures.

And the feeding began. At first, mother robin was still mostly sitting on the nest, but more and more often she’d be away getting food.

Within a week, they went from being amorphous lumps to looking like birds. Very hungry birds.

Sometimes when they were sated, they even started showing some curiosity about the world outside the nest.

In the second week after hatching, I was shocked at how much they had grown.

At this point, the mother was at the nest about half the time at best. She’d appear, spend a few minutes shoving food down her babies throats and fly off to get more.

And at this point I started seeing the father more often. One time while I was taking zoomed in photos from a good 20-25 feet away, he swooped down, landed on the ground right in front of me, scolded me loudly and flew away. Go Dad!

This was the last photo I got of them. They barely fit in the nest after two full weeks of growing. Then over the course of a day or two, we saw the babies fly out of the nest, one by one. Both parents were right there watching over them and screaming loudly at anything in the vicinity.

And then… literal empty nest. I can’t say I was sad exactly. But I missed looking out for them every time I went outside. The nest was abandoned and the wood shed was just a wood shed. The abandonment was just so sudden though. I thought they’d do some test flights for a few days or so, but still come back to the nest. Nope. They were gone and never returned.

I do wonder if they are ok. All three hatched, grew up perfectly and flew off on their own, so I’m assuming they are just getting on with their lives.

Anyway, it was a magical month. I’ll never look at robins the same. And every time I see one in the neighborhood, I’ll wonder if it’s one of “mine”.

I leave you with a bonus video of feeding time.

Your browser does not support the video tag.

Yamaha Seqtrak

kp@bit-101.com (Keith Peters) — Sun, 11 May 2025 05:15:24 -0400

Last week I got a new musical toy - the Yamaha Seqtrak. In this article, I’ll share my feelings about it, along with some unavoidable comparisons to my favorite child, the Dirtywave M8.

But first, some background…

A few weeks ago I cashed in on a few rewards programs and had some unbudgeted cash to spend. There was no question what I’d spend it on - something synthy. Initially I got a Sonicware Liven Texture Lab. This is a granular synthesis device. I’m going to assume you either know what that is, don’t care, or will figure it out on your own. Anyway, I was interested in granular, but wasn’t too impressed with this machine, so I returned it.

Next on my list was a 1010Music Lemondrop - also a granular synthesizer, but seems to be a far more capable device. But it was going to cost significantly more. I wasn’t sure I wanted another single purpose synth at this point. I started leaning towards some other more flexible device that can make full songs - like a groove box.

The Lemondrop is still on my list, but for now I decided on the Seqtrak. It arrived last week and I’ve been having tons of fun with it. It’s a keeper.

What is it?

The Seqtrak is a 3S machine (I just coined that) - Sequencer, Sampler, Synthesizer - just like the Dirtwave M8.

It has eleven instruments:

7 drum / percussion instruments
2 AWM2 (advanced wave memory 2 - I still need to figure out what that even is) synthesizers
1 FM synthesizer based on the classic Yamaha DX
1 sampler

Those eleven instruments come with literally thousands of presets and samples - and lots of space (500MB) to add more by downloading free sounds and packs from Yamaha, or importing or recording your own samples. The presets and overall sound design capabilities on this device are amazing. So many wonderful sounds even without tweaking them too much.

It also contains a pretty powerful sequencer where you can program in up to 128 steps per track. Each of the eleven instruments can have six different tracks. There’s a scene mode where you can play different combinations of tracks from the different instruments, and a song mode where you can sequence up to sixteen scenes to play back in series.

The good

All of the above is good. In general, it’s a fun machine to create stuff that just sounds good. It’s got mono, poly, and chord modes. For the chords, you choose the key and scale type and it picks the “right” chords for you (though you can also edit them). And it limits you to the keys of that scale, so it’s actually pretty hard to make something that sounds bad. But you can switch into a keyboard mode - or use an external midi keyboard, bypassing the limitations.

It’s powered by an internal rechargeable battery. The life seems decent and it charges via USB-C. So it’s super portable.

While the interface is pretty sparse (see next section), this device goes very deep with advanced actions and effects you don’t see on other similar devices.

The kind of weird

Synths and related devices often get criticized for “menu diving” - where you have to use a button to activate a menu in a tiny display, then twist a knob to choose an option in that menu, then hit a button, then twist a knob, sometimes several levels deep. None of that here, because there isn’t even a display!

Instead, there is a huge reliance on arcane key combos. Hold this button and press this other one, hold this button and twist that knob. In some cases, you have to hold two buttons and hit a third. Like a mini game of twister. ~~All~~ Most of the knobs and buttons are labeled, but sometimes none of the controls you are combining are logically connected to the action you are trying to take. For example, to activate Bluetooth you press and hold the “All” knob and tap the “Swing” button - of course! So there’s memorization required.

As a sort of menu, there is a list of back-lit items next two four knobs. Depending on what mode you are in, up to four of these items will light up, indicating that the four knobs will control these four parameters, e.g. attack, decay, filter, resonance. But there are more actions than there are back-lit items, so sometimes you just get the “other” item and you have to just know what the knobs are controlling.

There are also three touch sliders, labled “FX Level”, “High Pass” and “Repeater”. But what they actually do can be totally different than those labels as you go into different modes.

And in addition to the 24 “keys”, 17 knobs and one switch on top of the unit, there are 20 push buttons arranged around all four sides of the device. All labeled on the top.

So yeah, a little confusing, and a learning curve for sure. Keep that manual handy.

The app

I hope I didn’t scare you off with the “weird” section. You do start to remember the combos surprisingly quickly. But there’s also an app for that!

Actually several apps - Mac, Windows, iOS, Android. The app connects to the device via Bluetooth or USB and gives you the missing display. This makes many of the deeper functions much more usable. In fact, there are a few things you can only control via the app. The app is also very useful for dialing in exact numbers for settings. Again, since there is no display on the device itself. When you’re twisting knobs, there is a stack of eight white LEDs that light up to show you the rough level of what you’re adjusting. But in the app, you get the actual number.

As vital as all of that sounds, I find myself mostly using the device without the app. Just adjusting parameters by feel, or by ear, not knowing what the numeric value is. It’s kind of liberating. What’s the actual BPM here? No idea, but it sounds about right to me.

You also need the app for importing Yamaha’s presets, samples and packs, and your own samples. And for backing up and restoring your projects, which is important because you can only have eight projects on the device itself. When you use those up, you’ll have to back up and delete some to make space.

The not so great

While this is a nice device for trying out ideas, experimenting, jamming, etc. Its production features are lacking.

There is no concept at all of rendering a finished song, or even stems. You have to hook it up to a recording device or computer via line out or USB and record that way in real time. It would be wonderful to be able to do an internal render to an SD card for example.

The eight project limitation is really strange. Especially when they give you 500MB for your own samples. How big can a project be? It should just be a collection of parameters for each step/track/instrument.

I’m also quite surprised at the song mode limitations. A song can be up to sixteen scenes. Each scene is a set of tracks for each instrument. And each track can be up to eight bars. I did some math. At 120 BPM, that’s 0.5 seconds per beat. So a bar (four beats) would be two seconds. Each track/scene can be up to eight bars, so 16 seconds. This means a full song is 16x16 = 256 seconds. Or four minutes and 16 seconds. While that’s a decent length for many songs, as a top end limit, that’s very constricting. Of course there are hacks you can do with the BPM - like cutting it in half and doubling the beats. But you shouldn’t have to be jumping through hoops like that just to get a five-minute song.

What you can do is kind of jam with your arrangement as you record. Rather than try to prearrange everything and let it play out, you can use sequence mode to jump around your sequences and manually switch out instrument tracks as you go, or even change parameters on the fly. You can do that all day and make as long a recording as you want. It’s just not very repeatable.

This probably reflects more on me than on the device. If you just have a bunch of standalone synths and a sequencer, or you do modular, or (OMG) you play the keyboard or real physical instruments, you might be wondering what the fuss is all about. I don’t play anything and have a hard time remembering a melody five minutes after I play it. So I’m in a place where I need to be able to build little pieces of a song manually and save them, then add to them and iterate on them over time.

One more thing. Build-wise, it’s pretty light and all plastic. It looks good, but it feels pretty fragile.

Comparison to M8

I knew that this device would be competing for my attention with my beloved Dirtywave M8. They exist in very similar universes - “3S devices” with fantastic sound design features and tons of effects. But they have completely different workflows, so I figured they would complement each other nicely. I think that assumption has held true. It’s kind of like learning a new programming language. The paradigms unique to the new language make you rethink how you code in languages you already know. There’s already stuff I’ve learned on the Seqtrak that has inspired my M8 work.

The Seqtrak is perfect for real time jamming and coming up with ideas. Set a key and scale and start programming in steps and live jamming and recording. You can just bang away and it just makes everything you do sound good.

As for producing a finished song though, not so great, as covered above.

The M8 on the other hands is more analytical. You’re entering steps and parameters in a glorified spreadsheet. You can play live with an external keyboard, but a more common use case is looping a phrase and entering and tweaking notes and effects as you go hearing it over and over and fixing it until it sounds right. So generally not nearly as immediate and spontaneous.

But in most other aspects, the M8 shines. Unlimited projects (depending on SD card space), on-device rendering, including stems. There’s still a learning curve, but almost everything is right there on the screen. Yeah, there’s key shortcuts to learn, but it’s mostly select/copy/paste/edit motions that work generally the same on every section of the device. The build quality is superb. You could beat an ogre to death with it. (I’ve been playing a lot of Quake.)

Another comparison - what you can do per track. The Seqtrak has eleven instruments which can have up to six tracks each. But all those tracks can only play that one instrument - specifically the exact sample/preset you’ve chosen for that instrument. You can alter some of the parameters on a step-by-step basis, but you can’t choose a whole new sound for a given step or track.

On the M8, you only have eight tracks, but every step of every track can play any instrument at any time. Every step of a track could theoretically play a different instrument - out of I think 128 total instruments. So in the end it’s far more flexible.

Summary

I think I came off sounding somewhat on the negative side here, but I honestly like this device a lot. It’s a load of fun to use and quickly create stuff that sounds very nice. I have absolutely no regrets and I’m sure I’m going to keep using it regularly. Its core functionality is awesome. I’d love to see them make a sturdier model with an actual display of some kind. But that would probably double the price - or more.

I suspect I’ll be jamming on ideas on the Seqtrak and finishing songs on the M8. Of course, they can also collaborate via MIDI and/or line in/out, so that might be interesting to experiment with as well!

FM Synth Chords on the M8 Tracker

kp@bit-101.com (Keith Peters) — Tue, 22 Apr 2025 05:45:00 -0400

I count myself as an M8 noob (and, let’s face it, a music noob too), but the best way for me to learn is to write as if I’m explaining something to someone else. Occasionally someone else might actually wander along and learn something too. Bonus!

There are a few ways to create chords on the M8.

Using multiple tracks - there’s also a trick to automate this somewhat using internal MIDI routing, but it still uses multiple tracks
There are a few modules in the MacroSyn instrument that can create chords - the Triple Sine, Saw, Square, etc, Wav Paraphonic, and probably some others
The HyperSyn instrument
The FM Synth instrument

Using multiple tracks is pretty straighforward. It does use up tracks, but you can always bounce them down to a sample once you have a sound you like. Of course, if you make any changes you have to go back to the source and re-bounce. Using this method, you can also use samples as voices for the chord, which isn’t possible in any other method that I know of.

The MacroSyn modules do chords, but I haven’t found any in depth documentation for them (even the original Braids documentation is not super helpful). It’s probably worth digging into more, or trying to find more on this. Wav Paraphonic seems very powerful.

The HyperSyn instrument is amazing. Super easy to create multiple chords and easy to select one for any given note. I’ve used it a lot. But after a while, it just starts to sound the same. You have six intervals, each of which is composed of two saw waves that can be detuned. Plus a square sub bass. There’s probably a lot more I could dive into, but I haven’t really managed to pull out a lot of variety from this one. Saws will be saws.

Which brings us to the FM Synth solution. This is a well known route, but I started exploring it recently and found some pretty good tricks. It’s especially useful since version 5 of the M8 firmware, which adds wavetables to the voices you can choose in the instrument.

A lot of this is pretty obvious and has been used and documented well by others. But I think I’ve been able to work out a few tricks that someone might find useful.

Setting up the Synth

Basically, we’re using the FM Synth instrument as not-an-FM-synth. Just four different independent voices. So choose the last algorithm, A+B+C+D.

Next, set each operator to the same voice. Let’s try saw for all of them and see if we can replicate the HyperSyn sound.

With four operators, we can create a 7th chord. A major 7th would have hex intervals of 00, 04, 07, 0B. We’ll enter those with mods. Leave the RATIOs and LEV/FBs all at the default. For operators B, C and D, create pitch mods for mod 1, 2 and 3. i.e. 1>PIT etc.

Then below that set MOD1, MOD2 and MOD3 to 04, 07, and 0B. Operator A will not be modded, so it will get the root tone. Everything should look like this now:

If you play a note with that instrument now, you should hear a decent major 7th chord.

You can set up a HyperSyn instrument that will be almost identical like so:

If you play these two one after the other, you’ll find they sound pretty close.

Your browser does not support the video tag.

Not 100% the same, but pretty damn close. They’re two different synth engines, so there will be differences.

Making the chord dynamic

Of course, we don’t want to hard code the specific chord we’re using. So remove the 04, 07, 0B from the mods on the instrument screen. And set the table tick (TBL. TIC) to 0 so the instrument screens looks like this:

Then let’s go into the table for the instrument and create the mods there. In addition to the major 7th, we can create a minor 7th. This is 00, 03, 07, 0A. Again, we only need do to the last three.

We’ll use THO (table hop) to select which chord to play. But we only want it to play one chord per triggered note, which is why we set table tic to 00. This causes the table to only advance on a new note.

Now let’s set a chord progression, the classic I, V, vi, IV. In C major that will be C, G, A minor, F.

On each note we use a THO to set it to the major or minor chord.

If we slow down the tempo to 50 bpm, we get this:

Your browser does not support the video tag.

Cool, but all we’ve really done here is partially reproduce the HyperSyn. A lot more is possible.

Enter wavetables

As mentioned earlier, version 5 of the M8 firmware lets you use one of 61 different wavetables for each operator. And each wavetable contains 256 different waves. So… yeah, there’s some variety to be had there. 15,616 different sounds - for starters.

First, let’s set each one of the operators to use the same wavetable. I’m choosing W0F, but you do you.

Now, to choose which wave to use, we use the feedback (FBK) parameter.

So, set FBK of each operator to FF. And add a mod do each one: 4>FBK.

Play the song again, and as it plays, move MOD4 up and down slowly. You’ll hear a wide varity of sounds, from lush to harsh to bells to muted to mellow. And that’s just with one of the wavetables.

Your browser does not support the video tag.

By the way, if you’re wondering why I set FBK to FF for each operator, my post from the other week might help: https://bit-101.com/blog/posts/2025-04-06/m8-fm-wavetable-modulation/ In short, that value sets the maximum feedback. The MOD4 value chooses a value from 0 to that maximum. So if we want to choose among all the wave forms, we set FBK to FF (255).

A few more tricks

Right now every operator is using the exact same wave form. You can go totally wild and choose an entirely different wavetable for each operator. That might get a little chaotic though - like four different instruments playing a chord. But if you find something that works, go for it.

For a little more subtle effect, just change the FF on each operator to something slightly different. This changes the maximum of the range you are choosing waves from, so that each operator’s wave will be a little bit offset instead of all being the same. Even something like F9, FB, FD, FF can make enough of an offset to give each operator a different wave, making a slightly richer sound overall.

Also, try modulating MOD4 with an LFO. I’d suggest keeping the LFO depth (AMT) very low, like under 10, or making the LFO speed very low (i.e. setting FREQ high). Otherwise you’ll be ripping through a ton of different waves too fast. But again, if that’s what you want, do it. But a low depth, fast LFO gives a kind of vibrato, a higher depth, low speed, gives an evolving texture change. Great for drones or other long, drawn out chords.

Your browser does not support the video tag.

Naturally, play with filters, amps, limiters, chorus, delay and reverb, and anything else you can think of.

Add more chords

In this demo, I only set up a major 7th and minor 7th. But you can add whatever chords you want - augmented, diminished, suspended, inversions, etc. Just trigger them with a THO to the right row.

One thing I haven’t figured out a perfect solution for is how to have triads and four-note chords in the same instrument. For a triad, you’d want to set operator D’s level to 0. But the only way to do dynamically that is with a mod. And we’re plumb out of mods.

There are two solutions I’ve thought of. One is to just have two instruments. One with only three operators playing - for triads, and another with all four - for 7ths or whatever.

Another solution is for your triads, just set operator Ds mod to one of the other notes, either 00, 04, or 07 (for a major chord). This does make one note of the chord louder than the other two though. So for now I’m just going the separate instrument route. If anyone has any other ideas I haven’t thought of, let me know.

In parting, here’s a track I did while figuring all this out. The main chords that lead off the tune are created exactly as described in this article. I think they sound pretty good.

bit101 · Alternate 85

M8 FM Wavetable Modulation

kp@bit-101.com (Keith Peters) — Sun, 06 Apr 2025 15:44:28 -0400

OK, what the hell is that subject?

I’m going to talk about a sound design technique I figured out on the Dirtywave M8 hardware tracker device that can make some cool evolving sounds.

Most of what I’m talking about here won’t make a whole lot of sense if you don’t have an M8 device (or the headless version you can set up yourself). But even if you don’t, you might find it interesting.

Update 4/7/25

I just realized a simpler way of doing this and updated the post with it. I left the original post as-is because I think it’s good for understanding, and might be the best way to do things in some cases. But make sure you read to the end for the simpler way.

FM Synthesis

So the M8 has an FM synthesis module, which is very powerful for a small handheld device like this. But we’re not going to talk about FM sysnthesis at all here. The important thing is that the FM synthesizer has four oscillators. In FM synthesis, you’d have some of those oscillators modulate other oscillators. But we’re just going to use one of them and ignore the rest. Also, classic FM synthesis generally sticks with sine waves, but the M8 has a bunch of different waves you can choose, including a whole crap ton of wavetables. This is kind of insane, but can be very powerful. Each oscillator has a feedback parameter, which usually feeds the signal back into itself. But if you’re using a wavetable, the feedback parameter is used to index a particular shape in the chosen wavetable.

Setting up the Wavetable

OK, so the M8 comes with 61 wavetables built in. They are numbered (in hexadecimal) from 09 to 45. The M8 manual even shows all the waves in each table. Here’s the first eight:

OK, it doesn’t show all the waves, but it does show one out of four for each table. This is good enough because the missing ones are just interpolated between the shown ones. As you can see, each wave has a hex value from 00 to FF

Here’s a close up:

Say we wanted the wave in the center there, number 34. Looking at the top of that column, I see that’s in table 35

I’ve set up an FM synth instrument in a phrase on the M8. I’ve chosen algorithm 0B which just outputs each oscillator separately - no FM synthesis. And I set levels of the first three oscillators (A to C) to 0 and cranked up the final one (D) to AF just to make the wave’s amplitude as visible as possible without clipping. I set oscillator D’s wave to W35 to get the right table, and set the feedback to 34 to get the right wave in that table. It’s playing a D#5 note, and as you can see, it looks pretty close to wave 34 shown in the manual. Hurrah! First step accomplished.

Side note: it’s unlikely you’ll often want to do sound design by choosing interesting wave shapes. Though there are examples of just that out there. But I’m just walking through this in order to understand deeply how this all works. Bear with me.

Modulating a Wave

OK. We have a single wave playing. Now we want to modulate it. i.e. make it move between different shapes, which will change the way the tone sounds. Well, we can just modulate the feedback parameter.

Here, I’ve added a modulator for oscillator D, 1 > FBK. This means modulator 1 will affect feedback.

But you’ll notice that the waveform is no longer what we originally had. In fact, it matches wave 00 of table 35. And if you could hear the tone playing, you’d notice it has changed.

So what’s happening here? Well, in order to modulate something we need two values to modulate between - the start point and the end point. The value we enter in the oscillator’s settings is the end point. The start point is what’s shown in the MOD1 field just below and to the far left. It’s 00. If you move MOD1 up to FF, that will make the minimum and maximum values of the modulation the same, and we’ll be back to where we started, including having the right sound back.

So we need to set MOD1 to the minimum, or start value of the modulation. This is where it gets a bit complicated. And a video will help a lot.

Your browser does not support the video tag.

I’m starting with the same set up as the last screenshot. Then I move MOD1 down to 00. Now we’re back to showing wave 00.

Then I go into the Mods screen and set up an LFO to control MOD 1. It’s going from the default 00 to FF with a triangle wave, and I slowed it down a bit.

You can see that the wave is cycling through a whole bunch of different shapes. In fact, it’s going through every shape in table 35 from 00 to 34. And playing a bunch of different sounds as it goes through those different shapes.

But say I don’t want to go through ALL of those shapes. Maybe just from wave 24 to 34.

How to get there?

Well, we know that the feedback parameter on the oscillator determines the maximum or end value, and the MOD1 setting below sets the start point, but the start point isn’t simply the wave index in this case. It’s mapped so that 00 is the first wave in the table, and FF is the maximum set on the oscillator. So the value is kind of a percentage from 0 to the maximum, displayed in hexadecimal. You could do the math, and figure it all out that way, but that would be painful. Instead, just do it by eye.

First I’m going to remove the LFO by setting its destination to OFF.

Then I’m going to back to the instrument page, move MOD1 back up to FF and then slowly decrease it until it looks like wave 24.

When I get to D4, it looks about the same as wave 24 I think. Again, in real world sound design, you’re probably going to be doing this way more by ear than by eye, so don’t feel bad about winging it.

Now, I’ll turn back on the LFO and see what we get.

Your browser does not support the video tag.

OK, it’s modulating, but it does from 24 to 34 in a split second and then just seems to stay at 34 for the rest of the cycle.

By the way, don’t worry about the jumping every few seconds, that’s just the note being retriggered as the phrase loops.

What’s happening now is that the LFO is going from 00 to FF and adding that value to the minimum value from the MOD1 setting. That’s way too much. So it goes past the maximum value and just gets stuck there.

What we want is for the LFO to make MOD1 go from D4 to FF. If we subtract those two, we get 2B in hex. Of course, subtracting hexadecimal numbers is not easy for most humans, so I find it easier to count my way up.

I go into the LFO and set the amount to 00 then count up from D4 to FF. First by the left digit, hitting the up arrow: E4, F4, which gives you 20. Then by the right digit, hitting the right arrow for F5, F6, F7, F8, F9, FA, up to FF. When you’re done, you should have 2B.

When you play this now, it’s just going to cycle in the exact range we want.

Your browser does not support the video tag.

I also made the chain a bit longer so there aren’t as many jumps.

In case that’s not completely clear, here’s a diagram that might help.

The bottom arrow represents the full range of waves in the table, 00 to FF.

You can see the minimum and maximum we want: 24 and 34.

We set MOD1 so it corresponds with the minimum, and make up the rest in the LFO.

Final Project

Your browser does not support the video tag.

For a (somewhat) more real-world example, and to break down the steps…

I chose a different table - 3B.
I turned off the LFO.
Set MOD1 to FF.
Found an end point I liked the sound of - 3C.
Moved MOD1 down till I found a starting point I liked - 6F.
Then set the LFO to the difference of FF and 6F, which is 90.

I did all this by ear, not looking at tables and wave numbers.

This gives a complex, rhythmic sound, even though only a single note is playing.

Of course, instead of a free running triangle LFO, you could use a ramp or some other shape, retriggered, or just use an envelope of your choice.

The main point of this post was just figuring out the start and end points of the modulation and how to set them, which was not at all obvious to me for quite a while.

Also, I’m still a bit of a noob with this device. In fact, I’m using the headless version still, though my real M8 should be arriving this week I hope! Anyway, if there’s anything incorrect in this post, please let me know.

Update

I figured out a simpler way of doing this.

Set the feedback param of the oscillator to FF. This makes the modulation range 00 to FF, which exactly map to the wave indexes, so all values used will map exactly to the wave numbers.
Set MOD1 (or whichever mod you are using) to the minimum value, say 24.
Now set up the LFO with the range you want your modulation to cover. If you want the modulation to go from 24 to 34, set the amount to 10. It will add 10 to the minimum 24, making it range up to 34.

I think both ways are still valid and result in the same sound, so which one you use is up to you.

Plex, Navidrome, and Symfonium

kp@bit-101.com (Keith Peters) — Sat, 29 Mar 2025 17:37:56 -0400

Music

Like most people, I enjoy listening to music. I’m also a bit older, so most of the music I listen to is not contemporary. I gravitate towards the music I grew up with, and even more these days I have a lot of affection for the music that I heard when I was really young, the stuff that my mother listened to, or was just on the radio or tv, before I was consciously seeking out popular music. I was born in ‘64, so a lot of that is from the 60s or early 70s. It’s super nostalgic to me. And of course, my decade was the 80s. And I love a lot of the music from the 90s. I do listen to more modern stuff - mostly electronic music of one kind or another, just not so much of what’s popular with the kids these days.

That’s part of the reason I’m not a huge fan of streaming services. I don’t pay for Spotify, but I do have a paid account on Tidal. While both of those services have plenty of good older music, their main business model is pushing out the new and contemporary tunes. So I do use Tidal somewhat, but I’m mostly interested in curating and collecting my own library of music. I’ve been doing that for years and have a nice collection (almost 14,000 songs), ripped into lossless FLAC for the most part, organized into folders, all correctly tagged, album art added, even with all the replay gain settings calculated and added to the metadata of every single song. Yeah, I’m a bit obsessive about it.

How does all that music get into my ears?

If I never left the house, I could just have my libary on some device connected to a stereo system. But I do go into the office now and then, travel, walk the dog, drive the car around, go for runs, etc. All that music is too much to carry around on a phone - at least now that most phones don’t take extra storage. Enter…

Personal streaming

I’m talking about straming my own curated library from a self-hosted server. For a while I was using Google Music and Amazon Music. Both had services where you could upload your own music and stream it. Neither one does now. It was decent while it lasted, but when it was gone, I needed to find something else.

Subsonic

When you start looking into music servers, you’ll find a wide selection. But most of them are forks of a fairly old server called Subsonic. It started out as an open source project but eventually went with a closed license and offered a paid, premium version. In 2017, a fork of the open source version was launched, AirSonic. Over the years other forks emerged, FutureSonic, MadSonic, AirSonic Advanced, LibreSonic, Gonic, Ampache, Navidrome, and others. There’s even BookSonic that’s tailored to audiobooks. I don’t know of all of these are forks specifically, but they use the same Subsonic API, so they’re compatible.

The fact that all of these servers use the same API means that any Subsonic compatible client will work with them. And there are a TON of clients. Mac, Windows, and Linux desktop clients, Android and iOS clients, multiple web clients, even terminal clients. There are even hardware devices that will connect to these servers.

I’ve used a number of these in the past, and most of them work fairly well if you’re playing a particular artist or album or playlist. The big frustration for me was in trying to just get a random (shuffled) stream going. Either going to all tracks and choosing shuffle, or even choosing a particular genre and shuffling that. Instead of shuffling all the tracks in the chosen library, every client/server combo I tried would do something like grab the first couple of albums in the list and shuffle the two dozen or so songs in them. Not at all what I wanted. I tried everything and never got past this point, and finally gave up.

Screw streaming

At this point I decided to forget about streaming and got a dedicated digital audio player (DAP). I wound up with several of these over the next few years. They accept large capacity micro SD cards so they’ll easily hold even a large library, and they’re built with one purpose in mind - playing music. They have high quality circuitry for playing back high resolution audio files and usually have plenty of high end features for tuning the sound to your liking. This has been great, and serves most of my purposes, but not all.

Changing gears: Plex

In addition to music, I have a decent library of movies and tv shows. I’ve used various servers for these over the years as well. One or two media servers that I think grew out of the old Xbox Media Server, Kodi, Jellyfin, and Plex. All of these worked pretty well. Around the time where I mostly moved over to DAPs for music, I was using Plex. I still wanted a streaming service for occasional use cases. Running for one. I usually carry my phone on a run, and didn’t want to carry a second device for music. Or sometimes I was just out somehwere and hadn’t brought a DAP and wanted to listen to music on my phone. I realized that Plex, in addition to serving movies, also serves music. It has a dedicated app for that - PlexAmp. I tried it out and was blown away. It was the best mobile streaming music app I’d used. It handled random playlists of everything or just a genre perfectly. It even goes beyond that, with smart playlists based on a mood or music type or using songs or artists as a starting point.

In order to use PlexAmp, you need to have a PlexPass, which at the time was like $39.99 for a year. Or $119 for a lifetime pass. I felt it was worth it and paid for probably three years. The pass also gives you other features of Plex for videos. Don’t ask me what they are though. I really only bought it for PlexAmp. In retrospect I should have gone with the lifetime pass. And probably would have this year. But…

Screw Plex

So all was good for a while. Then I got an email the other day that Plex was raising its prices and making other changes.

The annual PlexPass was going from $39.99 to $69.99. Damn. That’s nearly double. And lifetime was going from $119 to $250. Holy cow, that’s more than double. In addition they were making it so you need a PlexPass to stream anything outside your local network. The change goes into effect soon, so I had to decide if I wanted to go with lifetime for the current lower price, or move on to something else. I decided to test the Subsonic clones anew. It’s been a good few years since my last attempt.

Navidrome and Symfonium

I did some research on the current clones and Navidrome was getting high points. Many of the clones were out of active development and were never much more than straight up clones. Navidrome seemed to put a lot more into their offering and were still active. I installed it on my home server via Docker. It was a piece of cake.

As for an Android client, I went with Symfonium. I recalled that it was hands down the best Subsonic compatible Android app before, and it was still getting high ratings. Got it installed and hooked up to my Navidrome server. Worth noting that I’ve seen the Navidrome/Symfonium coupling mentioned a few times as the ideal team.

The verdict?

Great! This honestly is a very good client/server pair. I’m really impressed with the current version of Symfonium. It comes close to the features of PlexAmp in a lot of ways. It’s got built in stuff for track or album or even decade mixes. You can also create a smart playlist where you can specify a whole load of parameters and it will choose any number of songs matching those.

Again, this is my secondary use case, for when I don’t have my DAP on me, or whatever. But that happens fairly often. Honestly it might just be that my DAP is upstairs and I want to listen to some music on my phone or laptop without having to go upstairs to get the DAP.

What about movies?

I’m going to cancel my PlexPass when this current year is done. At that point, I’ll only be able to watch my movies on my local network. But I’m not sure I’ve done anything else to be honest. So that will be fine. I might try out Jellyfin.

Q1 Update

kp@bit-101.com (Keith Peters) — Sun, 09 Mar 2025 18:21:39 -0400

Just a general update on what’s been happening since my last ramblings.

M8

Mainly just making music. Still rocking on the DirtyWave M8. Actually, still on the headless version, which runs on a Teensy 4.1 board plugged into my computer. I did manage to get in on the last M8 order window on February 10! So I have an actual hardware unit coming at some point - though they won’t be shipping until early April. Torture.

So many other M8 users express similar love for the device. Many say that they have stopped buying a lot of other musical equipment because this thing just scratches so many itches. I see the same thing in myself. I keep looking at other synths and trackers and devices, but I really just want to get my M8.

I hate to sound dramatic, but this device has changed my life. It’s just so much fun to make music on. Two years ago I said I wanted to make and release a song. I kind of did that in SuperCollider, but most of that was just sound design and generating random (or semi-random) notes. Not that there’s anything wrong with that, but I dreamed of creating a song that was properly put together - all the right pieces handcrafted and thoughfully put in place and all working together. So far this year, I’ve put out twelve songs. I’ve been aiming for one a week. Week 10 of 2025 just wrapped up, so I’m a bit ahead of the game. Not saying my songs are amazing, but I think some of them are pretty catchy. More to the point, I’m enjoying the hell out of the process.

Doing graphics code for 25 years was great, but when I look back at the early days, I was learning something new every damn day and it was so exciting. For the past many years, I’ve been doing a lot of revisiting of old projects, doing them better, combining one technique with another technique to make something new, occasionally learning something new, but not often enough.

But starting in on music, I’m back in that space of constantly discovering something new almost every day. I love it.

For my last song, I dug up a short animation I made last year and extended and expanded it and made a video of that along with the song.

The video is not reactive to the sound, or vice-versa, but I still like how it came out. Also this song has been stuck in my head for a week. I’ve never created my own earworm before.

Gravitar

While I wait for my actual M8, I tried another headless build, using a handheld retro gaming console. You can plug the Teensy into that and run the headless app on the console. I got it working, but I find I actually like working on the computer a bit more.

The device I got was the Anbernic RG40XX V. No big loss on not using it for music though. I’m having a blast gaming on it. My current favorite is an old arcade game, Gravitar.

But also playing other old arcade games and 3D FPSes like the original Quake, Doom and Wolfenstein.

Work

Work’s been OK. We had a company offsite in Ft. Lauderdale, FL in February. It was a cold spell up here in the Boston area, so that was a welcome relief for a few days. Even got to go on a group run with a bunch of coworkers. That was a highlight.

(We were staying at the Hilton you can see in the background.)

I’m working on hiring a senior front end software engineer. The posting got over 1600 applicants in the first few days. That’s pretty scary. Many of them have 10 or more years experience. It’s tough out there. Makes me feel bad that I can only hire one. Expecting this role to be filled this coming week. I just hope I am not in the position of looking for a position myself any time soon.

And…

I guess there is no and. That’s pretty much me over the past month or so. And extrapolate all that out into the next month, and you’re probably on track with what I’ll be doing then. But I’ll let you know.

Been Making Noise

kp@bit-101.com (Keith Peters) — Sat, 25 Jan 2025 13:49:11 -0500

In my 2024 retrospective I mentioned wanting to create some finished songs and put them out there in the world. And I suspected that the #jamuary project might be a way to do so without worrying about the songs being too polished. “Hey, it’s just a jam!”

Well this has worked out well - way beyond what I imagined. To date I’ve put out I think seven (SEVEN) songs! And they are all masterpieces. NOT. But some of them are OK in my opinion. Most of all, I’m just having a ton of fun doing this. I’m probably a bit obsessed to be honest.

Something just clicked this month. It’s been a couple of years that I’ve been dipping my toes in and out of trying to learn a bit of music here and there. SuperCollider, LMMS, Ardour, Ableton Live, various hardware synths and MIDI controllers, Reaper, other apps, various books, courses, videos, etc.

And then…

Renoise

The day before New Years Eve, I woke up not feeling good. By noon I had a fever, by dinner time I had tested positive for Covid. My first bout with it, so I can’t complain, but It knocked me down and kept me down for most of a week. During some moments of lucidity though I was watching some Youtube videos and stumbled upon this:

I had played with some trackers in the 80s on my Amiga and dabbled with them a few times since. But I saw Renoise mentioned in there and it was something I’d heard of more recently. It’s a modern tracker. In fact, it’s fair to say that it’s a full fledged DAW (digital audio workstation) that happens to have a tracker interface. You can use high res samples and just about any kind of synth or effect plugins via VST and other standards. MIDI in and out. Which is to say, it’s not just chiptune stuff.

On the first day messing with it, I managed to put together a full jam that I actually really liked and felt good enough to release.

Over the next week or so, I banged out another couple of songs. The tracker technique just really seemed to work for me.

Dirtywave M8 tracker

Having realized that the tracker paradigm gave me super powers, my attention went to a device that I’d been eyeing off and on for months. The Dirtywave M8.

This is a small, gorgeous looking device, machined aluminum case, display panel, and all of eight keys. Basically a handheld tracker.

There are other hardware trackers such as the Polyend Tracker and Polyend Tracker Mini.

The full size Polyend Tracker is definitely a desktop device. The Mini is also large in comparison to the M8:

6.7 x 5.12 inches (170 x 130mm) for the Mini
5.2 x 3.77 inches (113 x 96mm) for the M8

Here’s a screenshot from a video which shows the sizes more clearly. (This shows the first model of the M8, which is a bit different than the model 2, but the same size.)

The M8 is just so much more compact and portable. Something you can jam with on the train or at a restaurant or a park and most people would just think you were on your phone or playing some kind of game. The Mini is more like a very thick iPad. It doesn’t hurt that the M8 is a bit cheaper too.

The M8 has its history in software that was created for the GameBoy, called LSDJ (short for Little Sound DJ).

One of the brilliant things about LSDJ and the M8 is that little menu you see in the bottom left. Each one of the letters represents a particular screen in the overall system. So you can use the left and right arrow controls to go from Song, through Chain, Phrase, Instrument, and Table. Each of those main screens usually offer an up and down option for other screens as well. You can see on the Song screen there that you can go up to Program, or down to Groove.

But the other wonderful thing about the M8 is that there is a freely available version of the software that you can download and run on a Teensy 4.1 microcontroller, which you can pick up for around $30. This is a headless version of the software, so you need to run something else on a computer or other device to get a display, sound, keyboard and midi input/output.

One might think that offering the software for free might cut into sales of the hardware device, but after just a day of playing with the headless version, I was committed to buying a real M8.

I won’t go into the details of how the M8 works or how to set up the headless version, but awesome-m8 should answer all of your questions.

I got a 3d printed case made for my Teensy board (thanks Luke!), and its pretty much permanently been dangling off my computer for the past week. And I’ve already put out a few jams/songs that were done on it. This is the most fun I’ve had making music ever. I can’t wait to get the real thing.

Faircamp

One final update on the subject of making noise.

Now that I’m making music(al sounds) that I’m not afraid to share with others, I’ve been thinking about how best to share them. Last year I set up a SoundCloud account and I’ve been putting stuff up there. But the free plan is limited in space. I’m not close to quota and a paid account is not that bad if I get there, so I might eventually do that. There’s Bandcamp, but that’s mostly about selling music, and I’m not even thinking that way yet.

A few months ago I heard of Faircamp. This is obviously a response to Bandcamp and some changes that happened there last year. But it’s software that creates a static site for your music. I could do this with Hugo (like this site), but it would be a bit of an investment. So I decided to try Faircamp.

It’s pretty decent! You run the executable and point it at a folder that contains your music, images and some metadata files and it spits out a site. It’s really that simple.

You can run it in preview mode which sets up a local server to view your changes before you publish them. And there’s even a command that will sync your built site to your live server via rsync. It even does all the encoding to whatever formats you tell it to, sets up playlists, rss, embed codes, the whole lot. It’s also got some options for selling your music, thought it’s not a full fledged point-of-sale system. I haven’t actually looked into that side of things because, again, I’m not anywhere near there and may never be.

One thing is that the customization options are limited. You can set up a site theme and individual release themes with a few color options, but not much more beyond that. But it looks pretty good, so for the utter simplicity it offers, I can’t really complain. I would love it if you could include some custom CSS in a future release. Of course, there’s probably a way to do that now, but it would take a bit of hacking.

I had a domain I wasn’t using, keithpeters.org. So… there’s my Faircamp site. KP-BARA is the name I chose for my musical identity, just to keep it a bit differentiated from BIT-101.

I haven’t put up all the stuff I’ve made there. Some I only posted on Mastodon because they were just fun silly things. I guess the ones I put on the site are somewhat worth keeping around. If all goes well, I’ll look back in a year and be embarrassed by them because what I’m making then will be so much better!

2024 Retrospective

kp@bit-101.com (Keith Peters) — Thu, 02 Jan 2025 17:00:00 -0500

What happened in 2024. A rough guide to things that I remember.

January
- I participated in Genuary. Every day. Made some really cool stuff.
- I went to Nashville for work. First time. It was fun.
February
- I went to Portland, OR to meet my grandson. He was born in September. Also first time in PDX. Neat place.
May
- I created Wire - a graphics toolkit.
- I created Lures - a strange attractor toolkit.
- I rebooted Art From Code
- I had shoulder surgery for a torn rotator cuff tendon. Five weeks in a sling 24/7. Six months of PT.
June
- I bought a MIDI Fighter Twister. My favorite piece of MIDI equipment. I wrote a SuperCollider class for it which I use a lot.
- Lots of pieces created on Art From Code using Wire and Lures.
August
- Started running again after surgery.
October
- I turned 60. 😱
November
- I created and published a high res, longish video along with the musical soundtrack for it. See it here! This was a long term goal. It’s not amazing, but it felt great to get it done.
- I got a few more small hardware synths. Having great fun jamming with them.
December
- Learned that I have yet another grandchild on the way.
- Been pretty obsessed with music making technology. Code, hardware and software synths, DAWs, effects plugins.
- Got Covid for the first time. Spent New Years Eve and Day in bed, miserable.

Thoughts

I’ve been really into music creation stuff in the past few months. It’s been cycling in and out for the last couple years, but each time I go a bit deeper and understand a bit more. My initial goal was just to create soundscapes to go with longer animations. But now I kind of want to explore music on its own. I feel like I’ve been way to deep in the technology of it all. I need to focus on actually completing some short pieces. #Jamuary is on and people are posting good stuff. I might try to throw out a few thing here and there. I like the idea of doing quick jams, recording them and putting them on line somewhere. Not as a polished thing, but just - “hey, I did this”. It’s the closest to the visual “experiments” stuff that I’ve always done well with.

I also love doing and will continue to do visual creative coding. But honestly in the past few years it often feels like I’m going over the same ground over and over. I mean, I’ve been doing this for 26 years. I think I do it better now. And I occasionally find something brand new to experiment with. And often combining one technique with another I find something absolutely brand new (at least to me) - something I’ve never thought of doing or ever seen anyone else do. But it’s not the same as the early 2000s when everything was brand new to me. That’s where music is at for me now.

I’m pretty sure I’ll get back into running again. I started up again a couple months after the surgery, but around October when I got sick (minor cold), I got knocked off my schedule again. I ran last week - then later that day I got sick with Covid! As soon as I’m recovered from that, I’ll be back out there.

This year will see my youngest daughter graduate from college - CS degree at Boston University. I hope the economy improves a little bit. I know it’s rough out there for software engineers looking for jobs. And I’ll see my second grandson (from my older daughter).

Yeah, I’m in my 60s now. That just seems so weird. When I was in my late 20s, approaching 30, I started obessing about my own mortality. Oddly, the closer I actually get to it, the less and less worried about it I actually am. My biggest concern now it making sure that I actually enjoy the rest of my time on this merry go round and do meaningful stuff, not work myself to the bone and die chained to the machine. Sounds morbid, but it’s not. A while back I got this memento mori coin.

One side says “Memento Mori” - remember death. The other side is “Memento Vivere” - remember life.

Just as a reminder that I’m not going to live forever, so I should be living my life with meaning. And hey! I have a lot of years left, hopefully! Although my grandson is just a year old, no reason I couldn’t get the title of great-grandfather eventually.

I don’t do resolutions per se, but I really feel the need to work on reducing stress and living happier in the coming year. The past couple years have been super stressful. For six and a half years - from mid 2017 to early 2024, I did not get sick. Not even a cold. This year, I’ve had two serious illnesses, one minor cold and a surgery. I know that stress is a key factor. For the last few weeks I was thinking to myself that stress was going to make me sick. Sure enough. It’s mostly work stuff, and I resolve to fix it. Because this cannot continue.

I guess that’s about it. Not all rosy, but the new year is always a good cue to start fresh and make improvements. I’m in.

Journal

kp@bit-101.com (Keith Peters) — Sat, 28 Dec 2024 12:07:31 -0500

Over the years I’ve made many abortive attempts to keep some sort of journal. For no other reason than I think it’s good to reflect on your life on a regular basis in some kind of organized way. I love writing and find that by describing something in written words often helps me to clarify things. I’ve always maintained that teaching is the best way to learn something deeply. So maybe by writing things that are going on in my life, I’ll learn something. Better than disorganized worrying about them anyway.

Inspired by this post on mastodon, I decided to give it another shot.

OK, how to do it?

Choices:

Pen and paper or digital? Digital.
In the cloud or local? Local first, but backed up somewhere, somehow, securely.
Some service or DIY? A mix. I’m OK using a tool that makes it easier, but no proprietary formats. No lock in.

So we come down to the two universal tools - the only two things anyone needs ever. vim and git.

Disclaimer: I actually use neovim, but generally don’t make the distinction. It will be important here because I use neovim’s lua syntax for some of the stuff below.

I found a few articles that had a lot of valuable tips. Here are some good ones:

What’s great about using vim for stuff like this is that you have all kinds of configurations, tools, plugins, autocommands, etc. to make it do what you want it to do and look how you want it to look. The articles listed above have some great ideas on templates (vim skeletons) and autocommands and autocompletion sources. I’ve used a few of these and will add more as time goes on.

Enter vimwiki

While I was looking around for tips, I heard mention of vimwiki. I knew of this project but I didn’t know that it had a built-in diary mode. This sets up a section of your vimwiki to be a journal, exactly what I wanted. I already have another solution for my personal wiki knowledge base, so I don’t use the wiki part of vimwiki, but the diary part looked great.

Essentially, there are a handful of vim mappings (keyboard shortcuts) it sets up. For example, one takes you to your diary post for today (creating that file if it does not exist). If I have a thought, I just hit that shortcut, I’m in my journal entry for today, I write what I want, save and exit.

There are other shortcuts for yesterday’s entry and tomorrow’s, as well as the diary index file. And another shortcut when you are in the index to create a nicely formatted list of links to all your entries. Also, while you’re in any diary entry, you can tab to each link and hit enter to go to that page. And backspace to go back to the previous page. All very nice added functionality.

Templates

Off to a great start, but now when I go to today’s entry, if I haven’t already written anything, I get a blank document. I had a template in mind - things that I want to comment on each day. Prompts, you could say. I came up with something like this:

== Personal ==
* 

== Reading ==
* 

== Watching ==
* 

== Health ==
* 

== Work ==
*

One thing I discovered is that vimwiki uses wiki markdown flavor. I’m used to #, ##, ### for headers and - for unordered lists. But wiki flavor favors =, ==, === and *. I think it will work fine with the other format, but when I viewed my files in github, they were all messed up. I guess github was seeing the .wiki extensions and trying to render them with that flavor, and it was not good. So I use what I’m supposed to.

Now, to use this template every time I make a new entry. In vim, you have various “autocommands” that run on a file in various situations, like when it is created, opened, closed, before and after writing, saving, etc. We can tap into these to apply the template. Here’s the start.

vim.api.nvim_create_autocmd("BufNewFile", {
  pattern = { "*/vimwiki/diary/**" },
  callback = journalCreate,
})

This is a bit different than default vimscript, because neovim uses a lua syntax for it’s config. Actually, it can use either, but I moved over to lua a while ago.

So I’m creating an autocommand that will run when a new file is created. It’s limited to files created in */vimwiki/diary/ which is where vimwiki puts its diary files. When a new file is created in vim in that location it runs the journalCreate function, which I also created.

function journalCreate()
  date = vim.fn.expand('%:t:r')
  vim.api.nvim_command("0r ~/.config/nvim/templates/diary.template")
  vim.api.nvim_buf_set_text(0, 0, 0, 0, 0, {"= " .. date .. " ="})
end

First, vimwiki creates the file with a name like 2024-12-28.wiki. I get and expand the file name with % and limit it to the tail of the full path - :t and strip off the extension - :r. This gives me a date variable like 2024-12-28.

Next, I run a vim command of 0r with the path to the template I just showed above. To be honest, I have no idea what 0r means, but I know what it does. It loads that template and yeets it into the buffer that was just created.

Finally, I set the text in buffer 0 (current buffer), line 0, row 0 to line 0, row 0, to the date, surrounted with = chars.

Now, when I go to a yet-uncreated entry, I get something like this:

= 2024-12-28 =

== Personal ==
* 

== Reading ==
* 

== Watching ==
* 

== Health ==
* 

== Work ==
*

Note that I added a couple of blank lines to the top of my template to make room for the header insertion.

That whole file parsing, date header thing is more than just cosmetic though, as we’ll see in a minute.

Automating the index

As I said, one of the main vimwiki diary shortcuts takes you to the diary index page. By default this is empty. I want it to be an actual index with links to all my entries. This might be a case for another template, but we don’t even need that because vimwiki already has a function for generating such an index. It’s set up with another shortcut, but I don’t want to have to hit that shortcut or wonder whether or not the index is up to date when I open it. I just want it to be up to date as soon as I open it.

Time for another autocommand!

vim.api.nvim_create_autocmd("BufReadPost", {
  pattern = { "*/vimwiki/diary/diary.wiki" },
  command = "VimwikiDiaryGenerateLinks"
})

This one activates when I read a new file, but only matches my diary index file. When this happens it calls a vimwiki-defined function, VimwikiDiaryGenerateLinks. So the first thing that happens after I open this file is that it gets updated. Perfect.

By the way, here’s what it looks like:

Note that each entry has the name given in the header of the corresponding file. If I did not generate that header, each list item would just say “Personal” as that’s the first header on the page - unless I manually added another header. Hence the automation.

In github, this is at least human readable. Sadly, the wiki-type links do not become linkable there. Maybe there’s some github config for that? Not a big deal though.

A shortcut

Two things remaining.

One, I don’t want to have to run vim and then hit a shortcut to get to my diary. I wanted to have a command that would just open vim to the diary index.

Simple enough with an alias.

alias diary="vim ~/vimwiki/diary/diary.wiki"

Backups

Finally, I wanted to be able to back up my diary, not just have it be subjected to accidental deletion or eventual hard drive failure. Since it’s all just text, github makes sense. I made a repo and got it all tracking my vimwiki. But I didn’t want to have to manually do git add ., git commit -m <message>, git push every time. Because I know I’ll forget and get lazy.

My first thought was a cron job, set to push every hour or whatever. This all worked fine except for the push. The key I use for git has a passphrase and there’s no good way I found to get the cron job to use that. Most advice came down to set up a new key without a passphrase. Nope.

This section has had a few changes. Don’t get attached to anything you read here. A bunch will change by the end of this post. Leaving the trail here for future reminders though.

I came up with an easier solution, just adding on to my alias.

alias diary="vim ~/vimwiki/diary/diary.wiki && ~/diarycommit"

This opens the diary in vim and when it closes, it runs a script, diarycommit (not actually located at ~). Here’s what that looks like:

#!/bin/sh

cd ~/vimwiki
git add .
git commit -m "$(date)"
git push

This adds, commits - using the current date/time as a commit message - and pushes.

The result: I type diary, write what I want, close vim, it’s backed up.

This isn’t perfect. I might go into my diary and leave it open for while, or start editing some other files in vim. It won’t actually commit and push until I finally close vim.

Also, I might be editing something else in vim and have a thought, use a shortcut to open the current page, and close it and go on doing what I was doing. In this case, because vim wasn’t opened with the alias, my changes won’t get pushed when I do eventually close vim.

But it’s good enough for now. Eventually I’ll open things as described above. Or I can always just run diarycommit manually.

If anyone has any tips on how to use a passphrase from a cron job, let me know.

I’m also thinking of an autocommand that runs the commit script on buffer close.

Update: this works! See updates below.

Updates

One

I had one issue where if I was ever editing some non-wiki markdown file (like this very post I was writing via Hugo), vim was treating it like it was part of vimwiki, adding vimwiki mappings to it and other things that were messing me up. I found the solution here

Vimwiki considers every markdown-file as a wiki file

Vimwiki has a feature called “Temporary Wikis”, that will treat every file with configured file-extension as a wiki. To disable this feature add this to your vimrc:

let g:vimwiki_global_ext = 0

Alternative you can set vimwiki to use markdown syntax but a different file-extension, like the default .wiki.

This did the trick.

Two

I figured out something better for git saves. Yet another autocommand!

vim.api.nvim_create_autocmd("BufWritePost", {
  pattern = { "*/vimwiki/diary/**" },
  command = [[ silent! !/path/to/diarycommit ]],
})

This executes right after I write a buffer to a file (save it). Only if it’s in my diary files. It just runs the commit script. Save the file, it gets commited. Works great! I added silent! here so it didn’t give the whole commit message in vim. But you can leave it out too.

Hello, again!

kp@bit-101.com (Keith Peters) — Sat, 28 Dec 2024 11:21:24 -0500

I felt the urge to write a blog post today. When I started to do so, I happened to look at the list of posts in my Hugo site repo and saw that the first post on this “new” blog was exactly one year ago today. So happy birthday static blog!

Static Site for BIT-101 2017-2023

Speaking of static sites and this “new” site, there were two other versions of this blog before the one I started a year ago. The original started in 2003 and went until 2017. That got huge and unwieldy and I created a new one in 2017. Both of these were in Wordpress and I’ve kept them both up and somewhat functional ever since under at bit-101.com/2003 and bit-101.com/2017. But…

For one, the stuff going on around Wordpress these days is a bit ridiculous. I already distanced myself from WP for my own personal reasons around usability and functionality. But I want to even further distance myself from it now.

More than that though, even a dead Wordpress site takes a non-trivial amount of effort. Maintaining the install, keeping up to date with updates, not only of WP itself, but any plugins you have installed, maintaining the database. If I want to eventually switch hosts, I’ll need to migrate all that, which is not my idea of a good time.

So I went ahead and moved the 2017-2023 site over to a fully static Hugo site. There are a few projects that help with this. I found this one:

https://github.com/SchumacherFM/wordpress-to-hugo-exporter

This installs as a Wordpress plugin. Then you go to the plugin and tell it to export all your stuff. It took a few minutes and then prompted me to download a zip file which was a bare bones Hugo site setup, with every post as a markdown file, and a folder named wp-content/uploads which contained all the images and other assets I’d uploaded to my WP site. From downloading the plugin, figuring out how it works, to getting the site download was maybe twenty minutes. Not bad.

Now I needed a theme. I copied over the theme from this blog and did some tweaking to everything to get it set up like I wanted. And the new static site worked… kind of.

Problems

There were a lot of places where I’d hardcoded things to bit-101.com/blog - especially for images and other assets. This was actually already a problem as soon as I moved the site over to bit-101.com/2017 and I had made some progress on fixing these one by one via Wordpress. But now that I had all the markdown files in a folder on my local machine, it was pretty easy to just run sed and fix all of these in pretty much one fell swoop.

There were also just a bunch of dead links. Again, having a folder full of markdown files made this easy to check. I used this node.js tool: https://github.com/tcort/markdown-link-check and it worked great. Those are all cleaned up. Fixed where possible, removed otherwise.

There were some formatting weirdnesses. Like Wordpress makes apostrophes into some funky HTML entity that I didn’t like so I seded those back into simple single quote characters. A lot of the code blocks are still a mess of <pre> tags instead of triple quoted blocks. I’ll probably go back and clean those up at some point.

Another problem was that the exported wp-content/uploads folder seemed to be missing a few files. Rather than messing with that too much, I just copied over the same folder from the original Wordpress install that was already on my server. This had everything.

With that, the newly staticized site was up and running! Markdown and config checked into git. Assets backed up. No more WP updates, plugin updates, database updates, etc. to worry about. Migrating to a new host will just take updating and running my deploy script.

Next up

Next up will be the 2003-2017 site. This is a LOT bigger though, and will have way more broken links to fix and update. But I’ll probably do the initial export some time soon and get that out of the way.

And then there’s my Art From Code archived site. I did the same thing this year, where I moved the Wordpress version to a subdomain and created a Hugo site. I’ll staticize the archived one eventually too.

Search

Another thing I’d like to do with these static sites is to implement search. The 2017 site has a lot of content and the 2003 site has I think something like 1200-1300 posts. Hugo has nothing for built in search, but there are some projects that will help. Generally I think it’s a matter of

Building an index of content on the site.
Creating the functionality to search this content.
Creating a UI to enable the search and display the results.

Another thing to look into in the coming year.

Evenly Placed Points on Bézier Curves

kp@bit-101.com (Keith Peters) — Sun, 29 Sep 2024 10:03:15 -0400

Intro

This is something I need to solve every now and then - the need to interpolate smoothly along a Bézier curve. This can manifest in a few ways.

One, you want to evenly place points along the curve.

Two, you want to smoothly move an object along the curve.

Three, you want to find a point that is some percentage of the way along the curve.

I should note that as far as I know there is no mathematical way to perfectly accomplish this. Or at least not one that’s feasible to implement in code. So we have to resort to other usable methods.

Naive attempt

Well, you might think, the formula for a Bézier curve has a t value in it, where if t is 0.0 it’s the start of the curve, and at t = 1.0, it’s the end of the curve. So just interpolate t from 0 to 1 and create a point on each returned location.

For this post, I’m going to show you some of the code in my own personal geometry library. It’s written in Go and has a lot of dependencies on other code in the library, but I think you’ll be able to understand what’s going on.

I have a function called geom.BezierPoint which looks like this:

func BezierPoint(t float64, p0 *Point, p1 *Point, p2 *Point, p3 *Point) *Point {
	oneMinusT := 1.0 - t
	m0 := oneMinusT * oneMinusT * oneMinusT
	m1 := 3.0 * oneMinusT * oneMinusT * t
	m2 := 3.0 * oneMinusT * t * t
	m3 := t * t * t
	return &Point{
		m0*p0.X + m1*p1.X + m2*p2.X + m3*p3.X,
		m0*p0.Y + m1*p1.Y + m2*p2.Y + m3*p3.Y,
	}
}

Give it your four control points and a t value and it gives you back a point for that location along the curve.

So I’ll create some control points on an 800x800 surface and stroke a Bézier curve, then loop through 100 t values and draw a point at each location.

OK, so we do indeed have 100 points along the curve, but they are not evenly spaced. At the start and at the end they are further apart, while in the center, they’re very cramped. Not what was specified at all.

A better approach, but still bad

I’ve solved this in the past. And when I wanted to add this solution to my library, I attempted the same solution. And it works. Kind of.

The concept:

You start by figuring out how much space you want between each point. Say it’s 20 pixels.

You get the first point on the curve, the p0 control point.

Now you get additional points by increasing t in very small increments until the distance from p0 to the point you get is very close to 20 pixels.

Then you make this new point the starting point and continuing to increase t in tiny increments until you get another point that’s about 20 pixels from that.

The problem(s)

The first problem is that if you don’t increase t in small enough increments, you may not get very close to the target distance. So you have to check if you’ve gone beyond that distance. Then your points just aren’t super accurate. They’re kind of evenly spaced, but there’s still an occasional glitch where they are a bit further apart of closer together.

The solution to the first problem is to use very tiny increments for t so that you are always getting very close to the target distance. This solution introduces the second problem. I found I sometimes had to use increments as low as 0.0001 to get the errors down to a sub-pixel level. That’s 10,000 iterations to create a set of just 100 evenly placed points. This can get SLOW.

One thought is that for a low number of points you can get away with a higher increment and for a higher number of points, you need a much lower increment. I spent some time trying to figure out a formula that would account for that, without much luck.

The third problem is that even when you use low increments, the last point may be very close to the end of the path, less than the target distance. So you still get some glitchiness there.

Yet another problem is that this relies on knowing how much space you want between points. But say you just know that you want 100 points along the curve. Now you need to know the length of the curve. And that comes back to the same performance problem. The simplest way to get the length is to generate a bunch of Bézier points and add up the distances between each of them. But the more points you create, the more accurate the length. You may have to add a few thousand points to get a decently accurate length. This adds to the low performance problem.

I spent a lot of time trying to add error correction/propagation and other strategies to make this more performant and still look good. No real joy there.

So I went back to the drawing board.

A much better approach!

Rethinking this, I realized I could break the Bézier curve down into a segmented path and then interpolate along that path.

Before I even thought of this in terms of Bézier paths though, I needed a function that can interpolate along such a path. A path is simply a list of points. And in my library I have such an object, called PointList. This function should take a t value from 0 to 1 and return a point somewhere along this path, where 0 would be the first point, 1 would be the last point, anything else, somewhere in between, along one of the segments of that path.

Here’s the Go code I came up with. Again, you might not understand all of what’s going on here if you’re not familiar with the language, but you should be able to suss out the basics.

func (p PointList) Interpolate(t float64) (*Point, error) {
	if t < 0 || t > 1 {
		return nil, errors.New("t should be from 0.0 to 1.0")
	}
	if t == 0 {
		return p.First(), nil
	}
	if t == 1 {
		return p.Last(), nil
	}
	length := p.Length()
	accumulated := 0.0
	for i := range len(p) - 1 {
		p0 := p[i]
		p1 := p[i+1]
		l := p0.Distance(p1)
		if (accumulated+l)/length > t {
			tmin := accumulated / length
			tmax := (accumulated + l) / length
			t = blmath.Norm(t, tmin, tmax)

			return LerpPoint(t, p0, p1), nil
		}
		accumulated += l
	}
	return nil, errors.New("unable to interpolate")
}

In short, the whole path has a length, which is the sum of all the segments between each of its points. First, I look for the segment that contains t - where the percentage of the length so far is less than t, but after this segment it’s greater than t. Then I interpolate between the two points that make up that segment and return that point.

In this animation, I create a path of completely random points and then interpolate t from 0 to 1, drawing a point for that t on each frame. You can see it smoothly moves along the path.

Your browser does not support the video tag.

Alternately, I can just increase t by 0.01 to create 100 evenly placed points.

Back to Béziers

Now we just need to create a segmented path like this for a Bézier curve. That’s easy enough. Just go back to our naive approach from the start of this post and create a bunch of points with the BezierPoint function. For demonstration purposes, first I’ll do it in a lower resolution so you can see the segment’s clearly.

Here, I just increased t by 0.1 each iteration, creating 10 segments.

Now this is a PointList - a path made by discrete points - that I can interpolate over with my PointList.Interpolate function. This time I’ll draw the actual Bézier curve, and then 100 interpolated points.

Not great. The points are spaced nicely, but they don’t follow the curve too well. Because the path I created was too low res. Let’s try making that path with 100 segments instead of just 10.

Let’s see what this segmented path looks like with 100 segments.

That’s pretty smooth. You can’t really tell this is made from straight lines. It just looks like a curve.

Now let’s see what happens when we evenly interpolate 100 points along this path. This time with the actual smooth curve drawn in the background.

Woohoo! That’s pretty much perfect. The points follow the curve. They are evenly spaced. Start and end points are all good.

But one question remains:

How many segments to create for a given curve and number of points? Well, as you can see above, for 100 interpolated points, 100 segments worked out pretty well. In my final function, I just doubled that to be sure. That’s still only 200 segments, as compared to up to 10,000 iterations in my earlier attempt, and with even better results.

Here’s the function I came up with:

func UniformBezierPoints(count float64, p0, p1, p2, p3 *Point) PointList {
	path := NewPointList()
	for i := 0.0; i <= count*2; i++ {
		t := i / (count * 2)
		p := BezierPoint(t, p0, p1, p2, p3)
		path.Add(p)
	}

	points := NewPointList()
	for i := 0.0; i <= count; i++ {
		t := i / count
		p, _ := path.Interpolate(t)
		points.Add(p)
	}
	return points
}

Pretty simple actually, with the Interpolate function doing a lot of the heavy lifting.

Note that these points technically still aren’t perfect, Most of them will not perfectly lie on the curve. But by segmenting that curve enough, they’re close enough for anyone’s eye. And if that isn’t close enough, segment the curve some more.

Another example

Earlier I said that one application of this would be smoothly moving an object along a Bézier curve.

In this final animation, I have a Bézier curve. The red dot is moving along the curve using the normal Bézier curve formula. Note that it does not move at a constant speed. It starts out fast, slows in the middle and speeds up again at the end. The green dot moves at a constant speed along the curve. Also note that they both finish together. I guess you could use this kind of thing for a custom easing function if the existing ones aren’t enough for you.

Your browser does not support the video tag.

Summary

Hopefully this helps someone somewhere down the line. I’ll probably continue to improve this code myself. So the code you see here may look a bit different than what you see in my library. I’ll probably create an separate BezierCurve object with methods that do these things. And a function to create a segmented path from a set of Bézier control points. And… who knows what else.

[Update] I did end up creating that BezierCurve object. You can see it here: https://codeberg.org/bit101/bitlib/src/branch/main/geom/bezier.go

Goodbye Obsidian

kp@bit-101.com (Keith Peters) — Sun, 08 Sep 2024 10:50:30 -0400

Back in 2020, I set up a personal “wiki” on github. It was one of the most useful personal productivity things I ever did. Every time I installed something or made some change to one of my computers, it went in the wiki. Every time I got some new device and set it up, into the wiki. All the dates and costs of all my subscription services were there. References I wanted to keep track of, books I’d read, on and on. The best part of it is that it just became a habit. And it was very simple. Just markdown pages linking to each other. It was on line, so available anywhere, but I could also easily have a local copy on any device and sync the changes with git push and git pull.

You can see in the comments of the above post that many people suggested I try Obsidian for this purpose. That post also wound up on HackerNews and a lot of there were plenty of “Duh, this guy is dumb. Just use Obsidian, loser” type comments. (I’m exaggerating, but that was the sentiment.)

In a reply to one of those post comments I wrote:

“yeah, I’ve tried it. lots of bells and whistles, but I prefer the simplicity of vim and github.”

However, I eventually gave in and got deeper into Obsidian. I read all the articles and tutorials. I watched all the videos from all the knowledge management gurus.

Side note: Obsidian and some of these other similar apps and systems have become almost semi-religions. Zettelkasten and Build a Second Brain come to mind. Look those up at your own risk! Yes, there are blogs and whole YouTube channels devoted to this stuff, but there are also books on this stuff. And courses and workshops and personal coaching and foundation memberships you can pay hundreds or thousands of dollars for - each for just one individual school of knowledge management. I read the posts and watched the videos, but I didn’t join any of the cults.

I think I used Obsidian for close to a couple of years, but more recently I’ve stopped using it and I’m going back to my old github based wiki.

Reasons for leaving

Non-portability

Obsidian is often touted as being just pure markdown, like I already had, but with more features built in. The truth is that when you start to use those extra features, you start moving away from pure markdown. And this immediately makes your data less portable.

One example is media-wiki type links. These do indeed make it easier to create links. In fact, you can create a link to a non-existent page and then click on that link and the page will be created. But these types of links will be useless in other markdown systems, like github. When I ported things back over to github, I had to go through a lot of manual link fixing. I later found that there were automated tools that will transform your Obsidian formatted data to pure markdown. But the fact that you need such a tool points up the fact that once you start buying into Obsidian, you start to become locked in. It may take some work to export your data and move it somewhere else.
Another is the fact that Obsidian has a large base of plugins designed to make your data more usable. Some of these just affect how you view your data, or manage it, but some of them require you to write special non-markdown syntax. In fact, there’s a whole query language type thing where you can write a block of … code basically, that pulls in data from all over your documents. It’s super powerful, but completely useless outside of Obsidian. And for a lot of this stuff, there’s no way to export it into pure markdown because it’s all dynamic queries to live data.
Obsidian also has a lot of reliance on “front matter” which is metadata in a special format at the start of a file. I’m pretty sure this is not standard markdown.

Of course, you don’t have to use any of these non-pure-markdown type features. But that’s where all the Obsidian gurus will be pointing you at, in order to get the full power of the tool.

Syncing

Obsidian is an app that runs on your computer. Or on your phone or tablet. It creates a local data file where it stores all your data. But what if you want to see the same data on multiple computers or on your computer and on your phone or tablet. Now you need to sync somehow.

And Obsidian has you covered here! But Obsidian syncing is a paid service, and it is not cheap. As of this writing, it’ll cost you $48 USD a year for one repo, 1GB storage, one month history. For $96, you get 10 repos, a 12 month history and 10GB (or 100GB for $192 a year). The lowest plan is not horrible, but honestly I’m just software-subscriptioned out at this point.

So I decided to make my own sync system. How hard could it be? Turns out, a little bit hard, but let’s walk through it. Obsidian stores all your markdown files in a file tree, and you need to sync the root folder of that to whatever machine you want to use them on. Easy. Dropbox can do that. Syncthing can do that. So could Google Drive or multiple other services.

But, it also creates a .obsidian folder there with a bunch of json files inside it. This all amounts to a kind of manifest of your whole install, all your plugins, preferences, styling, and all your markdown files and their structure. When you change a file, Obsidian notices the change and updates the relevant json files in the .obsidian folder.

So you make some changes on computer A. Obsidian updates the .obsidian stuff and at the same time your sync service (say Dropbox) starts syncing all the changes. Computer B starts getting changes synced and notices markdown file changes so updates its own .obsidian content. But at the same time, computer A’s .obsidian content gets synced. And to make it worse, computer B starts syncing it’s changes in .obsidian and now you have two computers trying to sync the same files, quite often with different content. And this causes conflicts. Just like a git conflict, some of your data is rendered useless. I tried all kinds of timing and delays to try to avoid the conflicts, and tried multiple different syncing systems but always got lots of conflicts. Eventually I just excluded the obsidian.md file from being synced. That eliminated the conflicts, but meant I had to manually set up all my plugins and preferences on each device. Meh. I assume the paid-for syncing service handles this perfectly.

Also, not using Obsidian’s official sync means you have no history of changes. So I also set up a cron job that pushes the whole folder to a github repo a couple times a day.

Obsidian tries to do everything

Again this is my own fault for buying into the hype, but yes, Obsidian tries to be the one tool to organize everything in your life. It’s your knowledge base, it’s your note taking system, it’s your planner and calendar, it’s your research tool, it’s your daily todo list.

And yes, I had all the plugins that let me do all this stuff and used it that way a lot. Particularly at work. I wound up having two Obsidian repos. One for my personal stuff, which was mostly my wiki, but also planning and task management and research, etc. And one repo for work, which was mostly task management, but also meeting notes, planning, knowledge base, etc.

Back to basics

All of this was fine, but it wound up growing into something I never really wanted. But a lot of people thrive on this stuff. In the reddit Obsidian group, you’ll often see people posting images like this:

Each dot is a page in their Obsidian repo. The lines are where one page links to another one. You can drag this around and it has amazing physics where the dots will repel each other and self-organize into the most efficient shape. Totally neat to watch and play with, but it does absolutely NOTHING to help keep me organized. It actually made me want to add more and more content and more and more links between my pages so I could have a really complex network graph too.

I thought back to my earlier comment response: “yeah, I’ve tried it. lots of bells and whistles, but I prefer the simplicity of vim and github.”

I really started feeling this earlier this year. Mostly because I largely stopped using Obsidian other than for my daily todo list at work. In spite of all the nifty plugins and query languages and visualizations of how all my notes connected to all my other notes, etc. I got to the point where I just didn’t even want to open it anymore.

So I saved out my personal repo and converted it back to plain markdown and uploaded it back to github. I’ve been starting to use it more, but it’ll take me a bit of time to fully get back in the habit like I was a couple years ago. That git-based wiki is purely a knowledge base, like it was before. I’m happy with this. Almost totally pure markdown. No overly complex syncing system or cron jobs for history management. No conflicts. I love it.

The right tool for the job

Moving my wiki back to being a straight up knowledge base left me with some gaps to fill.

Note taking

What about one-off notes, meeting notes, capturing random ideas or lists of things that I don’t necessarily need to save long term? I went through a bunch of things trying to find the best solution, including Google Keep, Simple Note, some self hosted solutions. Then I remembered that a while back I paid for a lifetime license for Upnote. I reinstalled it on all my devices (free syncing!!!) and deleted all my ancient notes and started using it on a regular basis. I really like it for this purpose. It has a wonderful UX and it’s a joy to use. I know there are other very similar apps out there, so I’m not trying to sell you on Upnote, just saying that I’m very happy with it.

Task management

And we can’t forget our venerable todo list! What todo? What todo?

I went back to an old standby, todoist.

Honestly, this is the toughest category. Whenever I try a new task management system, I get all excited about it and jam every possible task in there and I’m productive as hell for about two weeks. Then I’m down to the things I don’t really want to do, and I start avoiding it and everything gets backlogged and before long weeks have gone by without me using it. Then I find a new system and the cycle starts over.

Todoist isn’t anything magical. But it’s a solid system. If I’m not using it, I can’t really blame the app. As I’ve said, I’ve used it a few times in the past and I’m happy with it. I just need to commit to not letting it get stale.

Actually, it’s going quite well with todoist at work. I got in a pretty good habit with my Obsidian based todo list at work. Honestly, that’s all I really used Obsidian for in the past six months. So it was just a matter of switching that habit over to todoist and that’s going well. I had a bunch of Obsidian plugins that did things like recurring tasks and rollover/rescheduled tasks, etc. but it was not perfect. Todoist does all this stuff out of the box and does it much better, so I think I’ll be fine there.

I’m also trying to use if more for personal task management and that’s somewhat promising. Having the solid habit at work, and using the same app at home in the same way is helping. I actually have two separate todoist accounts - for work and personal. I got my company to pay for the full subscription for the work version, which I mainly wanted so I could have more than five “projects” which helps split up things into multiple areas. But for personal stuff, the free version is adequate at this point.

Summary

Anyway, long rant about personal crap. If you actually read this far, I am amazed. I hope this did not come across as too angry. Obsidian really is a remarkable tool. And a lot of people love it and get a lot out of it. It’s just not for me, given the reasons given above.

The moral is to do whatever works for you and figure out what works for you. Don’t buy into the hype of this one particular app or system being so amazing that it will do all things for everybody.

Header photo by Alina Vilchenko from Pexels

Joy of Geometry: Part II

kp@bit-101.com (Keith Peters) — Sun, 01 Sep 2024 05:50:32 -0400

Others in this series:

Part I

I moved through a bunch more examples. I am now up to figure 1.24. I backfilled some of the ones I missed yesterday, but there are still a few I skipped.

Figure 1.6 - I’m just convinced this one is wrong. Or it’s at least written incorrectly. Or I am unable to read it correctly.
Figure 1.22 - Just too complex. It would not be clear what was happening and would require a bunch of explanation.
Figure 1.24 - Same.

Maybe I’m just being stupid on the first one and lazy on the other two. Maybe I’ll circle back to them later. But here’s the rest!

Figure 1.12 - This one is pretty interesting, though only applies to equilateral triangles. The text on the bottom is not just static text. It actually does the math on each frame. It really is just exactly 450.00 each time.

Your browser does not support the video tag.

Figure 1.13 - There’s not much to animate here.

Figure 1.16 - You can pause the video and do the math if you want. There may be some rounding errors there.

Your browser does not support the video tag.

Figure 1.17 - The “center of gravity” bit is pretty interesting. I kind of want to cut something out and see if it balances on that point.

Your browser does not support the video tag.

Figure 1.18 - I was going to say that I couldn’t think of a practical use for this kind of thing, then it occurred to me that this relates to a trick for drawing things in perspective.

Your browser does not support the video tag.

Figure 1.19 - A bit of a no-brainer here.

Your browser does not support the video tag.

Figure 1.20 - This relationship applies to any concurrent lines in a triangle like this, whether altitudes, medians or bisectors. Maybe others? In the case of medians like this, it’s also a bit of a no-brainer, since the median bisects each side and each red/green segment pair is equal.

Your browser does not support the video tag.

Figure 1.21 - Something about the incircle strikes me as very elegant.

Your browser does not support the video tag.

Figure 1.23 - This one feels a bit contrived. “A long walk for a short glass of water” as they say… but it looks cool.

Your browser does not support the video tag.

Joy of Geometry: Part I

kp@bit-101.com (Keith Peters) — Sat, 31 Aug 2024 10:10:20 -0400

Others in this series:

Part II

Some while back (maybe a couple of years or more ago) I picked up the book, Joy of Geometry by Alfred Posamentier.

I’ve dived into it several times, coding up the examples he gives in the book and it’s been quite fun. The first chapter is “Concurrent Lines” and there is a seemingly endless array of examples of relationships that are formed by different parts of geometric shapes - mostly triangles so far.

In the past week, I’ve been working through these a bit more diligently. The book is 150 pages, and I’m only up to page 9, but I’ve learned a lot and created several new functions in my geometry library already. I haven’t coded up every example from the first nine pages so far, but I’ve done the majority. I’ve included a bit of explanatory text on each one of them. But I highly recomment actually getting a copy of the book if you’re interested in this stuff. There are much deeper explanations of these concepts in the book.

Chapter 1: Concurrent Lines

Note, this is only a fraction of the examples in chapter one. I’ll be continuing to work through these and will post more in another post.

Figures 1.1, 1.2, 1.3 - The orthocenter.

Your browser does not support the video tag.

Figure 1.4 - Three non-collinear points will form a unique circle. This is known as the circumcircle of the triangle formed by those three points. The center of that circle is the circumcenter of the triangle.

Your browser does not support the video tag.

Figure 1.5 - The altitude as a bisector.

Your browser does not support the video tag.

Figure 1.7 - The side of a triangle as a bisector.

Your browser does not support the video tag.

Figure 1.8 - A corner of a triangle as a bisector.

Your browser does not support the video tag.

Figure 1.9 - Similar triangles. Introducing the orthic triangle.

Your browser does not support the video tag.

Figure 1.10 - Relationships with the orthic triangle.

Your browser does not support the video tag.

Figure 1.11 - More on the orthic triangle.

Your browser does not support the video tag.

Figure 1.14 - Another magical property of the orthocenter.

Your browser does not support the video tag.

Figure 1.15 - Congruent circles with the orthocenter.

Your browser does not support the video tag.

Questioning Sierpinski

kp@bit-101.com (Keith Peters) — Wed, 17 Jul 2024 19:55:18 -0400

I had a stray thought about the Sierpinski triangle this morning. A different way of looking at it. I’m under no illusions that I’m the first one to try this idea out, but in the 35+ years I’ve been obsessed with fractals, chaos and strange attractors, I’ve never run across any examples like this, which is surprising, because it’s a very simple idea.

I tried the idea out and found it created some very nice patterns, some very Sierpinski-like, and some quite unexpected. So I figured I’d share.

Background

A quick refresher on the Sierpinski triangle. I assume you all know how it works, so I’ll be brief.

You take an equilateral triangle, split it into four equal triangles and remove the center one.

Then you do the same for the three remaining triangles.

And so on, down to the level where the scale is so small you’re not seeing the results anymore.

You know all this.

Question Authority

My stray thought this morning was, “why do we always remove the center triangle?”

What happens if we start removing other triangles instead?

Let’s try it. We’ll remove the top triangle.

Note: there’s some artifacting here - you can see a faint line between each rendered triangle. Though not initially intended, it kind of works to my advantage, letting you see the different triangles to some degree.

OK, fine. Now when we go to the next level, we have that center triangle still. But when we split that up, it’s not going to have a “top triangle”. Well, we can just say that triangle was rotated 180 degrees, so we rotate the algorithm by 180 degrees, and remove the bottom triangle. Now level two looks like this.

The next level…

And all the way down (visibly anyway)

Now what happens if instead or removing the top triangle, we always remove the bottom right?

Or the bottom left?

It’s just the same thing, rotated. Makes sense.

So far, it’s still vaguely Sierpinski-ish, but… there’s a lot of other stuff going on there.

Keep Questioning

We made an easy assumption there when we decided how to deal with the upside-down center triangle. We said it was just rotated 180 degrees. But it could have been rotated 60 degrees to the left or 60 degrees to the right and also appear as an “upside down” triangle. Now our “top”, “bottom left”, bottom right” terminology starts to get confusing. So let’s label the different triangles A, B, C and D.

In the conventional Sierpinski, we always remove D. So how we rotate the center triangle doesn’t matter. When we’re removing A, B or C, it starts to make a big difference. We have three possibilities, with three different outcomes. We can rotate the center triangle so that A is at the bottom, or B is at the bottom or C is at the bottom.

In the example we did above, we had A at the bottom. But let’s rotate the triangle 60 degrees clockwise so B is at the bottom like this:

Removing the top triangle, one iteration looks no different.

But two iterations…

Three…

Four…

And finally…

Now we’ve got something very different!

Now let’s do the same thing, but rotating the center triangle so C is at the bottom.

And removing the top triangle and rendering that all out…

We have the same image, flipped.

Just to be clear, we can still remove the bottom right triangle (B) or bottom left (C) and we get the same image rotated:

Question Me

Earlier, I said when we had three choices on how to orient the center triangle, I lied.

There are three choices if we only rotate. But we can also mirror that triangle as we rotate it. Which gives us 3 more possibilities:

Let’s take the first one first, removing the top triangle.

No difference from what we originally had. Actually, it is flipped on the x-axis, but since it’s symmetrical, you can’t tell.

But when we remove the right or left triangles, we get something brand new.

Now we have a bunch of squashed Sierpinskis lined up. Neat!

What happens when we put B on the bottom?

Removing the top triangle…

Again we have infinite Sierpinskis lined up, but oriented differently and skewed a bit.

Bottom right…

Bottom left…

This one goes back to the original again.

Finally, for the sake of completeness, we’ll do the same with C on the bottom, removing top, bottom right, bottom left:

All the same, but differently oriented.

OK, we got all six orientations of the center triangle covered. So we’re all done, right?

Not even close!

Question Harder!

What about each of those three outer triangles? So far we’ve been orienting them all the same as the original: A on top, B on the bottom right, C on the bottom left.

But each one of those could be oriented in six different ways. That’s four triangles, each of which could be oriented six ways. 6x6x6x6, or 1296 possible variations!

Not to mention the fact that you can choose which triangle to remove - A, B or C, which triples the possibilities. Of course, many of the iterations will be the same as others, some just mirrored or rotated a bit differently, but there are still quite a few interesting ones.

I’m not going to go through all of them, but here’s a few neat ones I found:

Some of these are looking curiously like Julia fractals!

Question the Question

Everything is too stable. We’re always removing the same triangle throughout all the iterations of an image. I took just ONE of the examples from the above list and altered it so that on each iteration, it randomly decides which triangle to remove.

And that’s only the start. You could randomly choose which order each triangle is in - in each iteration. Or not randomly - do it according to some scheme. I feel like I’ll be exploring this one for a while.

Question the End

After posting, I kept finding more nice looking ones. Here’s a few more.

Image Sequence Viewer - isv

kp@bit-101.com (Keith Peters) — Fri, 28 Jun 2024 21:18:36 -0400

This is a project I am really happy about. I don’t know if anyone else will ever use it and that’s fine. It scratches a major itch for me and that’s great. I really love crafting my own development tooling so I have an environment that works just like I want it to. In this case, there were no tools that I could find to do what I wanted to do, so I made one. And it works!

Background - my workflow

I create animations. In code. Generally I write the code for a single frame of an animation. This has a percentage variable passed to it that ranges from 0.0 to 1.0, depending which frame is being rendered. All the animation is based on that single variable and allows me to create perfectly looping animations.

I render each frame out as a .png file in a frames folder, then run ffmpeg on those frames to output either an animated gif, or more recently, an mp4 video. A single animation can have several hundred frames, and if the code is complex, can take a few minutes to render.

I don’t want to wait all that time to see what the animation is going to look like. Maybe it’s all wrong and I’ll be able to see that within a few frames, cancel, fix things and re-render.

Everything I do is on the command line. I code in neovim, in a terminal, compile my code with the Go toolchain, run ffmpeg to create the animation, which then launches a gif or video viewing program to play the result. So there’s no IDE with automatic previews of what’s going on. But I do have that folder of .png s sitting there.

Early on, if I wanted a preview, I’d open up a new pane in tmux and navigate to the frames folder, then open the first file there in an image viewer. Eventually I got smart and made a vim mapping, ,p that will auto-launch the image viewer with the first frame. From there I could use the left and right arrows to scroll through each frame and see how things were coming along.

This worked great for a while, but it requires holding down the arrow key. And sometimes not enough is rendered, so it’s jumping from the last frame to the first, which is jarring, and I’ll just wait for a while, then go back to holding the key again, until enough frames were rendered to give a good representation of the animation.

Not awful, but I had this idea of an application custom build for my use case. So I made it.

Enter isv

isv is an image sequence viewer. So you have a sequence of image files that will be eventually compiled into a standalone animation. This lets you play that animation back before it is compiled. It is NOT at all optimized for large images and is not intended to be a general purpose image viewer. But it’s great at playing back image sequences of frames up to a few hundred kb each. This is fine for me because even if I’m doing a higher resolution project, I’ll usually start by rendering it in low res at first to work out all the details, and only switch over to the final version when everything is worked out.

Usage

Playing

isv is meant to be run from the command line, even though it opens up a gui window. So it’s not going to work in a terminal only vm or while sshing into a remote machine. You can either open it up in the directory where your frames are, by typing isv there, or giving it a path to where those frames are: isv path/to/frames. It will automatically load and display the first image (png, jpg or jpeg) it finds. You can then use it just like I described using any other image viewer - scrolling back and forth through frames using the left and right arrow keys. When it hits the last frame it will loop around to the first, and vice-versa if going backwards. Other minor features include hitting f to go to the first frame and l to go to the last. So far though, that’s no better than what I had.

But hit p and the frames start auto-playing through just like a video player, auto-looping when it gets to the end. Hit p again, or space, and they stop. Same thing with r - it will play the frames in reverse. And even better, b will play in “bounce” mode. The animation will play forward to the last frame, then bounce back and play in reverse to the first frame, over and over. This is really good in the early stages of a render where there are fewer frames. Rather than that jarring jump back to frame one, it bounces forward and back smoothly and is a much better experience. As you get a few seconds into the animation, you may decide to switch it back to regular play mode.

By default, isv attempts to play the frames back at 30 fps. Really just a bit slower than that, since all it really does is put a 33 ms delay in between each frame and doesn’t account for any other overhead. But this is just a preview, so I decided not to overcomplicate it. Using the up and down arrow keys you can increase the speed to 60 fps (aspirational - you very well may never achieve that) or unlimited (no delay), and decrease the speed to 10 fps, 5 fps and 1 fps, in case you want to inspect the details of your animation in slow motion.

The status bar will report the current play mode and attempted fps.

Watching

When isv launches, it reads the directory and makes a list of the supported image files there. This is what it starts playing through. But if you’re actively rendering the frames of your animation, you will want to update that list from time to time.

Hit w and it will start refreshing the image list every four seconds. Hit it again to stop watching. Of course this is not really watching, in the sense of an event-driven system where it knows when the directory contents are changed. It’s just reading the directory again. I may eventually do real file system watching, but it may be overcomplicating things, as it seems to work pretty well as it is.

If you want to refresh the list more often, hit the < key (technically , since you don’t need to shift). Or to refresh less often, > (or . ). These will set the update interval in one-second intervals from 1-10.

The watch state and timing data is also displayed in the status bar.

Command line flags

By design, I didn’t want to assume anything when you start isv , so it doesn’t auto-play or auto-watch by default. But you can do any of that with command line flags.

-p will have isv auto-play when it starts. Likewise -r and -b will play in reverse or bounce mode upon startup.

-w n will start watching the directory for changes every n seconds. i.e. -w 2 will refresh the list every 2 seconds.

You can combine these like isv -p -w 2 or even isv -pw 2.

In my vim mapping to preview the currently rendering animation I set it up as isv -bw 2 & so it starts in bounce mode and refreshes every 2 seconds. The final & has it run as a background thread so it doesn’t block my vim output.

isv -h will of course show the full help file, which includes the command line flags as well as the keyboard shortcuts available while playing.

Behind the scenes

isv was built with fyne (https://fyne.io/) which is a cross platform gui framework for Go. I found it super simple to get started with and really easy to figure out just about everything I needed to. The only real hard part was the “cross platform” part. Normally cross-compiling Go code is a piece of cake. But since fyne links to external C libraries for native ui elements, it needs access to a C compiler such as gcc or clang even to compile on your current OS. Compiling for a different OS is even more complex. They also have a tool called fyne-cross (https://github.com/fyne-io/fyne-cross) which helps, but can get pretty complex in itself. I was able to compile Linux on Linux and Windows on Linux. But I had to compile the MacOS version on a Mac. Even still Apple now quarantines unsigned apps on their new arm64 systems and you can only fix it on the command line, not in the control panel like you used to. (Instructions in the readme in the repo listed below.) So if you want to install a pre-compiled binary on your M1, you’ll have to jump through some hoops. Sorry, but I’m not paying Apple so that you can install my app on your computer.

Source

Of course, this is all open source. You can check out the code and try to build it for yourself, try to install it through the go install mechanism, which is pretty sweet, or just grab a binary release:

https://github.com/bit101/isv

At this point, I’m pretty much done with development on isv. I’ll be using it daily in my animation creation and fixing any bugs that might crop up or adding any new features that spring to my mind. But for now it does just what I want.

June Update

kp@bit-101.com (Keith Peters) — Thu, 27 Jun 2024 20:07:53 -0400

Lots going on this past month. I’ve been lax in writing about it, largely because writing long form content has been tough, one-handed.

Shoulder

My last update was about my shoulder surgery. This has been going quite well. I’m just wrapping up week four now and the overall level of pain has been far, far less than I could have expected. I found I didn’t really need the heavy pain killers at all. Mostly got by with ibuprofen and ice. And after a week or so I started only taking that as needed.

I won’t subject this to you directly, but if you’re interested in an internal shot of my shoulder mid-surgery click below. I find it fascinating.

Warning! Internal surgery photo.

Not being able to move my arm 23+ hours a day has started to make it pretty stiff. I started physical therapy a week and a half ago. While that’s helping range of motion, the combo of the two is making the shoulder a little painful on some days. But less than one week left in the sling! Honestly, a little nervous to take it off. As annoying as it is, it’s also been a bit of a security blanket. How can I be trusted to use my own arm without something protecting it?

I’m sure I’ll be fine.

Wire, ArtFromCode, Strange Attractors

Last month I wrote a lot about my wire library https://github.com/bit101/wire. I’ve had a lot of fun creating daily animations for 41 straight days and posting them on https://artfromcode.com . This past week I decided I needed to take a little bit of a break from the daily posts to explore some other ideas, but you’ll surely see more wire pieces in the future.

There are two neat updates to wire. One is the ability to load and save models I’ve created and to load in existing 3rd party point cloud models. Currently in terms of 3rd party models it only loads .xyz files. These were originally created for modeling molecules. It’s a simple text format where each line is an x, y, z vertex, with an optional element name, and some optional metadata at the top, so parsing it was fairly easy. Although it’s not a super common format, there are on-line converters that will take 3d models in many other formats and save them down to .xyz files. I considered building something to save models in .xyz format, but decided to create my own format which includes both points and segments. This is how objects are represented in wire. So you can save and load from that format. There’s also the .obj format which, although it can be way more complex, can be reduced to saving points and edges. This would work, and maybe I’ll look into that someday.

The other addition to wire is a new strange attractor library, lures - https://github.com/bit101/lures. Lures… attractors… yeah. This is a super lightweight library. It’s really just a collection of 2d and 3d strange attractor formulas with a mechanism for setting properties and iterating through them to create a series of 2d or 3d points. It doesn’t do any rendering at all, just generates the numbers. But wire can now generate a model from a lure formula and make a point cloud with as many points as you specify. You’ll see a lot of examples of that in the https://artfromcode.com entries in the past several weeks.

Your browser does not support the video tag.

isv - Image Sequence Viewer

This is one of the reasons I wanted to take a break from daily animations. I’ll do a whole post on this, but this is an idea that I’ve been thinking of for a long time and finally decided to sit down and create it. It 100% scratches an itch and I’ve added it to my creative workflow and love it. More to come, but if you want a bit more info… https://github.com/bit101/isv

Music / Audio

I restarted my exploration into creating music or at least soundscapes to go along with longer creative coding videos. Again using SuperCollider. I bought a little shiny thing that I’ve been eyeing for a long time - a Midi Fighter Twister. I think this will also be worthy of an entire post, along with the library I created for interfacing with it from SuperCollider.

But essentially, it’s a box with 16 midi knobs in a 4x4 arrangement. But it is amazingly customizable. Each knob can spin infinitely in any direction, so there is no set minimum and maximum orientation. The value for each knob is indicated by a series of white LEDs around the knob. This means that you can dynamically set the value of a knob. The knobs are also push switches and are configurable in four banks, so it’s really like having 64 knobs and 64 buttons. In fact, you can set it up so that pushing and turning acts as a separate control, so you could have 128 knobs. And there are 6 configurable buttons on the sides. And… well yeah, I’ll make another whole post about this.

I spent a few days figuring out exactly how to interface with this thing in SuperCollider and wound up making my own library: https://github.com/bit101/sc_mft that lets you access just about all the features through a single object. This has also been really fun. Though I admit I kind of abandoned this over the past few days working on isv.

Eventually there should be some sounds to listen to. As much as I am fine with vomiting out a million images or animations, I’m extremely shy about my musical creations.

Summary

I guess that’s about it. Three new repos, each of which I’m pretty excited about. A couple more blog posts to write in the coming days. And a million more ideas floating around in my head. I can’t complain.

Surgery

kp@bit-101.com (Keith Peters) — Fri, 31 May 2024 07:53:39 -0400

One month ago I posted about my rotator cuff.

And a few days later about my MRI.

Bringing things up to date, spoke to my specialist and surgery was an option. She didn’t push it hard, offered other options, and made it clear that surgery was the toughest one to go through. But in discussing it, it was also clear that if I just wanted to deal with it once and for all, surgery was the way to go.

Factors I considered:

The amount of pain and discomfort I’ve been in on a daily basis.
The growing level of disability this was causing: problems driving, eating, dressing, showering, walking the dog, etc.
The sleep impact. Almost every morning I’ve been awakened with throbbing pain at 3:00 or 4:00 AM, sometimes earlier. Sometimes I can find a better position and get a bit more sleep. Sometimes I struggle and give up and just get up and start my day, maybe try to sneak in a nap later. I think I’ve had maybe 2-3 decent nights sleep in the last couple months.
The lack of solid gains on previous treatment attempts.

Given all that, I scheduled the surgery and had it yesterday. It’s a relatively short outpatient procedure. Started with a nerve block injection, which numbs and paralyzes the whole area. That lasts up to 24 hours. Then general anesthesia. She said if the tendon was torn less than 50% through she’d just abrade the edges of the tear so they would heal better. As it turns out, it was a 70% tear. So it was a full repair. This meant drilling a hole and placing an anchor in the bone, then attaching a couple of sutures to the anchor to reattach the tendon. Honestly I’m glad she did the full repair. Also, 2-3 anchors is not uncommon in larger tears, so just one is not bad.

Went home with “the Cadillac of slings”. It’s huge with a memory foam pillow to keep my arm in the right position and has an integrated exercise ball to keep my hand and wrist bust. I’ll be wearing this 24/7 for the next five weeks. So I’m glad that it’s pretty comfortable - at least it feels like that now.

A couple different painkillers, but as of this writing the nerve block has just pretty much worn off. Zero pain all night and so far only minor discomfort this morning.

I took off work for a week and a half, so I can just relax and heal and get used to doing everything left-handed. Typing is slooooow.

So Day zero. Post-day one beginning.

I would like to add that absolutely every single person I dealt with at the hospital was amazing. Friendly, explained everything, answered every question I had and checked for more. Health care workers sometimes get a bad rap and I imagine in some places they are under a lot of pressure and that might sometimes affect how they deal with people. But they are heros.

Funny anecdote: as I was waking up from the anesthesia, I had the distinct impression that there were three beds lined up together, with three of me - one in each bed. Later, when a nurse asked how I was doing I almost told her that I thought I was doing good because there was only one of me. It briefly seemed very important, but then I realized at best it would get an amused laugh.

Art From Code

kp@bit-101.com (Keith Peters) — Mon, 27 May 2024 09:35:38 -0400

Background

Many years ago I started a site called Art From Code. You can see the archived version at archive.artfromcode.com.

I’ve updated this site in fits and starts but (as I described in the intro to this blog) Wordpress had just become not fun anymore. So I made yet another Hugo site, moved the old Wordpress site to a subdomain and made the new one live. The new site:

artfromcode.com

One of the main reasons for this change was a need to better document and preserve what I’m working on. Over the past few years, I’ve done a lot of random experiments and posted a few (or a ton) of images or animations on the site formally represented by a bird, or more recently on Mastodon.

Generally I just start a project and riff on it until something cool comes out, post a render, and start riffing some more. The code for any particular image is lost immediately, and the image is only preserved on social media.

The Problems

Here are the problems with my workflow up to this point:

No real curation. I’m just spewing a half dozen images into the void in a single evening. I fear that people might get sick of the firehose. Some are great, but some… meh. There’s a principle in photography that you should take as many pictures as you can, but only publish the really great ones.
No record of the outcomes. No gallery or portfolio that I can point to and say I did this piece on this day. Sometimes I’ll also blog things, like in the genuary series. But in general, the pieces go off into social media and (maybe) stay there. But they are mostly undiscoverable after a few weeks.
No record of what created those outcomes. The code only shows what created the most recent image. What parameters were used to create the one three iterations earlier? Who knows?

The Solution

The new workflow:

I have a sandbox project which is 100% throwaway code. This is where I can riff to my heart’s content, break things, wipe things out, start over, whatever.
When I come up with something that starts to look cool, I copy that code over to a new project in the code folder. The whole code folder is under source control.
From that location, I continue to refine the piece until it satisfies me.
The source code gets checked in.
The output gets archived and backed up in a media folder.
I create a post on Art From Code with the new piece and schedule it so I’m doing generally one post a day. Some days I don’t do much, but on weekends I might crank out half a dozen pieces and schedule them. I like to stay ahead. This all happens in the site folder.
On the day the piece goes live on the site, I post to Mastodon.
I’ve also been posting to a new Tumblr blog, as a secondary source of discovery.
And I’m starting to think again about longer form videos with audio. Those are in the longform folder.

But…

It’s going well!

I’ve posted something every day for a while and have more lined up.

I’m not flooding Mastodon with a ton of stuff every day - just the daily scheduled piece. But I still do post some random explorations there - nothing that I feel the need to save, just random interesting experiments.

The pieces that I am doing have way more thought and effort put into them. There is often a point where something looks OK and I probably would have posted it to Mastodon, but I feel like it’s not really good enough to be the official daily piece. So I work it over until I’m really happy with it. Occasionally I’ve even abandoned a particular piece when I couldn’t quite get it to where I wanted it. So my standards are raised.

I’ve upped my dependency management game. I have several self-created libraries I use for most of my work. In Go, you can import a specific version of a library officially from its published location. But you can also just use a library by pointing to the local file system location of its source code. Doing this, you get whatever code happens to be in that repo at that particular time. This is super useful for experimenting. If something in my library doesn’t work just the way I want it to, or it’s missing a function, I can just add or update that function and it’s instantly available to the project. But this is a horrible practice for finished pieces because you have no idea what version of the library that piece was created with. This has gotten my to link each daily project to a specific version of each library. If I need to add or change something in a lib, I update its version and update the project to use that version. A bit more friction, but well worth it.

Wire: Part IV - Points

kp@bit-101.com (Keith Peters) — Sat, 18 May 2024 19:15:28 -0400

Points are a core data type in the wire library. Every shape has a list of 3d points, which generally make up paths by joining up pairs of points in segments.

But rendering points all by themselves is also an interesting technique. All you have to do is call shape.RenderPoints(radius). This loops through the point array, projecting each 3d point to a 2d point and drawing a circle of the given radius at that location. Of course, that radius is also scaled via perspective, so the final rendered radius will vary depending on the point’s z value. Here’s an example. I can create a sphere and draw all its segments by calling sphere.Stroke():

Your browser does not support the video tag.

With the same sphere, I can just call RenderPoints instead and get this:

Your browser does not support the video tag.

Here, I put a heavy blur on to make the points show up a bit better.

You can even do strokes and points:

Your browser does not support the video tag.

You can also just create an empty shape and add points to it:

shape := wire.NewShape()
for range 5000 {
  shape.AddXYZ(
    random.FloatRange(-300, 300),
    random.FloatRange(-300, 300),
    random.FloatRange(-300, 300),
  )
}

Your browser does not support the video tag.

In fact, there’s a method to do that even more easily:

shape := wire.NewShape()
for range 5000 {
  shape.AddRandomPointInBox(600, 600, 600)
}

But why stop with boxes? There’s a similar method for creating random points in a sphere. But I went one further than that and just made a method that lets you say how many random points you want in that sphere and it does it all for you. Just specify the radius of the sphere and the number of points.

shape := wire.RandomInnerSphere(400, 5000)

Your browser does not support the video tag.

This leads us in an obvious direction - what about random points ON a sphere? Got you covered:

shape := wire.RandomSurfaceSphere(400, 5000)

Your browser does not support the video tag.

Some Code?

I want to take a quick break and look at some of this sphere code because it’s super useful even if you don’t use this library. How do you create random points on the surface of a sphere? It’s a harder question than it seems at first glance. A naive method would be generate a point at an x, y, z of radius, 0, 0, then rotate it around the y-axis at a random angle between 0 and 2 * PI. Then rotate it around the z-axis at a random angle from -PI to +PI. Or something like that. But that will leave a concentration of points at the poles, rather than having them evenly distributed. Other methods will leave other artifacts.

I’ve done this a number of times and always spend too much time looking it up. So I made sure I bookmarked a good reference for it this time.

https://mathworld.wolfram.com/SpherePointPicking.html

And here’s the actual code that does it in the lib (yes, some room for optimization here, now that I look at it!):

func RandomPointOnSphere(radius float64) *Point {
  u := random.FloatRange(-1, 1)
  t := random.Angle() // returns a number from 0 to 2*PI
  x := math.Sqrt(1-u*u) * math.Cos(t)
  y := math.Sqrt(1-u*u) * math.Sin(t)
  z := u
  return NewPoint(x*radius, y*radius, z*radius)
}

Getting a random point INSIDE a sphere is another interesting problem. You might try multiplying the radius value by a random value from 0 to 1. But if you do that, here’s what you get:

Your browser does not support the video tag.

While the points ARE evenly distributed along the radius, they wind up being more cramped near the center of the sphere, and more sparse near the outer edge. It’s a neat effect in itself, but if you want an even distribution, you have a problem. The solution is to multiply radius by the ~~square~~ cube root of a random value from 0 to 1. (See note at end of post.) This will have the effect of adding a bias to larger radii, evening out the visual distribution.

func RandomPointInSphere(radius float64) *Point {
  u := random.FloatRange(-1, 1)
  t := random.Angle()
  x := math.Sqrt(1-u*u) * math.Cos(t)
  y := math.Sqrt(1-u*u) * math.Sin(t)
  z := u
  radius = math.Pow(random.Float(), 1.0/3.0) * radius
  return NewPoint(x*radius, y*radius, z*radius)
}

Yet another side quest

One more note about that problem of clustering points around the poles of a sphere, I actually run into the same thing rendering a non-random sphere. Look at the first couple of animations at the top of this post and you’ll see what I mean. The rings at the two poles have the same number of points in them as the points on the equator, so the poles render a bit more bright than the rest of the sphere. I could use the same trick I used here for random points, but after actually trying it, I found that it looks worse. It feels like the poles are artificially sparse. But the bright poles look kind of normal. To me anyway. So I’m keeping it.

Where were we?

There are other random point functions for creating random points on or in a cylinder or a torus. And I’ll probably continue to add other similar functions for other random point placements.

Your browser does not support the video tag. Your browser does not support the video tag.

The code for these is interesting too, but I’ll leave it to you to examine the code if you so desire.

There’s one more challenge on some of these that I’ll be tackling soon. It has to do with points on various surfaces. You might notice that on the cylinder example, there are no points on the caps of the cylinder, just on the body. Putting points on the caps is no problem, but how many points to put there? Basically I’ll have to figure out the surface area of the body of the cylinder, and then the surface area of each cap itself, then distribute the number of points accordingly. If each cap contributes 10% of the surface of the cylinder, then each one should get 10% of the points, and the body gets 80%.

There will be a similar challenge for a box. The left and right sides will get a certain percentage each, top and bottom another percentage, and front and back another. Not rocket science, but I haven’t gotten around to coding these cases up yet.

A cone presents another set of challenges I haven’t even started to think about yet. But it should be interesting.

Summary

Well that’s a bit about rendering 3d points. Hopefully some interesting stuff even if you don’t use the library.

Note

Earlier I said that to correctly distribute random points within a sphere, you should take the square root of a random value between 0 and 1, and multiply that by the radius.

Thankfully, I was corrected on this point. You use square root for 2d. For a sphere, you’ll want to use the cube root. I fixed the text and code and image above. But it’s worth looking at the different methods and what they create.

Multiplying by a random number:

Your browser does not support the video tag.

Multiplying by the square root of a random number:

Your browser does not support the video tag.

Multiplying by the cube root of a random number:

Your browser does not support the video tag.

Obviously the last one is “correct” as far as making an even distribution.

In fact, here’s an animation moving between a single random number and the 16th root of a random number, which effectively hollows out the sphere. Mmmm… math!

Your browser does not support the video tag.

Wire: Part III - Evolution

kp@bit-101.com (Keith Peters) — Fri, 17 May 2024 08:10:56 -0400

I’ve been a bit quiet here, but have been very much heads down working on this wire library. I refactored (or more correctly, rewrote) a lot of the core code and added a bunch of new features.

Code Changes

Points and Paths

The biggest fundamental change is in how I store paths. Many objects consist of multiple paths. Consider this torus:

Your browser does not support the video tag.

Each ring in the torus is a single path composed of a list of points that is stroked. Initially, I stored each path as a separate list of 3d points and then projected those points and stroked a line between each point in the path. This was a good way to get started, but problems cropped up. Imagine a 3d cube. There’s no way to draw it in a single path without overlapping, so you get multiple paths, but no matter how you arrange it, you’re going to need to use the same point in multiple paths. I have a set of randomize functions that randomly moves each point to shake up the shape. This meant that those shared points would diverge and the cube would break apart. You might try to share the same actual point object in multiple paths, but then when you transform the shape, you’d be transforming some points multiple times.

Also, having a list of points as a path turned out to be not so useful. When I draw each path, I scale the width of each line segment in the path and draw it separately to make sure each one is scaled separately. So it’s moveto, lineto, stroke, moveto, lineto, stroke, ... rather than moveto, lineto, lineto, lineto, ..., stroke.

Due to these (and other) issues, I changed each shape to have a single list of points and a single list of segments. A segment connects two points. This was a pretty extensive refactor, but simplified a lot and has proven to be well worth it. In the cube example, there are only eight points, and twelve segments connecting them. Transformations and randomizations just affect those eight points and each segment continues to connect the same two points, so it all works out well.

Operations like cloning a shape did become a bit more complex. Rather than just cloning each separate path, I had to clone the list of points, and then make a new segment list that connected the correct pairs of points. It might have been easier if segments contained two indices to points rather than references to point objects, but I’m glad I kept it that way because it made other operations easier.

Colors and Z-sorting

I toyed with the idea of having each point contain its own color. This required z-sorting the list of points before drawing them. I got it working for drawing points, but realized that drawing segments this way was going to be more complex. And then, to draw multiple shapes, you’d have to pool all the points and segments and z-sort them all together and draw them in order.

In the end I decided I liked the constraint of monochrome. So I removed the color and z-sorting code. It might come back at some point, but I’m happy with the decision for now.

Fog

I implemented fog. This just sets a nearFog and farFog value and linearly interpolates between them, changing the drawing alpha. Anything with a z value less than or equal to nearFog is drawn with full opacity. Anything further away than farFog gets zero opacity. This is a nice effect that was pretty easy to create.

Your browser does not support the video tag.

World

I updated this world singleton so all client operations on it are via methods.

Shape Updates

I updated a bunch of shapes. In particular, the sphere now has options to draw latitude lines, longitude lines, or both.

Your browser does not support the video tag.

The cylinder, cone and torus have similar options.

Your browser does not support the video tag. Your browser does not support the video tag. Your browser does not support the video tag.

I also added a couple of new shapes, grid plane and spring.

Your browser does not support the video tag.

Finally, there is now a way to combine shapes using shape.AddShape(otherShape). This simply appends the points and segments onto the points and segments of the parent shape. Amazingly simple, but now you have a single, complex shape that you can transform and draw simply. This is one area where I’m glad I kept segments containing references to points, not indices to points. Doing it this way means I can just append the new segments to the existing segments, not have to worry about re-indexing them.

There are a few other goodies I threw in there as well, but I’ll probably cover those later.

A New(ly updated) Site!

A bit of a teaser here as I’ll have another post on this soon, but I updated my artfromcode.com site and it has a bunch of neat wire examples there so far. Been working on updating it daily.

Wire: Part II - Rendering

kp@bit-101.com (Keith Peters) — Wed, 08 May 2024 07:27:01 -0400

Aliasing

In the first post of this series, I showed some of the default renders of various 3d shapes. These were rendered with white lines on a black background. Here’s how that looked for a cylinder shape:

Your browser does not support the video tag.

Not bad, but there’s more aliasing of the white lines than I’d like. To quote someone I used to work with, it looks “crunchy”. My libraries use CairoGraphics, which has a few settings for antialiasing. This is on the default. There’s also subpixel antialiasing and gray antialiasing (supposed to be good for black and white graphics). Neither of these makes it look much better for me. But let’s compare what happens when we turn antialiasing off completely:

Your browser does not support the video tag.

Even worse, as expected. All of this is just a natural side effect of drawing very thin, very contrasting lines. But I developed a trick for making these kinds of graphics look pretty cool.

Glow Effect

It goes like this:

Draw the shape using a thicker stroke than you really want.
Apply a strong gaussian blur.
Redraw the shape with a narrower stroke.

This gives the graphics a very cool, very customizable glow effect. Step-by-step:

Thick stroke:

Your browser does not support the video tag.

Blur applied:

Your browser does not support the video tag.

Thinner stroke on top of the blur:

Your browser does not support the video tag.

To me, this smooths things out nicely and gets rid of most of the “crunchiness”. It’s subtle here, but you now have three control values to change the effect: the original stroke size, the amount of blur, and the final stroke size. In this example I used an initial stroke of 2 pixels, a blur radius of 10 and a final stroke of 1.

In this example, I increased the initial stroke and blur radius quite a bit.

Your browser does not support the video tag.

I’d actually consider the glow here to be too much, but it’s a tuneable effect, so it’s good to experiment and see what the parameters do.

Inverting Colors

Of course, there’s no need to have things always be white on black. We can do black on white:

Your browser does not support the video tag.

And we can apply the same blur effect on that.

Your browser does not support the video tag.

I really like this look. I find that when you switch from white on black to black on white, you need to tone down the blur a lot, or it looks muddy.

Colors

And of course, you can use any colors you want, not just black and white. Here I’m using the String object to display the hue value I’m using in an HSV color model. And a glow.

Your browser does not support the video tag.

This one gives me a bit of a Blade Runner vibe. Our eyes see different hues differently, so some of the colors aren’t very visible on a black background - particularly the blues. You could compensate with wider lines, more glow, etc.

Summary

These are just some of the rendering tricks I’ve been using. They’re very applicable well beyond this particular library. So hopefully you can try them out on whatever system you’re using and get some fun results.

Wire: Part I - Intro

kp@bit-101.com (Keith Peters) — Tue, 07 May 2024 14:52:53 -0400

Over the last couple of weeks I’ve been working on a somewhat unique 3d library. I call it wire. It’s written in Go and is designed to work with my own blcairo and bitlib libraries.

What’s somewhat unique about this library is that it doesn’t support drawing 3d solids or even filled triangles. It’s all about drawing 3d lines or paths … or wires if you will. It will also draw indiviual 3d points. It also purposely does not do any z-sorting and is really designed to be monocrhomatic so that it doesn’t matter what order lines and points are drawn.

To get us on the same page on what I mean before we dive in deeper, here’s an example.

Your browser does not support the video tag.

Primitives

There are a small set of primitive objects that everything else is made from.

Point

At the base of everything is a 3d point. This just has an x, y, z position and a bunch of methods that can operate on it. Your usual selection of 3d transform methods and some others, and of course a method that projects the 3d point to a 2d screen coordinate.

PointList

Points then go into point lists. These are used either to create a single path of “wires” or to render a bunch of individual points. All of the point transform methods also apply to the point lists. So you can rotate, scale, translate, etc. an entire list of points with one command. This follows the same pattern I described in an earlier post on 2d points. And there are methods to render the list of points as a single path or do draw each point individually.

Shape

One step up from there is a shape. A shape is a list of point lists. Some shapes can be drawn with a single path - a circle for example. Others need multiple paths, like a 3d box - you could do it in a single path, but you’d have to retrace lines, which would make some lines darker/brighter. There are multiple ways to solve this. I went with multiple paths. You can also call just about any point method on a shape, which will cascade down and operate on every point in every path in that shape. And then you can draw that shape in the same way. For a quick snippet of what this looks like, here’s how you could draw a 3d box.

box := wire.Box(300, 300, 300)
box.Rotate(0.2, 0.5, 0)
box.Stroke(context)

World

Finally, there is a world object. This contains values that help define how things are rendered, like the center point of the world, the perspective, and near and far clipping distances.

Predefined Shapes

Those are the main types of the whole library. Beyond that, there are a bunch of predefined shape objects, including:

Box
Circle
Sphere
Cylinder
Cone
Pyramid
Torus
Grid Plane
String
Torus Knot

These are just shape objects with predefined sets of paths in them.

The box you’ve seen above. But here it is in motion.

Your browser does not support the video tag.

The circle creates a 2d circle that can be transformed in 3d. The sphere, cylinder, cone and torus are all created with a series of circles.

Your browser does not support the video tag. Your browser does not support the video tag. Your browser does not support the video tag. Your browser does not support the video tag. Your browser does not support the video tag.

The pyramid and grid plane, like the box, are drawn with longer straight lines.

Your browser does not support the video tag. Your browser does not support the video tag.

The string object creates vector text objects from a string of characters. I spent a lot of time on this and it was really fun. I’ll probably do a whole post just on that.

Your browser does not support the video tag.

Finally, the torus knot is a complex mathematical structure. It’s actually what started this whole project. I ran across the concept of a torus knot and coded it up. That gave me the base point and path types, and the 3d transform and rendering code. This was pretty easy to put together because I’ve been hand-coding 3d systems like this for well over 20 years. I’m not bragging. Just saying that I keep doing the same things over and over. OK, maybe a humble brag. Anyway, once I had the torus knot looking good, I just started adding other shapes to it.

Your browser does not support the video tag.

Render Quality

I’ll acknowledge that the quality of these animations is a bit aliased. This is just how things look straight out of the box. There’s a lot more to cover here, including some tricks I’ve been using to make these look a lot better.

I also want to cover rendering points directly, which doesn’t sound very exciting, but can actually be very cool. So stay tuned on a few additional posts on this library.

The Code

The library is up on github already at https://github.com/bit101/wire. As I said, it all depends on my other two personal libraries, so it’s unlikely you’ll wind up using it directly, but there might be some interesting techniques in there for you to peruse.

MRI

kp@bit-101.com (Keith Peters) — Fri, 03 May 2024 19:28:36 -0400

I went in for my MRI today to see what’s happening with my rotator cuff.

I had an MRI once before, quite a few years ago. I tend to have a claustrophobic reaction to small tight places like tunnels. Even the thought of crawling through a tunnel or watching a movie where that’s happening gives me a physical reaction. It’s something to do with the thought of not being able to raise my arms or turn around or go backwards. Just writing this, I’m shuddering. Anyway, last time I had an MRI they asked if I’d have a problem. I looked at the machine and was like, “Nah, it’s so short and I can see out both ends. No problem.” Then as I was moved into the tube, I started to panic. It took a lot of self control and measured breathing, but I got through it.

So in we go again today. Last time they offered a sedative. I would have taken it this time, but it wasn’t offered and I was embarrased to ask. My wife suggested shutting my eyes before I go in and keeping them shut. So I did that. They also offered headphones and asked what kind of music I wanted. I said classic rock. Yeah, a lot of it’s crap, but it’s my crap. I didn’t want to be in there listening to current music that I have zero connection or affinity with. So we got some Bad Company, Bon Jovi, Doors, Led Zeppelin, ZZ Top, and some others. Incidentally, the sounds the MRI was making during the ZZ Top tune matched the guitar track to a degree that was uncanny. Rhythm, tone, timbre. They actually started resonating with each other causing a fascinating audio effect. Although I’m not a huge ZZ Top fan, this was magical.

But I digress.

Shut my eyes and kept them shut, rolled in, listened to the music, rolled out twenty minutes later. No problem! I was very happy about that.

This evening I got the written report. I’ve got a partial tear on one side of my supraspinatus tendon with a 9mm retraction and another tear on the other side. Two other posisble tears elsewhere.

You know what that means? Me neither. At least it’s not a full width tear. But I’m not sure if these are small enough that they could potentially heal on their own, or if surgery will be necessary. I’ll talk to my doctor on Monday.

Header photo by MART PRODUCTION

I Broke My Rotation Matrix

kp@bit-101.com (Keith Peters) — Tue, 30 Apr 2024 21:48:26 -0400

The clickbait title actually refers to my shoulder / rotator cuff. And it’s not actually broken, though it might be torn.

Phase Zero

I’ve had pain in my right arm down under the biceps for over a year. I thought it was a pinched nerve and hoped it would go away on its own. I did stretching and some light strength training. It didn’t seem to make anything worse, but didn’t help either. Finally, last September I talked to my doctor about it.

Turns out it’s my rotator cuff. I didn’t even know what a rotator cuff was. In case you don’t, it’s the term applied to the four muscles that come in over the front, back and top of your shoulder and stabilize the top of your upper arm bone (humerus). They attach to the head of that bone via tendons. These can become inflamed, or frayed, torn, or totally detached.

Phase One

First order of business was try to physical therapy. I did that from the beginning of October 2023 through mid-December. I’ve had a couple of really amazing experiences with PT in the past. This one was not great. I was actually in more pain when I finally quit. I continued doing some PT on my own until I saw my doctor again in January.

Phase Two

Phase two was to go to an orthopedic specialist. I did that in early February. She was great. Answered a lot of questions and gave me a choice:

Get a cortisone shot and see how that plays out over the next couple of months. Cortisone reduces inflammation and can sometimes actually cure the condition if that’s all that’s going on.
Get an MRI to see what’s going on. If there’s a tear, then we could be looking at surgery.

I opted to get the shot and see what that did for me. It helped a lot! For a couple of weeks. Then, week by week, it reverted to how it was before, and worse.

Sleeping is really bad. Most nights by 3 or 4 am I am awakened with throbbing pain. Sometimes I can get in a better position and go back to sleep. Other times I give up, get out of bed and go do an early dog walk. Moving around helps and sometimes I can even get another couple hours of sleep after that.

There are a lot of motions I do inadvertently with my right (dominant) arm that cause extreme pain. I can compare it to stubbing your toe really badly - where you just have to stop everything and close your eyes until it subsides. Some days I’ll just have shooting pains down from my shoulder to my elbow, twitches, and sudden weakness to the point of almost dropping things. Doing day to day activities like getting dressed, washing my hair, driving, getting things off a shelf are becoming increasingly difficult.

Phase Three

The next action comes in a couple of days (Friday) when I go back and get an MRI. Given my steadily worsening symptoms, I’m self-diagnosing that I’ve probably torn something in there. But we’ll see when I talk to the doctor again on the following Monday.

I’ve been educating myself on rotator cuff surgery and while it’s a relatively quick process - generally under an hour as an outpatient procedure - the recovery can be brutal. But again, we’ll see if that’s my path or not within the next week.

By the way, the header image above is actually my shoulder. When I went to see the orthopedic doctor, they did an xray to rule out any issues with the bones of the shoulder. All good there, but they can’t see the muscles and tendons with an xray.

Aliexpress Impulse Buys

kp@bit-101.com (Keith Peters) — Sat, 27 Apr 2024 11:51:22 -0400

I was perusing Aliexpress a week or two ago and bought some cheap, fun things. Three purchases, which came to a total of around $13, with free shipping. This was part of some kind of buy three things for a reduced price offer. Here is what I got:

Mini Weather Cube

This is the top left. It’s delightful!

USB-C powered, wifi connected. There are zero interactive controls on this thing. You plug it in and it creates a wifi network. You connect another device to that network and from there configure it to connect to your actual network. Then you hope and pray and trust in your good karma that this thing doesn’t just exist to infiltrate you.

From there on, it’s just on your local network and it sets up a mini server with a page to configure all the device’s options. The config program is pretty extensive. There are a few different weather screens and a few different clocks, and an image slideshow with predefined or custom images. You can also set it to cycle through modes. It has day/night mode to dim it while you sleep.

I freaking love this thing.

Link: https://www.aliexpress.us/item/3256806253672298.html

Cheap Bluetooth Earbuds

I’m not sure why I chose these. They’re pretty bad. To be fair, they actually sound better than I expected. But I still don’t know which is right and left because they are unmarked. The tips they came with are tiny, but I have plenty of spare earbud tips around so I was able to swap them out. They paired and connected OK, but they don’t seem to auto-connect later, so you have to manually have to connect them each time. There is a tap to play/pause, but there’s no beep or feedback. I didn’t test other double/triple tap or hold gestures. They have bright LEDs on them that flash intermittently when they’re in use, which is annoying in a dark room. But I just figured it’d be good to have some emergency spares around, and for the price, I can’t complain too much.

Link: https://www.aliexpress.us/item/3256806172256904.html

Three-key + Knob Control Pad

This is pretty neat, but was a major pain in the butt to set up. It connects to your computer via USB-C and is theoretically configurable. We’ll get to that.

I’ve built some DIY devices similar to this:

This one with six media keys
And another one that’s just a single volume knob.

But this looked pretty neat and is a combo of keys plus knob.

I plugged it in and it did nothing! It came with zero instructions, but the product page on Aliexpress gave a URL to get config software. That URL 403ed. I couldn’t find any other link to the software. I almost resigned myself to the fact that this was going to be a little lucite brick.

But after a pretty extensive search, I found this one guy who created his own software for configuring these kinds of devices. What a hero!

https://github.com/rOzzy1987/MacroPad

It’s Windows only, so I booted into my Windows partition, ran the config program and set up the keys to do previous track, play/pause, and next track. The knob controls volume, and clicking on the knob mutes/unmutes. But really you can set up they controls to do just about anything you want. Once you send the config to the device, it’s all set and should work on any computer/OS. Booted back into Linux and it’s golden.

It’s nice since my Thinkpad does not have media keys. It has volume keys, but having the knob is really nice.

Link: https://www.aliexpress.us/item/3256806557559192.html

Anyway…

Just some fun stuff. I imagine those links won’t last forever, but there are plenty of other similar devices all over the site, and similar ones on Amazon and other places.

Another Paradox

kp@bit-101.com (Keith Peters) — Mon, 22 Apr 2024 12:49:56 -0400

Take the sentence:

You have not read this sentence.

It was true, up until the point you read it. Then it was false. Alternately…

You have read this sentence.

This was false, until you read it. Then it was true.

By reading the sentence, you changed its meaning. A Heisenberg sentence?

Oddly, the first sentence is still true for those who have not read it yet. But I think “yet” is important here. What about the billions of people who will never read that sentence? The “you” in the sentence refers to the reader, so if they never read it, they are not described by “you”, so the sentence has no meaning for them. I think it is neither true nor false.

Even more oddly, once someone does read it, that’s when the “you” takes on meaning. But at that point, the “you” refers to them even before they read it. So the fact of observing the sentence changes its meaning not only in the present and future but in its past as well. Weird.

Keith's Paradox

kp@bit-101.com (Keith Peters) — Mon, 22 Apr 2024 10:21:29 -0400

Definition

When I say “edit” here, I mean “to change from the original.”

The Paradox

If I write the following sentence:

This sentence has been edited.

It is false. I wrote the sentence and didn’t edit it. OK, so I fix the sentence to make it true:

This sentence has not been edited.

But now I’ve edited it. So it’s still false!

There are many of these self-referential statement paradoxes, like “This sentence is false.” I haven’t seen this one around, but wouldn’t be suprised if someone has used it before.

MacOS Home Directory

kp@bit-101.com (Keith Peters) — Sat, 13 Apr 2024 11:00:17 -0400

Background

A funny (not) thing happened the other day. My beloved Thinkpad slipped off the edge of a chair. It wasn’t a big fall and it fell to a carpeted floor so all seemed fine. But the next day I went to plug something into one of the USB-C ports and discovered it was just an empty hole. When the laptop fell, the power connector was in and it pushed the USB-C port in and seemingly broke it right off. Luckily there are two of them, so it all works fine and charges OK. But I plan on taking it in to see if I can get it repaired. I know it will be in for a few days, so I’m prepping my Mac Book Air to serve as a backup.

What I use the Thinkpad for is mostly my personal coding stuff as well as browsing, media consumption, etc. My biggest concern was having a functional coding setup for my creative coding projects and any other learning / exploration I want to do. I pretty much had everything set up earlier. I use MacPorts to install all my dev stuff and it all works mostly fine. But for one thing…

Why do they do this???

In all its infinite wisdom, Apple has chosen to ignore the standard Unix/Linux standard of using /home as the location of user’s home directories, instead opting for /Users. Think Different, I guess. In general, I’ve mostly gotten around this by using the ~ shortcut in scripts, or $HOME. This will go to /home/keith on Linux and /Users/keith on Mac.

But there’s one place I can’t do that. In my Go projects, I set up a workspace that links to local copies of my personal libraries. For this I need to give the go.work file a path to those libraries. Normally it’s like this:

/home/keith/Code/golang/libs/bitlib
/home/keith/Code/golang/libs/blcairo

Not the best setup in the world, but this lets me add new functions to my libraries on the fly as I’m developing a particular project. They’ll then be usable in other projects.

Sadly, you cannot use ~ or $HOME here. So every project I want to build on my Mac, I’d have to go in and change the path to /Users/... and then when I get my Thinkpad back, I’d have to go change them back. No thanks.

Warning

Some of the stuff I’m going to describe in the next two sections is potentially dangerous to your Mac’s configuration. That said, I’m not deleting anything, just commenting out one line in a file and adding one line in another file. Easily revertible. But do your own research and make up your own mind as to the safety of this.

Easy fix?

My brilliant mind said, “Just create a /home symlink that points to /Users.”

In the end, that’s what I did, and it works great! But there were a couple of hoops to jump through first.

First, oddly enough /home DOES exist on a Mac. It may be hidden. But it’s there and you can’t replace it. It points to /System/Volumes/Data/home. It’s probably empty. But if you try to delete it or unlink it, it won’t let you.

Apparently, this is part of a system designed to auto-mount network volumes. The line that sets it up is in /etc/auto_master. It’s this one:

/home			auto_home	-nobrowse,hidefromfinder

When your Mac boots, it will auto-mount /System/Volumes/Data/home to /home. You can disable this by commenting out this line (you’ll have to do this with sudo of course):

#/home			auto_home	-nobrowse,hidefromfinder

To be perfectly honest with you, I don’t know for certain what side effects this might have. But from everything I’ve read, if you’re not mounting network volumes, you should be OK. I’m not and so far, so good. Worst case, uncomment the line and you’re back to normal.

Reboot your Mac and that /home symlink will be gone.

OK, now it’s an easy fix, right? … right???

Yeah, so now we just do ln -s /Users /home right? Simple.

Nah. It’s gonna tell you you can’t do this on a read-only file system. OK. Workaround, please?

Yes, one workaround coming right up!

You create root symlinks using a special file, /etc/synthetic.conf. Of course! Why not? That file probably won’t exist, unless you’ve done this before, so just create it (using sudo) and add this line:

home	Users

Super important guidelines for this:

No leading / character. It’s assumed.
The whitespace in between home and Users has to be a single TAB character. Not a space or multiple spaces or any other whitespace characters or anything else.

If you mess this up, you may wind up in a boot loop, at which point you’ll have to do some kind of recovery mode nonsense. So look this up and have the instructions on hand in case you need to do that. I was lucky enough to not mess it up the first time.

Success

Everything works fine! All my projects that link to /home/keith/... now work on the Mac. And these projects will continue to work on Linux as well, with no changes needed for either system. I’m happy.

But again, do this at your own risk and have a revert and recovery plan before you start messing with this stuff. I’m using this for a very niche use case. There may be a better solution for this, but I couldn’t figure one out. In 99.99% of other cases, I’d recommend using $HOME or ~ to point to your home directory.

That all said, it’s pretty cool to be able to do this!

UPDATE!

While the above worked great, it still felt pretty hacky. After much thought, I came up with another way.

Rather than relying on the actual path to my libraries, I created a symlink in the root directory that pointed to the library folder.

# On Mac via synthetic.conf:
/bitlib -> /Users/keith/Code/golang/libs

# On Linux via ln -s:
/bitlib -> /home/keith/Code/golang/libs

Now in my go.work file I can point to:

/bitlib/bitlib
/bitlib/blcairo

This is way more portable and only requires a one-time set up on any given machine. I reverted the changes I made above and I’m going with this solution for now. I’ll leave the post up in case it helps someone with some future problem though.

Subpixel Sampling

kp@bit-101.com (Keith Peters) — Tue, 09 Apr 2024 16:18:52 -0400

The Set Up

In my last post on Graphing Equations I showed how I graph an equation by visiting every pixel in the canvas, seeing if it satisfies that equation (or comes close to solving it) and then coloring that pixel appropriately. In that case, I was just coloring each pixel either black or white. In general it looked great, but you may have noticed some aliasing going on. The dreaded stair steps. Today we’ll look at how subpixel sampling can help solve that.

Recreating the problem.

I’m going to start with some code and the image it creates. It’s a similar kind of thing to what I created in the earlier post. I just experimented with cosh in place of some of the cos instances and tweaked some other values. If you read the previous post, you should be able to work this out.

for (x = 0; x < width; x++) {
  for (y = 0; y < height; y++) {
    xx = map(x, 0, width, -4, 4)
    yy = map(y, 0, height, -4, 4)

    a = sin(sin(xx) + cosh(yy))
    b = cos(sin(xx * yy) + cosh(xx))
    d = abs(a - b)

    if (d < 0.125) {
      fillRect(x, y, 1, 1)
    }
  }
}

Anyway, to recap, we’re hitting each pixel, mapping it to a -14 to 14 range, running two expressions and seeing if the results of those two are equal (or close to being equal - less than 0.125 close). If so, we fill that pixel with the default color, which will be black. Otherwise, it remains white.

This is what we get:

It looks … OK. But let’s zoom in a bit…

Yeah, not very OK. The problem is we’re getting the value of each pixel and setting it to black or white. So it’s all very hard, rectangular images that don’t look great.

We could apply a blur…

That smooths it out a bit, but blurs everything and doesn’t really solve the problem. What we want is to have more shades of gray around the edges. One way to do this would be to render it at a higher resolution and then scale it down. The scaling algorithm then blurs those pixels on the edge just right usually. I rendered this at 3200x3200 and then scaled it back down to 800x800, and it looks pretty good:

But then we’re rendering 10,240,000 pixels (3200x3200) to get 640,000 (800x800) and having to open up an image processing program or call out to ImageMagick or something. It would be nice to be able to figure this out in code alone.

A Solution: Subpixel Sampling

The basic concept of subpixel sampling is that for every pixel you are rendering, you sample it multiple times, as a small offset, but still within the bounds of that pixel. Then you take the average of all these samples and use that as the color for that pixel. It’s kind of the same as rendering the image larger and scaling it down, but you’re doing it all in one go, for each pixel.

There are a few different strategies for this.

One is to randomly sample various points within the pixel.
Another is to sample a grid of subpixel locations.
The most complicated is to recursively sample smaller and smaller grids until the values converge for each section, then merge them back up to the top to get the final pixel value.

The final strategy is sometimes used in ray tracing, but it would be overkill here. We’ll examine the first two, though.

Random Sampling

This is pretty simple. We choose a number of samples and randomly generate a bunch of floating point pixel values between the current and next pixel. For example, if we’re looking at pixel 300, 400, we’d get say 20 different values for x, from 300.0 to 301.0. And from 400.0 to 401.0 on y.

Here’s how that might look:

The pixel we’re sampling lies on the edge of a curved shape. Without antialiasing, we’d just sample at exactly 300, 400, and color that whole pixel white. Here, we’ve sampled twenty subpixel locations. About half of them are on the shape and half are off, so this pixel value would get somewhere around mid level gray rather than just plain black or white. The pixel one down and to the left only overlaps the shape a little bit, so would likely get fewer hits of black and would be a lighter shade of gray. The top left pixel would be all white and any of the pixels entirely in the shape would be entirely black.

So let’s code that.

for (x = 0; x < width; x++) {
  for (y = 0; y < height; y++) {
    value = 0
    for (s = 0; s < 20; s++) {
      x1 = x + randomFloat()
      y1 = y + randomFloat()

      xx = map(x1, 0, width, -4, 4)
      yy = map(y1, 0, height, -4, 4)
  
      a = sin(sin(xx) + cosh(yy))
      b = cos(sin(xx * yy) + cosh(xx))
      d = abs(a - b)

      if (d < 0.125) {
        value += 1
      }
    }

    // theoretical color function. 0 is black, 1 is white.
    setGray(1 - value/20)
    fillRect(x, y, 1, 1)
  }
}

We start with a value for the pixel set to 0. Then we loop through 20 times.

Each iteration, we add a random floating point value from 0.0 to 1.0 to x, storing it in x1. And the same for y to y1.

Then we do our previous calculation with x1, y1. But if it’s within range, rather than coloring the pixel then and there, we just add 1 to value. When we’re done with our sampling, we divide by the number of samples (20) and use that as the pixel value. I have a theoretical function setGray that will set the drawing color to black if it gets 0 and white if it gets 1. Because we started at a value of 0 and added up, we’ll need to reverse that by subtracting value from 1. Then we fill that pixel. Here’s the result:

That looks pretty good. But not quite as good as the scaled up and scaled down version I think. And how many samples did we take? 800 x 800 x 20 = 12,800,000.

And look what happens when we reduce the number of samples to 10 and zoom in:

Because of the small sample size, sometimes the samples clump in a white section of the pixel and sometimes in the black section. So you can get ragged edges. For random sampling to work well you really need to increase the sample size. I think it also may work better on more colorful images. We’re really challenging ourselves here with black and white.

But let’s change directions now and try method two, a subpixel grid.

Subpixel Grid

The idea here is rather than a random sampling, we purposely spread the sampled subpixels evenly across the pixel. Here’s a few examples.

Each square represents a single pixel. In the first one, we sample nine subpixels, sixteen in the second one and twenty-five in the third one. Of course, the more subpixels you sample, the slower your rendering is going to be. You’ll have to figure out the balance of speed and quality that works well for your purposes.

The code for this is fairly easy, not too much different from the random version. We just use a couple of for loops.

Le code:

sampleSize = 3
spacing = 1 / (sampleSize - 1)

for (x = 0; x < width; x++) {
  for (y = 0; y < height; y++) {
    value = 0

    for (i = 0; i < sampleSize; i++) {
      for (j = 0; j < sampleSize; j++) {
        x1 = x + i * spacing
        y1 = y + j * spacing

        xx = map(x1, 0, width, -4, 4)
        yy = map(y1, 0, height, -4, 4)
  
        a = sin(sin(xx) + cosh(yy))
        b = cos(sin(xx * yy) + cosh(xx))
        d = abs(a - b)

        if (d < 0.125) {
          value += 1
        }
      }
    }

    // theoretical color function. 0 is black, 1 is white.
    setGray(1 - value/(sampleSize * sampleSize)
    fillRect(x, y, 1, 1)
  }
}

We create a sampleSize variable that holds the number of samples we’ll take on each axis. The total number of samples will be the square of this number.

The spacing between each sample will be 1 (pixel) divided by one less than the sample size. So for a sample size of 3 the spacing would be 0.5, which fits in with the first square in the image above.

The i, j for loops get each sample. It’s just recreating what you see in the grids above. And the rest you’ve already seen. In the end we divide value by the square of sampleSize. So for a sample size of 3, we divide by 9.

Here’s how it looks with a sample size of 3. That’s 9 samples per pixel.

And a sample size of 4, 16 samples per pixel.

This last one winds up samping the same number of pixels as the 4x image scaled down: 10,240,000. If I’m honest, the manually image-processed one looks a little better. But I’m pretty happy with this. If I’m generating a ton of frames for an animation, I’m not going to manually process each one. And in that case, the sample size 3 one would probably be just fine.

I find that increasing the sample size beyond 4 has very diminished returns.

Just to see how far we’ve come, here’s a combo image. On the left, subpixel sampling with a sample size of 4. On the right, there’s no antialiasing at all.

Summary

I’m no expert in this field, but this stuff was pretty easy to work out and gives decent results. I’m not sure if sampling edge-to-edge like I’m doing gives the best results. Maybe instead of going to both edges like in the first square below…

…it might be better to avoid one edge like in the second square. Or avoid all the edges and sample completely inside the pixel like in the third square. I’m sure smarter people than me have worked on this problem and have come up with better answers than me. But now you have something to look into maybe.

You’ll probably find the most resources on this stuff in ray tracing books and articles. That’s where I first ran across practical applications of the concept when I built my own ray tracer a year and a half ago.

Graphing Equations

kp@bit-101.com (Keith Peters) — Sun, 07 Apr 2024 12:26:57 -0400

Set up

This post is about programming strategies for graphing various equations and functions. I often see some interesting graph and an accompanying equation or functions and I want to recreate it and play with it a bit and see what else I can do with it. The thing is, there are a few different ways these things can be presented. So I’ll go through some of these and we can explore some strategies.

Functions

Functions are pretty easy in general. If you think back to your school days, a mathematical function looks something like this: y = f(x), where f(x) = x + 3 for example. Pretty similar to programming functions and you can usually map between the two pretty easily. An important thing to remember about functions is that for any given x, there will always be exactly one y.

So say we had f(x) = sin x. Easy. We create a function in code like so (in somewhat JavaScript-y pseudocode):

function firstFunc(x) {
  return sin(x)
}

Now we can run through a bunch of x values, say from 0 to the width of our canvas, and get y values for each of them and draw segments. Maybe something like:

for (x = 0; x < width; x++) {
  y = firstFunc(x)
  lineTo(x, y)
}
stroke()

Now you just see some blips on the top edge of the canvas. So we need to do a few things to adjust the range of the output graph. Move it down a bit, increase the amplitude and wavelength. You can do this by translating and scaling but that can have resolution side effects, so we’ll do it with math.

for (x = 0; x < width; x++) {
  y = firstFunc(x * 0.1) * 40 + 50
  lineTo(x, y)
}
stroke()

I multiply x by 0.1 to increase the wavelength. Multiply the result of firstFunc by 40 to increase the amplitude, and add 50 to that to move it down to the center of the image.

Notice that for each point on the x-axis, there is exactly one point it matches on the y-axis. So functions are pretty easy. But what if you have something like a circle?

For some x values there is no corresponding y value. And for some there are two y values. So this is not a function and we need to do something else.

Parametric Equations

Parametric equations take a parameter (or multiple parameters). For two-dimensional graphs, there’s often two functions based on the same parameter. One function returns the x coordinate of a point, and the other returns the y coordinate. In this case, I’m using “function” in the less strict, or programming sense.

It’s common in trigonometric parametric equations to have the parameter be called “theta” and is often defined by the letter t. In these cases, theta is an angle that can range from 0 to 2 PI. The parametric equation(s) for a circle might then look like:

function circle(t) {
  x = cos(t)
  y = sin(t)
  return x, y
}

My theoretical pseudocode language allows returning two values from a function. If your real world language doesn’t, you might have to return some kind of point object.

Now you can draw a circle by passing in a series of t values from 0 to 2 PI

for (t = 0; t <= 2 * PI; t += 0.01) {
  x, y = circle(t)
  lineTo(x * radius, y * radius)
}
stroke()

Since this circle function will just return values from -1 to 1, I multiplied them by a radius value. You could also offset the circle on the canvas as well. You might want to make radius and center point additional parameters to the function.

Note that the amount you increase t by (0.01 in this case) will affect the resolution of the circle. Increase it by too much on each iteration and you’ll get something more like a polygon. Increase it by too little and you’ll get a super smooth circle, but you might be doing way more work than you need. The trick is finding the balance.

For a more interesting parametric equation, we can turn to the rose curve.

The equation for a rose curve is r = a * cos(n * t). This equation takes three parameters:

a is the amplitude - kind of the max radius of the rose.
n is the number of nodes in the curve.
t as usual is theta.

The result of this equation is not an x, y point just yet, but a radius corresponding to a particular theta angle. We can turn that into a function that returns x, y by using cosine and sine on t, multiplying the result by r. It will look like this:

funcion rose(a, n, t) {
  r = a * cos(n * t)
  return cos(t) * r, sin(t) * r
}

And we can use it like so:

for (t = 0; t <= 2 * PI; t += 0.01) {
  x, y = rose(150, 5, t)
  lineTo(200 + x, 200 + y)
}
stroke()

In this case, since the returned point has a useful radius, I just translate it by 200 on both axes to center the shape. Here’s the result:

And if I increase n to 17, we get that many nodes:

This gets even more interesting if you make n a rational fraction, like 11/13. In this case however, it will take more cycles to get all the way back to the starting point. 13 * PI does the trick.

for (t = 0; t <= 13 * PI; t += 0.01) {
  x, y = rose(150, 11.0/13.0, t)
  lineTo(200 + x, 200 + y)
}
stroke()

There are a lot of parametric equations that return a radius. Now you know how to handle them.

More Complicated Equations

Often you’ll come across an equation that isn’t in parametric form. For example, the standard form of the circle equation is:

(x - h)^2 + (y -k)^2 = r^2

Or, in code…

pow(x - h, 2) + pow(y - k, 2) == pow(r, 2)

Any set of x, y, h, k values that satisfies that equality is part of the circle. And we can rewrite this by taking the square root of each side, like so:

sqrt(pow(x - h, 2) + pow(y - k, 2)) == r

Now normally, for a circle, we wouldn’t even bother with this form, because we already know the parametric form. But let’s give it a shot as a simple use case for how to deal with things that are represented this way.

One strategy is to look at every point on the canvas and see if it satisfies that equation. So we loop through x from 0 to the canvas width and y from 0 to its height and plug those numbers into the formula. If the conditional is true, we color that pixel. Something like this:

h = 200
k = 200
r = 150
for (x = 0; x < width; x++) {
  for (y = 0.0; y < height; y++) {
    if sqrt(pow(x - h, 2) + pow(y - k, 2)) == r {
      fillRect(x, y, 1, 1)
    }
  }
}

Here, h and k define the center of the circle and r is its radius.

And that gives us…

…almost nothing. You can see a few points on the edge of the circle, but not many. The reason for this is that we’re being way to strict. We’re rarely going to find an x, y point that is exactly on the circle. if it’s even 0.00000001 of a pixel off, that’s a false. So we need to build in wiggle room. If it’s close enough to equal that’s ok.

So we say that square root value is the distance from the center of the circle. If that’s exactly equal to the radius, then the difference is 0 and we want to plot that point. To give us that wiggle room, we can say if the difference between the distance and the radius is less than a small amount, that’s close enough. We’ll subtract the two and we’ll take the absolute value to get the actual un-signed distance.

h = 200
k = 200
r = 150
for (x = 0; x < width; x++) {
  for (y = 0.0; y < height; y++) {
    dist = sqrt(pow(x - h, 2) + pow(y - k, 2))
    if abs(dist - r) < 0.5 {
      fillRect(x, y, 1, 1)
    }
  }
}

Here I checked to see if that difference is less than 0.5, which gives us this:

Of course if we increase that value, we can get a circle with a much wider boundary. Here I’m comparing to see if the distance is less than 20.

OK, now I said normally we wouldn’t use this method for something as simple as a circle where we already have a better method. But what if you have an equation like this?

sin(sin(x) + cos(y)) = cos(sin(x * y) + cos(x))

I’m not sure there even is a parametric form of something like this, or how I’d go about finding it. But we can pretty easily plot this the same way we just did for the circle.

There will be a few steps to go through though. First we want to map our x and y canvas pixel values to a much smaller range. -14 to 14 will work well. I’m going to use the map function I described in this post:

FOTD: Map

I’ll assign the mapped x, y values to xx and yy

for (x = 0; x < width; x++) {
  for (y = 0.0; y < height; y++) {
    xx = map(x, 0, width, -14, 14)
    yy = map(y, 0, height, -14, 14)
  }
}

Now I’ll put those values into both sides of that equation. I’ll do this separately, calling the first one a and the second one b. Then I’ll see if they match to within a certain distance, and if so, plot that point.

for (x = 0; x < width; x++) {
  for (y = 0.0; y < height; y++) {
    xx = map(x, 0, width, -14, 14)
    yy = map(y, 0, height, -14, 14)

    a = sin(sin(xx) + cos(yy))
    b = cos(sin(xx * yy) + cos(xx))

    if abs(a-b) < 0.25 {
      fillRect(x, y, 1, 1)
    }
  }
}

You can try setting different values in the map function to zoom in and out, and different values to compare the distance to. Here I used 0.25. Smaller values will make thinner lines and shapes. Higher values will give a more blobby look.

Then start experimenting with the equation itself. And find other equations to graph. And animate them. And…

March Update

kp@bit-101.com (Keith Peters) — Sun, 10 Mar 2024 14:29:03 -0400

Oof! It’s been a while. Why the silence?

Portland

A few days after my last post, I headed out to Portland, OR for several days. A couple of my older kids have wound up migrating out that way and not too long ago, my daughter had a baby boy, making me a grandfather! I’d love to show you pictures of him. He’s a very cute, very smiley kid. But I’ll let him discover the consequences of social media when he’s old enough to do that on his own. I had a very good time out there, getting to know the city (I’ve only passed through one time before) and catching up with family. It was also just great to get a good chunk of days off work, which has been pretty stressful lately.

Sick

I caught a red-eye on the way back, took the next day off and headed into the office on the following day. In the afternoon, my stomach started feeling not-great. When I got home, I was pretty damn nauseous. By early evening, I was sicker than I’ve been in many years. Feverish and delirious, purging everything in my system in every conceivable way. I could not even tolerate a single sip of water and wound up super dehydrated. Late in the evening, my wife dragged me to the ER where they put me on an IV to get fluids into me. Spent the night there. They did a CT scan but all was fine. In the end it was just a norovirus. “Just”, sure. The next few days were better but it took almost a week before I could eat three real meals. Mainly just quarantined up in my room. But I’m back to normal now.

Graphs

During my trip and since then, I was pretty obsessed with graph theory. I’ve been digging through books and other resources on graphs, algorithms, data structures. It’s been really good. I’ve been implementing some graph data structures and learning a lot. So many cool algorithms that you can do with graphs. I’ll be doing more in the coming weeks.

Hashes

Not entirely related, but while reading up on algorithms and data structures, I ran across something that described how hashmaps/hashtables/dictionaries/etc. are implemented. This blew my mind in a few ways.

For one, I always had in the back of my mind that hashmaps and hashtables and dictionaries and other similar data structures were very different things. But for the most part they are the same thing. Some languages give them slightly different different features, like one will be read-only while one is dynamic, one might only take strings as keys while another one can take other values, etc. But as far as I can tell, in general, across languages, they all refer to the same basic concept: you store a value using a key.

But as for how they were implemented, I was like, “What? Implemented? They just exist, like native arrays exist as sequences of memory addresses.” I never really thought how they might exist, or why they had “hash” in their name. The material I was reading just gave a general description, but it amazed me. They are not native data structures at all. They’re pretty complex implementations that sit on top of arrays.

You might know more about this than me, but for the uninitiated…

First you create an array with a given number of elements. Let’s say 100 for sake of argument. When you add something to the hashmap, you give it a key and a value. You run that key through some kind of hash function that’s going to return an integer. You can then mod that integer by the length of the array (100 in our case) so you have a number from 0 to 99. And you put the item in that element in the array. Now you’re saying, “but I’ll wind up with the same index now and then, won’t I?” Yes! You absolutely will! That’s known as a collision. So rather than just having an array of individual items, it has to be an array of collections - an array of arrays, or maybe an array of linked lists. You hash the key to get the index of the main array, then add it to the collection in that array element, along with the key itself. Or, if another element with that key is already in that sub-collection, you update its value. To get that value back, you hash the key, find the index, and iterate through the sub-collection in that element to find the actual key you’re looking for.

Iterating through a long list of sub-elements takes you from the ideal of O(1) - where every item has its own element - towards O(n) - where the time is relative to the length of the sub-collections. So when your array starts to get full, you want to resize it and redistribute the elements to make those sub-collections smaller.

Now there are lots of variables here. What’s the best hash function? How many elements to start with? What data structures should you use for the sub-lists? When should you resize the array? How should you resize the array, and how big?

Of course, every language has its own implementation of this kind of data structure, but I just had to create my own implementation in Go. It was pretty straightforward and a lot of fun to do. I created my own hash function and used linked lists as sub-collections. I ignored resizing because this is something I’ll never actually use anyway in real life. But it was a fantastic exercise and maybe I will be able to use the concepts for something or other on down the line.

Other algorithm stuff

I also made a Go implementation of binary search and quick sort. These are things I was already familiar with, but they’re things you rarely implement in real life, so it was fun to do them again. It wound up being a good exercise to get a better handle on Go generics, which are relatively new and something I haven’t done a lot with. I wound up making a generic queue, set and stack and adding them to my toolkit for re-use in future projects. Also working on generic linked list and graph implementations in Go.

Wrap it up

OK, that’s what’s up with me. I’ll post another “Read/Watched” post soon with a lot of the stuff I’ve been reading recently. And I’ve been noodling on some other things as a pure distraction that I want to post about as well. See you soon.

Header image: Mount Hood from the Pittock Mansion land in Portland, OR, by me.

Read/Watched: 2/12/24 - 2/18/24

kp@bit-101.com (Keith Peters) — Mon, 19 Feb 2024 05:00:00 -0500

https://tannerhelland.com/2012/12/28/dithering-eleven-algorithms-source-code.html
- Dithering algorithms. I implemented five of these. I wrote my own code for doing the actual work on this stuff and figured out the first couple with the help of Wikipedia. This page was great for grabbing a few more.
https://www.karlsims.com/rd.html
- Reaction-Diffusion simulation. Finally figured this one out. It took more than just this reference, but most of what I needed was here.
https://mitpress.mit.edu/9780262539029/algorithms/
- Book on algorithms. Not really hands on coding, but very readable. Got it Saturday and tore through a quarter of it pretty quickly.
https://clig.dev/
- Command Line Interface Guidelines. I like making command line programs. There is useful info here. An early excerpt I found very interesting:
Traditionally, UNIX commands were written under the assumption they were going to be used primarily by other programs. They had more in common with functions in a programming language than with graphical applications.

Today, even though many CLI programs are used primarily (or even exclusively) by humans, a lot of their interaction design still carries the baggage of the past. It’s time to shed some of this baggage: if a command is going to be used primarily by humans, it should be designed for humans first.

Image Processing 01: Greyscale

kp@bit-101.com (Keith Peters) — Sun, 18 Feb 2024 08:11:15 -0500

I’ve been creating a bunch of image processing functions for my blcairo graphics library. It’s been a lot of fun and most of them have been surprisingly easy. So I thought I’d write about some of them. This all started when I when I wanted to try some dithering algorithms. Before applying the dithering, I wanted to convert the images to greyscale, so I put together a function for that.

Grey vs Gray

This has always bothered me. In the back of my mind, I always thought there was some rule about the spelling of this word. And I was the only one who didn’t know the rule. It turns out that either one is correct. If you’re British, you’ll probably write about the colour grey. If you’re in the US, it’s the color gray. In my codebase, I use both versions all over the place. So in this article, I’ll give a shout out to my UK friends doing maths on the lifts on the way to the loo before heading back to their flats. Grey it is.

Actually, per usage stats I’ve seen, it seems that grayscale wins handily over gray-scale, gray scale, greyscale, grey-scale, or grey scale. But I’m sticking to my greyscale guns.

Greyscale conversion

Here, we’re going to loop through every pixel in an image and convert it to a shade of grey based on the three RGB color channels.

I’m going to assume that you can get an array of bytes that represents your image. HTML’s canvas can do this, as well as anything based on Cairographics. This array will contain whole numbers in the range of 0-255. There will probably be four elements for each pixel - red, green, blue and alpha. They might be arranged that way - RGBA. That’s how HTML’s canvas works. But they might be in BGRA order which is how Cairographics works. Another system might do something else like ARGB. You’ll have to do the research for your platform. In this case, all we really need to know is which one represents the alpha channel because we’re not going to mess with that one, only the red, green and blue channels. We want to average them out somehow and set all of them to be equal. So say we have an array of bytes called data, we can loop through the values like so:

for (i := 0; i < data.length; i += 4) {
    // data[i] is one color channel. Probably red or blue.
    // data[i+1] is one color channel. Probably green.
    // data[i+2] is one color channel. Probably blue or red.
    // data[i+3] is probably the alpha channel
}

Let’s assume the last one is alpha, so we just need to deal with the first three. We want to set them all to the same value. There are a few strategies here. We could just choose one channel and set the others to be equal to it, like so:

for (i := 0; i < data.length; i += 4) {
    data[i+1] = data[i]
    data[i+2] = data[i]
}
// you'll now have to put the updated data array back into the image to have it reflect the changes.

Assuming the first channel is red, this sets the green and blue channels to that value. This will work, but you’ll get very different results for different images. An image with a lot of reds in it will appear very light, like so:

Source:

Photo by Bruno Scramgnon

Red channel copied to green and blue:

However an image with less red will wind up very dark.

Source:

Photo by veeterzy

Red channel copied to green and blue:

Another option would just be to average the three channels. Add them up and divide by three.

for (i := 0; i < data.length; i += 4) {
    grey = (data[i] + data[i+1] + data[i+2]) / 3
    data[i] = grey
    data[i+1] = grey
    data[i+2] = grey
}

The above code is very simplified. If your data array really is made of the data type byte, adding three bytes together will overflow the 256-value limit and give you some strange results. In that case, you’ll need to convert to a larger data type such as int or float, do the math, then convert back to byte. Or, separately divide each one by three and then add the results together.

This gives us:

This is better, but kind of muddy. It has to do with the way the human eye (and brain) perceives different color frequencies. We see green as being much brighter than blue, with red somewhere in between. So just averaging the three channels doesn’t give us the results we want. We have to take those human perception biases and account for them somehow. Luckily, a lot of research has been done on this and we have values for each channel that represent our way of seeing things. Of course, this isn’t an exact science since everyone’s eyes and brains are slightly different, but here are some numbers that are pretty well accepted and give decent results:

Multiply the red channel by 0.299
Multiply the green channel by 0.587
Multiply the blue channel by 0.113

Add these together and you’ll get a decent greyscale value. You might find other values listed in other articles. Feel free to try them out and use whatever you think looks best. Here’s some pseudocode for this:

for (i := 0; i < data.length; i += 4) {
    grey = data[i] * 0.299 + data[i+1] * 0.587 + data[i+2] * 0.113
    data[i] = grey
    data[i+1] = grey
    data[i+2] = grey
}

This assumes your bytes are in RGBA order. Also, this is again simplified in regards to data types. You’ll probably need to do some conversion to multiply the bytes by floats and then convert back to reset the data elements. I know you can figure that out.

Now we get:

Here’s a montage of each of the results to get a better view:

You might not see a whole lot of difference in the final two images. In fact, you might even like the straight average better than the perceptual average. But bear in mind, both of these images are very biased towards a single color. Let’s try an image with more varied color.

Photo by Gülşah Aydoğan

I won’t tell you what to think, but for me, the last image - perceptual greyscale - has more contrast and better details and reflects the feel of the original better.

Actually, here’s a video of the two images toggling back and forth. The slightly brighter one is the perceptual average version.

Your browser does not support the video tag.

Now you can clearly see that the center of the flowers “pop” a lot more, and the green background is lighter with more contrast.

Of course, simple greyscale conversion is only the beginning. Later we’ll look at doing gamma correction, adjusting brightness and changing contrast, which can all have an effect on how the final greyscale image looks.

Header photo by João Cabral

Super Triangle

kp@bit-101.com (Keith Peters) — Sun, 11 Feb 2024 12:22:58 -0500

Delaunay Triangulation

Recently I was working on implementing Delaunay triangulation using the Bowyer-Watson algorithm. I used the exact steps outlined in that link and, while fairly complex, it worked perfectly. There was just one point that was a bit more vague than the others.

add super-triangle to triangulation // must be large enough to completely contain all the points in pointList

Making the super Triangle

To visualize what we need, here’s an idea of what I had in front of me. I have a bunch of random points (actually created through Poisson disk sampling). These were distributed across the full canvas like the rectangle shown here:

For the first step of the formula, I needed a triangle that completely encompassed all of these points. To get things working at first, I just made a really huge triangle. But “really huge” wasn’t very mathematical. I tried creating increasingly larger triangles until all points tested as inside the triangle, but that was very brute-force and inelegant. I saw one method where you calculate the convex hull of all the points and iteratively use the points of that to create triangles and check the area of each triangle until you have the one with the smallest area. Neat, but WAY overkill, imo.

I had an idea in my head that if I could take the two left points of the rectangle and extend them up and down some amount, and then take the right midpoint of the rectangle and extend that more to the right some amount, I should wind up with a triangle that fit the rectangle.

My first shot at was to extend the left points up and down one screen height, and extend the right midpoint to the right one-half screen width. Using this formula:

triangle(
    x, y - h,
    x, y + h * 2,
    x + w * 1.5, y + h * 0.5,
)

Here, x and y are the top left corner of the rectangle (often 0, 0), and w and h are the width and height of the rectangle (generally the canvas size). I gave that a go and … lo and behold, it worked perfectly!

I was sure I just got lucky with my measurements, so I tried changing the size of the rectangle…

And it kept working perfectly. I love it when that happens.

Now, I’m not saying that this is the only triangle you can use, or even that it’s the best one, or even the one with the smallest area or … really anything other than:

It works.
It’s super easy to calculate.

I had one tiny concern though. There’s a chance that one of the points could end up at exactly x = 0.0 or close enough to potentially cause some kind of problem. I’m not entirely sure it would cause a problem, but I wanted to play it safe and give myself a little leeway just in case. So I made the triangle just a little bit larger be subtracting a bit off the x values on the left, and adding a bit on the right. Like this:

triangle(
    x - w * 0.1, y - h,
    x - w * 0.1, y + h * 2,
    x + w * 1.7, y + h * 0.5,
)

Here’s the result:

Now there’s no doubt at all that every point is safely inside the triangle.

In my library I made a function that iterates over the point list finding the min and max x and y values and uses that to create the triangle. It’s a bit more complex but it saves me from having to manually pass in the values to create the triangle. I just pass in the point list and all is done for me.

By the way, this is the result of Delaunay triangulation, which I’ll probably write something about at some point.

Age

kp@bit-101.com (Keith Peters) — Sun, 11 Feb 2024 09:35:08 -0500

I remember the year I turned 29. I kind of freaked out because that meant I was only a year away from being “old”. But when I turned 30 I was like, meh, no big deal.

The same thing has happened every decade since. Last year I hit 59. Oh my god, I’m going to be 60 soon. Four months have gone by and as usual I’m starting to get used to the idea and even embracing it. Sixty. Hell yeah!

When I was young and went to my grandparents house every weekend, they were in their 60s and I thought they were OLD. I do not feel as old as I thought they were at the time. The man in the mirror looks pretty good for his age. But sometimes I see a picture of myself and I’m shocked to see my grandfather there instead of me. Funny how photos and mirrors seem so different. Some kind of psychology going on there.

Physically, I’m doing pretty damn well. I’m actively running. My weight is stably in the best place it’s been ever. BMI in the right place and all that. Just had my annual physical and the blood pressure and all the blood chemicals are just where they should be. The usual complaints here and there, but if I’m honest, no worse than any other decade. I certainly feel FAR better than 16-17 years ago, when I was 60 pounds heavier than I am now.

Mentally I think I’m pretty good. I find my memory isn’t the photographic wunderkind it used to be. And I struggle to find the word I’m looking for more than I think I used to. But that’s all normal stuff.

But, more and more I have to face the fact that I’m not immortal. Maybe I have another 30 something years left. More likely 20 something. That seems startlingly few. But I was way more obsessed with my mortality and how long I had left when I was in my 30s and 40s than I am now. More than being afraid of dying I am sad that I will eventually have to stop experiencing life. Because life is an amazing gift and even thought it’s not always perfect I love being part of it. This sentiment is expressed succinctly by Hemingway in For Whom the Bell Tolls in an untitled chapter often referred to as “Sordo’s Last Stand”. Sordo is a guerrilla fighter in the Spanish Civil War. He and his group are under attack and surrounded on a hill. He knows there’s no way out. He will die on this hill.

Dying was nothing and he had no picture of it nor fear of it in his mind. But living was a field of grain blowing in the wind on the side of a hill. Living was a hawk in the sky. Living was an earthen jar of water in the dust of the threshing with the grain flailed out and the chaff blowing. Living was a horse between your legs and a carbine under one leg and a hill and a valley and a stream with trees along it and the far side of the valley and the hills beyond.

I’m not religious, so I don’t believe in heaven or hell. As to what happens after, I have no idea. But it’s nothing I worry about. If anything, I’m just jealous of all those who will go on experiencing life after me.

Anyway, this post wasn’t supposed to be morbid. More of a memento mori. Knowing that I have limited time here makes me appreciate what time I have left even more. And I’m trying to make that time mean something. Sometimes I think about wanting to leave some kind of legacy. That’s hard in technology where everything changes so fast. I’ve written a bunch of books, but most of them are out of print and based on a language and platform that no longer exists. That’s why I try to write more in pseudocode these days. The concepts and algorithms will live a lot longer than the syntax. In any case, I know that my writings and videos and open source code and courses and conference talks have helped many people. It still brings me a lot of joy when someone reaches out these days and say that my work helped them started in tech, or helped them get through school or in their first job. And I hope that some will pay it forward and keep it going.

There’s a lot of messed up things in this world and a lot of bad things to protest and fight against. Sometimes I fear that we’ll all be so focused on fighting the bad that we’ll forget to do good. It’s not the same thing. Going around punching Nazis might be necessary, but it doesn’t - by itself - make the world a better place. Someone has to deliberately pump some positivity into the world as well. Weeding your garden doesn’t do any good if you forget to actually plant some flowers or vegetables too.

So I continue to try and learn and share what I learn in a way that hopefully lets the next person learn that thing with a bit less effort than it took me to learn it. In fact, now that I just wrote that sentence, I realize that’s kind of been my mission statement for the past 25 years. So I’m going to keep on doing that as much as I can for as long as I can and hope that one or several people not only learn something but get inspired to do the same.

So we’ll end on a bright spot!

Photo by Hatice Baran

Read/Watched: 2/05/24 - 2/11/24

kp@bit-101.com (Keith Peters) — Sun, 11 Feb 2024 05:00:00 -0500

https://www.youtube.com/watch?v=cKOSJ5AAwfc
- Betty Boop: Snow White, 1933. What an amazing piece of animation.
https://www.youtube.com/watch?v=qXZRWa8kyAE
- The previous video led me on a Cab Calloway rabbit hole. Which led me to this video, where I discovered the Nicholas Brothers. My jaw hung open through most of their dance routine.
https://github.com/jasonwebb/morphogenesis-resources
- An amazing list of resources on digital morphogenesis. I’ve explored a lot of these in depth over the years and some I’ve touched on only lightly. Others have been on my todo list for a while. And some are new to me. This will be a dangerous rabbit hole.

Read/Watched: 1/29/24 - 2/04/24

kp@bit-101.com (Keith Peters) — Mon, 05 Feb 2024 05:00:00 -0500

Shader Workshop
- Shader workshop.
The House Dog’s Grave
- I’m the last person to ever respond to poetry. I’ve tried. It usually does nothing for me. This one destroyed me.
Memos
- Been looking for a self-hosted Google Keep replacement. We’ll see how this goes.
Modeling organic branching structures with the space colonization algorithm and JavaScript
- This has been on my read later list for a while. Finally started digging into it.

FOTD: RoundTo, RoundNearest

kp@bit-101.com (Keith Peters) — Fri, 02 Feb 2024 05:00:00 -0500

Rounding floating point numbers is a pretty common need. Occasionally needed in calculations, but way more common in displaying numbers. You don’t want to be telling someone that two locations are 7.394739782 miles apart. 7.4 is probably fine. And 7.39 will satisfy most anyone. So let’s make a function that will let you round to any level of precision.

RoundTo

Rounding a floating point number to a whole number is generally built in to most math libraries. We’ll use the magic number 3.14159 for our examples.

round(3.14159) // 3.0

If you want to round to one decimal place:

First multiply by 10 to get 31.4159
Then round to get 31.0
Finally, divide by 10 for 3.1

For two decimal places:

Multipy by 100 = 314.159
Round = 314.0
Divide by 100 = 3.14

And so on. Multiply and divide by 1000 to get 3.142 (the final 1 gets rounded up to 2).

So, if we want to be able to round to an arbitrary number of decimal places, we need to be able to equate:

1 = 10
2 = 100
3 = 1000

The function is pow. It’s in most math libraries and raised a number to a certain power. So we can say:

pow(10, 1) //   10
pow(10, 2) //  100
pow(10, 3) // 1000
// etc.

In fact, pow(10, 0) equals 1. This means multiply by 1, round, divide by 1, which is the same as just rounding to zero decimal places.

Since we’ll be using the value twice, we might as well store it. So we get:

function roundTo(val, places) {
    mult = pow(10, places)
    val = val * mult
    val = round(val)
    val = val / mult
    return val
}

I wrote it out line by line for clarity, but you can shorten this to:

function roundTo(val, places) {
    mult = pow(10, places)
    return round(val * mult) / mult
}

And testing this…

roundTo(3.14159, 0) // 3.0
roundTo(3.14159, 1) // 3.1
roundTo(3.14159, 2) // 3.14
roundTo(3.14159, 3) // 3.142

There’s another bit of magic here though. It works with negative numbers too!

It happens that pow(10, -1) is 0.1. Take a number like 6789 and multiply by 0.1 and you get 678.9. Round it to 679 and divide by 0.1 to get 6790. You just rounded to the nearest tens place. So…

roundTo(6789, -1) // 6790
roundTo(6789, -2) // 6800
roundTo(6789, -3) // 7000

RoundNearest

Since that one was pretty easy, here’s a related one. Say instead of rounding to the nearest power of ten, you wanted to round to the nearest multiple of a number.

For example:

7 rounded to the nearest 3 would be 6. 8 rounded to the nearest 3 would be 9. 7 rounded to the nearest 4 would be 8.

This one is even simpler to code than roundTo:

function roundToNearest(val, mult) {
    return round(val / mult) * mult
}

And some examples:

roundNearest(7, 3) // 6
roundNearest(8, 3) // 9
roundNearest(7, 4) // 8

You might wonder when you’d need something like this. It’s good for quantizing values, or creating stairstepping kind of functions. Say I’m plotting values across a 400x400 pixel canvas with x going from 0 to the width of the canvas. And y will be the height of the canvas minus x.

for (x = 0; x < width; x += 5) {
    y = height - x
    circle(x, y, 2)
}

This gets us something like this:

Now let’s quantize the y value so it’s always a multiple of 40.

for (x = 0; x < width; x += 5) {
    y = height - x
    y = roundNearest(y, 40)
    circle(x, y, 2)
}

The result:

An even more useful example would be quantizing the colors of an image. Say each channel of each pixel had a value from 0 to 255. You could say:

value = roundNearest(value, 64)

Now each value would be 0, 64, 128, 192 or 256. You’d have to clamp that last value to have it equal 255. Now you’ve reduced the values from 256 to 5. It might not be the best way to go about this, but it should give you an idea of how you can apply the function.

Genuary Recap

kp@bit-101.com (Keith Peters) — Thu, 01 Feb 2024 08:47:55 -0500

I know, I just did a post entitled Genuary Wrap. So what’s this all about?

Well, mostly, the wrap was just about my last six entries. Every week I did a post showing the previous six or seven entries, so I just wanted to wrap up that series.

But Genuary 2024 was a pretty intense experience for me, and I wanted to say a few words about it as a whole. And there is one piece I wanted to do a deeper dive into. I could have actually stretched this into two more posts, but I need to move on.

Thoughts on Genuary 2024

As I just said, this was pretty intense. Thirty one prompts. Some were easy. Some were hard. Some were super interesting and I was excited to do them. Some I really didn’t want to go near. But it was good to get outside of my comfort zone.

Some of the prompts I liked the least were “do something in this particular style” or “… in the style of this particular artist.” It felt a bit too art-schooly. But OK, I did them and got something out of each one.

I do like the fact that the rules are so loose. “You don’t have to follow the prompt exactly. Or even at all. But, y’know, we put effort into this.” There were definitely days where I stretched the bounds of the prompt almost beyond recognition. I just took things where they led me.

Here are some of the things I learned, discovered, re-learned, re-discovered, took new deep dives into, etc.:

OpenGL and shaders - in Go! Big deep dive into this.
Signed Distance Functions. I still haven’t figured out what to use these for, but they are neat.
Poisson Disk Sampling. I forget how I wandered into this, but I figured it out, added it to my library and you’ll see it in multiple entries.
Easing functions. I put these in my library a year or two ago, but never used them. They got a lot of use this month.
Scenes and acts. A couple of new paradigms in my rendering engine. It’s going to make it way simpler to do larger, more complex videos this year. This is one of my goals and the idea of multi-scene videos is one that I’ve been trying to work out for some time now. I think I’m pretty close to nailing a really usable sytem for this.
Sound and music. Supercollider. I did a big deep dive into this last year, but it’s been on the back burner for a few months. At the end of 2023, I kind of clarified (for myself) my musical goals. I don’t necessarily just want to make songs. I want do be able to work with sound and music as accompaniments for visual pieces - videos. So I actually did that this month - in two pieces.
Isometrics. I had an old, unused library for this, and started to revive it. Hope to do more with it later.

Day 28 - Skeuomorphism

Your browser does not support the video tag.

This was my favorite piece of the month, and probably the one I spent the most time on. There’s a lot going on here, and a lot that went into it, so I wanted to deconstruct it a little.

The prompt is “skeuomophism”. Here’s a definition:

A skeuomorph … is a derivative object that retains ornamental design cues (attributes) from structures that were necessary in the original.[3] Skeuomorphs are typically used to make something new feel familiar in an effort to speed understanding and acclimation. They employ elements that, while essential to the original object, serve no pragmatic purpose in the new system.

From https://en.wikipedia.org/wiki/Skeuomorph

I wasn’t sure what I wanted to do for this. At some point I ran across this image:

From https://hackaday.com/2024/01/24/steamboat-willie-never-sounded-better/

This is a frame from Steamboat Willie, which recently went into the public domain, so is seeing a lot of use. I was interested in the jagged shape on the right edge. It’s the soundtrack, optically encoded into the film.

I’d thought of doing stuff like that before - encoding audio in a bitmap frame. Currently, I don’t have a way to analyze or play back audio in my Go-based workflow. There are libraries for that, but I didn’t want to go that deep for this piece. But I thought, what else can I encode in a bitmap? How about another bitmap??? Crazy, right?

The skeuomophism became replicating reading an image off of a tape drive. Rather than encoding the image as spots of magnetic charges, I wanted to try encoding the bits in pixels. Insane, but worth trying.

First I built an encoder. It takes a 100x100 pixel, grayscale image and loops through each pixel. Since it’s grayscale, I can get each pixel as a 256 level value. One byte. Eight bits. Each bit is a black or white pixel. Each byte goes into one column of a 10,000 by 8 pixel image.

Here’s a link to that image

Here’s a 800-pixel wide crop of that image:

And here it is zoomed in 10x:

You can see each column of the image is made up of eight pixels that are either black or white and represent one bit of that 8-bit byte.

With that in place, came the decoding.

First, I loaded the encoded data - the virtual “tape” containing the encoded bits of the image. I wanted to display it as it was reading, so I again scaled it by 10x. But it came out looking like this:

Not great. It’s got to do with the filter used in scaling. The default one in the Cairographics library interpolates, which usually results in a better result. But zooming in on pixels, you need to use a different filter. Unfortunately, my library had not implemented the different scaling filters. So I had to build out the C library wrappers for those. Wasn’t a big deal and now I have them there if I need them in the future.

Now I just go through the encoded bitmap, column by column. I sample each pixel in each column, turn them into bits, then into a single byte, set the color value based on that byte, and draw a single pixel.

I visually scrolled the encoded bitmap, but that’s just for show - so you can see which column is currently being read.

The final movie runs at 60fps, and it’s processing one pixel per frame, so it’s pretty damn slow. Just like reading data off of a tape drive.

So, like a true skeuomorph, this whole process serves no pragmatic purpose whatsoever. It’s literally encoding a bitmap into another bitmap that is eight times as large. Eight times larger in dimensions, though only 1k bigger in disk size. But it was a fun project and offered several cool challenges and people seemed to like it a lot on mastodon.

Genuary Wrap

kp@bit-101.com (Keith Peters) — Thu, 01 Feb 2024 07:00:00 -0500

Day 26 - Grow a seed

Your browser does not support the video tag.

Starts with a seed. It grows up and curls. At random points, it branches off. And on and on. Fades out in the end - I’ve been obsessed with smooth loops. Fading out before starting over is one way to deal with that.

Day 27 - Code for one hour

I had an isometric (bimetric technically) library kicking around. I dusted it off and started adding it to my main bitlib library. I used Poisson Disk Sampling with different radii for shading on the sides, which was neat.

Day 28 - Skeuomorphism

Your browser does not support the video tag.

I think this is my best piece of the month. It requires a little (lot) of explanation. So I think I’ll save that for another whole post.

Day 29 - Signed Distance Functions

Your browser does not support the video tag.

This prompt was in last year’s Genuary too. I was not happy with my example for that last year and not really happy with it this year. I did start adding SDFs to my main library, but I haven’t really figured out what to do with them yet. This was just a lot of random experimenting until I came up with something that looked interesting. I think it’s kind of using an SDF to create a flow field and drawing lines based on that. Meh.

Day 30 - Shaders

Your browser does not support the video tag.

This was done with a shader. That’s about all it’s got going for it. It did get me learning more about how to set up an OpenGL project, and I’ve been documenting my learning here: glsteps

Day 31 - Generative music

Your browser does not support the video tag.

One of my goals this year is to do longer, multi-scene videos with soundtracks. I totally rushed every part of this, but figured out a lot of the mechanics of how to accomplish this. Big learning on this: recently I added “scenes” to my rendering library. This was cool, but it was still all one big render. So if I wanted to change one parameter in scene 10, I still had to re-render every frame of every scene. With this piece I realized it’s going to be way more efficient to render scenes in chunks, export the video for each chunk and stitch them together at the end with ffmpeg. Here, I rendered anywhere from two to five scenes at a go. I started calling each chunk an “act”. This was all kind of ad-hoc, but I think I’ll adjust my library to allow for acts and scenes.

Read/Watched: 1/22/24 - 1/28/24

kp@bit-101.com (Keith Peters) — Mon, 29 Jan 2024 12:15:31 -0500

A new feature inspired by Arpit Mathur

https://www.youtube.com/watch?v=ikF0pZqpfYw
- Uconsole. Trying to convince myself not to buy one of these.
https://learnopengl.com/Introduction
- Been doing a deep dive into OpenGL. This is a good resource.
https://kylewbanks.com/blog/tutorial-opengl-with-golang-part-1-hello-opengl
- The OpenGL I’ve been doing has been in Go with the go-gl library. This article was indispensible.
https://www.quantamagazine.org/the-quest-to-decode-the-mandelbrot-set-maths-famed-fractal-20240126/
- I’m a sucker for an article on fractals. I thought this would be a fluff piece, but it’s actually a long, interesting read.
https://wedistribute.org/2023/11/introducing-faircamp/
- Interesting concept. Bills itself as a free alternative to Bandcamp, but it’s actually a static site generator aimed at musicians. Not sure if I fully understand it just yet.
https://buttondown.email/ownyourweb/archive/issue-08/
- I’ve been following this series. It’s part of what got me to reboot this blog in Hugo.

Photo by Pixabay

FOTD: Wrap

kp@bit-101.com (Keith Peters) — Mon, 29 Jan 2024 05:00:00 -0500

The next Function Of The Day is wrap. This one works in a similar way to clamp - if the value is in the range, it does nothing. But if the value is outside the given range, it will wrap it around to the other side of the range.

Wrap

Sometimes you want a value to remain in a specific range, but if it goes outside that range you don’t want to just clamp it. Instead maybe you want it to come back in at the other end of the range. Think of the old computer game, Asteroids. If you fly off one edge of the screen, you reappear on the other side. That’s how wrap works.

An example:

for (i = 5; i <= 12; i++) {
    x = wrap(i, 0, 10)
}

This should have x set to 5, 6, 7, 8, 9, 0, 1, 2.

In a simple example like this, wrap is acting as a modulus operator, just as if you said:

for (i = 5; i <= 12; i++) {
    x = i % 10
}

But that’s only because our range starts at zero. With wrap, the range can be anything. for example…

for (i = 2; i <= 12; i++) {
    x = wrap(i, 5, 10)
}

This wraps the value to always be in the range of 5 to 9. So x will equal 7, 8, 9, 5, 6, 7, 8, 9, 5, 6, 7.

Note that the range is inclusive of the lower value, but exclusive of the upper value.

This also needs to work with negative numbers and ranges. We have to be careful here because, oddly, the built-in modulus function (usually %) works differently in different languages when it comes to using negative numbers.

For example, in Go, as well as C, C++, Java, and many other languages:

-7 %  3 = -1

 7 % -3 =  1

However, in Python and some other languages:

-7 %  3 =  2

 7 % -3 = -2

In case you’re interested, here’s a full write up on why it’s different:

https://torstencurdt.com/tech/posts/modulo-of-negative-numbers/

To be careful with this, I just avoid using the built-in modulo operator and use floored division, as defined by Donald Knuth. The floored division implementation of modulo looks like this:

a - b * floor(a / b)

This gives us the same results that Python gives for modulo. Whether or not you agree that’s the “correct” way, it will work the way we expect it to for wrapping, and using this definition rather than the language default operator will work the same whatever language you use it in.

Code

So finally we come to the implementation of wrap

function wrap(val, min, max) {
    val -= min
    max -= min
    return min + val - max * floor(val / max)
}

First, we just subtract min from both val and max. This puts the range into the form of zero to whatever. Once it’s there, we can just do a modulo (using the floored division method), and then add min back to the result.

Example(s)

In this example, I made 100 random points that move left to right. There’s a 400x400 canvas, but I want them to stay in a square in the center which goes from 100 to 300 on both the x and y axes.

Your browser does not support the video tag.

I drew the center rectangle and then drew each point, then added 5 pixels to each point’s x value. And then I used wrap to wrap the x value to a range of 100, 300.

// for each x, y point...
circle(x, y, 1)
x += 5
x = wrap(x, 100, 300)

Now when a point goes beyond an x of 300, it wraps back to 100.

Another way I use this function is in adjusting angles. Generally, it’s nice to have your angles in a range of 0 to 2 * PI. Or maybe -PI to PI.

But sometimes you calculate an angle using some complex algorith, which may result in the angle being way outside of those ranges. Luckily angles naturally wrap around a full circle, so that 370 degrees is the same as 10 degrees, and -10 degrees is the same as 350 degrees. So our wrap function is an easy way to get these into line.

I made two functions that are useful for these cases.

// TAU = PI * 2 

function wrapTau(value) {
	return wrap(value, 0, PI * 2)
}

function wrapPi(value) {
	return wrap(value, -PI, PI)
}

I know this isn’t always necessary. Trig functions will still work with multiples of angles, but if you need to display the angle number, or use the angle in a function like norm or map that requires it to be within a range, this is helpful.

FOTD: Clamp

kp@bit-101.com (Keith Peters) — Sun, 28 Jan 2024 06:00:00 -0500

Clamp

We’ll continue the discussion of functions that work with values within ranges. The clamp function is a pretty simple one. It just makes any value that’s higher than the range equal to the max value of the range, and any value that’s lower than the range equal to the min value.

Code

function clamp(value, minimum, maximum) {
	if (value < minimum) {
		value = minimum
	}
	if (value > maximum) {
		value = maximum
	}
	return value
}

If value is either greater than maximum or less than minimum, it “clamps” it to be within that range.

This is the brute force method. If your language has min and max functions, this can be simplified a bit. These are often part of a math library, i.e. Math.min, Math.max or something similar. In pseudocode, I’ll just call them min and max:

function clamp(value, minimum, maximum) {
    return max(minimum, min(value, maximum))
}

Working inside out, it takes the smallest value, comparing value and maximum. This will result it maximum at the most. It feeds that into the outer call to take the largest value between that and minimum. It has the same effect of clamping it into that range.

There’s one upgrade that can make the function even more valuable. Allow for the minimum and maximum values to be reversed. If you’re calling the function directly, it’s easy enough to make sure the smaller of the two goes first. But if it’s part of a larger algorithm, it might happen that minimum gets assigned a value that’s larger than maximum. If so, you can just swap them and have it work fine. Here’s an easy take on this using a temp variable:

function clamp(value, minimum, maximum) {
    if (minimum > maximum) {
        temp = minimum
        minimum = maximum
        maximum = temp
    }
    return max(minimum, min(value, maximum))
}

If you’re lucky enough to be using a language that allows the following syntax, it makes it much easier…

function clamp(value, minimum, maximum) {
    if (minimum > maximum) {
        minimum, maximum = maximum, minimum
    }
    return max(minimum, min(value, maximum))
}

Example(s)

One real life example where I had to use clamp recently was in calculating color values. Normally rgb color channels can either go from 0.0 to 1.0 as floats, or from 0 to 255 as integers. In Cairographics, which I usually work with, it’s 0-1, and if you assign a value outside of that range, it doesn’t matter. I guess internally it clamps it to be within that range.

But somewhat recently… OK, a couple of years ago, I was creating a bitmap encoder. This stored an array of pixel channel values, which eventually got written to a file with the correct headers. Here’s a very simplified pseudocode version of the original function I came up with for setting the value of a single pixel:

function setPixel(x, y, r, g, b) {
	index = (y * imageWidth + x) * 3
	pixels[index] = b
	pixels[index + 1] = g
	pixels[index + 2] = r
}

Because this was a one dimensional array, I had to calculate the index of the array to store the red, green and blue values. They went in sequential elements in reverse order - blue, green, red. Each element was a float from 0.0 to 1.0.

For the most part, this worked great. But now and then, I’d get a group of pixels that would just glitch out and render random, unpredictable colors - generally in very bright areas of the image. After a bit of debugging I realized that sometimes the calculated value of one or more channels would be greater than 1.0. It was either wrapping around or overflowing or something. At any rate, it looked like hell. Once I realized what the problem as, all I had to do was clamp the values to a range of 0 to 1.

function setPixel(x, y, r, g, b) {
	index = (y * imageWidth + x) * 3
	pixels[index] = clamp(b, 0, 1)
	pixels[index + 1] = clamp(g, 0, 1)
	pixels[index + 2] = clamp(r, 0, 1)
}

Problem solved! If a channel value was too bright, it would just max out at 1.0. I don’t think I had an issue of values going negative, but the code would have guarded against that as well.

OpenGL Steps: The Beginning

kp@bit-101.com (Keith Peters) — Sat, 27 Jan 2024 21:35:20 -0500

Just a link to a project I started. I’ve been working on learning OpenGL using go-gl. It’s a lot to take in, so I’ve been trying to break it down step by step, from the bare minimum to display a window, and then adding as little code as I can on each step - and understanding as much as I can about what’s going on in each step.

I’ve been working on preserving each step as a separate file as I go, and making each one executable. And in each one I comment all the major changes since the last step. This has been a great process. I’m up to step 7 now and learned a whole lot today. This will continue.

Here’s the link if you want to follow along:

https://github.com/bit101/glsteps

FOTD: loopsin

kp@bit-101.com (Keith Peters) — Sat, 27 Jan 2024 06:00:00 -0500

This function, loopsin, is one of my favorite functions ever, and one I am quite proud of. Though I’m sure I’m not the first to think of it, I came up with it on my own and have not seen it used or written about anywhere else. It’s not rocket science, but it’s immensely useful in animating perfect loops. I’ll also go a bit into the history of how I evolved this function over time.

LoopSin

Background

A popular (with me anyway) animation technique is to use a sine wave to oscillate some property of an object - make it go back and forth or up and down or get big and small, or whatever. You just feed the sine function an ever-increasing t value, which gives you values that go back and forth between -1 and 1. Multiply that by some offset and add it to a center position, and you have oscillation.

// t initialized to 0
offset = 10
center = 100
speed = 0.01

x = center + sin(t) * offset
t += speed

Sometimes you don’t really know what the center or offset are though. You just know you want this object to go back and forth between two values, say, 117 and 236. Well, our old friend map is useful here.

// t initialized to 0
speed = 0.01
min = 117
max = 236

s = sin(t)
x = map(s, -1, 1, min, max)
t += speed

Evolving the function

When I started using Go for my creative coding, I moved away from interactive JavaScript pieces that ran forever and started creating animated gifs or videos that had a finite number of frames. One thing that really annoys me is gifs that run for a few seconds and then jump back to the start. It’s jarring and kills any positive aesthetic value the animation set out to create. So I wanted to create perfect loops.

Well, I knew how many frames I had, and I knew the current frame. By dividing those two, I could get a t value that went from 0 to just under 1 (see below). I could then map that to an angle from 0 to 2 * PI, and then run that through sine and map it to my range. This would form one full cycle through the range, which should then loop perfectly. So my function sort of looked like this:

t = frame / frameCount
min = 117
max = 236

x = loopsin(t, min, max)
circle(x, 50, 5)

function loopsin(t, min, max) {
    a = t * 2 * PI
    s = sin(a)
    return map(s, -1, 1, min, max)
}

And here’s what that looks like, using x as the position of a drawn circle:

Your browser does not support the video tag.

A perfect loop!

I used something almost just like this for years. But when I started doing animations that had multipe scenes, I had to improve this somewhat. To show the problem, I added a second scene to the above. I’m expecting the first scene to animate the circle from 117 to 236 and back. In the second scene, I want to keep the x position at 117, and animate the y position from 50 to 100 with this code:

// scene 2
t = frame / frameCount
min = 50
max = 100

x = 117
y = loopsin(t, min, max)
circle(x, y, 5)

So again, I’m expecting it to go right, left, then down, up. But here’s what I get:

Your browser does not support the video tag.

Ouch.

The problem is that I’m using sine, and mapping -1, 1 to min, max. The result of sine with an angle of zero will be zero. As the angle increases, it will go up to 1 and back to 0 then -1 and back to 0.

So the animation will start and end at the midpoint of min and max. Meaning it will end at the midpoint of the ranges in scene 1 and jump to the midpoint of the ranges in scene 2. Aha!

I can change that by using cosine rather than sine. The cosine of 0 is 1, it will then go down to 0 as the angle increases, then down more to -1 and back to 0 and 1. Better. But I’m mapping -1, 1 to min, max. This will have the animation start at max and animate to min. So I need to swap those so it goes from min to max instead. Here’s the new function:

function loopsin(t, min, max) {
    a = t * 2 * PI
    s = cos(a)
    return map(s, 1, -1, min, max)
}

Now maybe you’re thinking that I should change the function name to loopcos. Whatever. Name it what you want. It’s still doing a sinusoidal curve of the values, so I’m not going to get hung up on it! :)

Making that change, things now work as expected:

Your browser does not support the video tag.

Now that we’ve got it working just right, we can simplify it a bit…

function loopsin(t, min, max) {
    return map(cos(t * 2 * PI), 1, -1, min, max)
}

You can apply this to more than just position. Here I’m using it to animate three different properties of objects:

Your browser does not support the video tag.

In the first one, I’m calculating a radius with loopsin. In the second, I’m using it to calculate a color value. And in the final one, a rotation. Perfect loops in every case. Rough pseudocode just to give you the idea:

t = frame / frameCount

r = loopsin(t, 10, 60)
circle(80, 75, r)

v = loopsin(t, 0.0, 1.0)
rgb(v, 0, 1 - v) // rgb values go from 0.0 to 1.0 here
circle(200, 75, 50)

a = loopsin(t, -PI/2, PI/2)
translate(320, 75)
rotate(a)
rect(-40, -40, 80, 80)

If you see an animation made by me, there’s a 90% chance that it uses this function.

Advanced implementation details

Earlier I said the by dividing the current frame number by total frames, will give us a number from zero to just under one. This is because I zero-index my frames. If I have 60 frames, the frame numbers are 0 to 59. But I divide by the frame count, 60. So frame 0 gets a t of 0.0. But frame 59 will get a t of around 0.98333…, which seems like a flaw. But in fact, it’s exactly what we want.

When we convert t to an angle, we’ll get angles from 0.0 to something just under 2 * PI. And when we pass that to the cosine function, the values we’ll get back will start at exactly 1.0, go down to -1.0 and back up to just shy of 1.0.

When we map that to our min/max range, we’ll start with exactly min, move through to max and then back to just shy of min. That’s perfect. Because on the next loop, it will start of at exactly min again.

If we “fixed” things so that t went all the way to 1.0, then we’d start and end each loop at exactly min. Meaning we’d have one duplicate frame where it looped. You might not notice it, but in some cases, it could cause a tiny stutter.

Alternately, you could have your frames start at 1, rather than 0. For 60 frames, this would mean the last frame would get a t of 60/60 or 1.0, but the first frame would be 1/60, or slightly over 0.0. It should work either way, as long as you’re avoiding that duplicated frame. For me it makes more sense to start at 0.

Genuary2024, Days 19-25

kp@bit-101.com (Keith Peters) — Fri, 26 Jan 2024 18:54:09 -0500

Days 19 through 25 of genuary 2024!

Day 19 - Flocking

Your browser does not support the video tag.

This isn’t really flocking. I explored flocking algorithms many years ago. I may have even written a book chapter about them at one time. They’re super interesting but I wasn’t super interested in diving into them at this point in time. So I did a little particle system. Experimented with scenes and how to make it loop smoothly. I just have the particles swooping in from the left, randomly wandering for a bit and then swooping out to the right. That leaves the canvas clear for the next swoop.

Day 20 - Generative typography

Your browser does not support the video tag.

This is an old technique I’ve been using for years, but I like that I was able to push it to a bit of a new level this time. Just drawing some lines across the screen. Test a hidden canvas that has something drawn to it. If you hit the drawing, you change the y position of the line. I tweened the height, and got some moving waves going in the lines. Kind of looks like an island rising out of the sea. Is it “generative typography”? I’d say no, but I like it.

Day 21 - Use a library that you haven’t used before

Your browser does not support the video tag.

Well, this is OpenGL. I’ve used it before, but it’s been many a year. Done some WebGL too. But really only scratched the surface of either. It’s been on my list to explore it a bit more again, so this was a good opportunity. There’s a go-gl library for Go, that I was able to get running. Most people to the colored triangle as their GL “hello world”. I render a few thousand points. Anyway, I got a project template set up for OpenGL projects in Go, so once I’m done with the rest of this genuary melarky, I’ll be able to explore some more. I’ll also use this for Day 30, I think, which is Shaders.

One thing I did here to create the animation was to export a bitmap after each frame render. I used a custom bitmap encoder I made a couple of years ago. Worked like a charm.

Day 22 - Point - line - plane.

Your browser does not support the video tag.

I’m really not a fan of the “do something in the style of such and such artist” prompts. This one was pushing Kandinsky. Aight. I just did an animation of a bunch of points, then lines connecting them, then they turn into a plane, which turns into a point and starts all over again. A few cool techniques in this one though. I figured out how to do Poisson Disk Sampling, which is pretty neat. And added that to my library. Then I did my tried and true “connect-nearby-points-with-lines” technique. Followed by a convex hull algorithm, and an easing function. All tied together in a number of scenes.

Day 23 - 32x32

Your browser does not support the video tag.

As a lot of people discovered, this prompt was purposely rigged to give different x-by-x numbers to different people. I got 32x32 on my computer, but 128x128 on my phone. I went with 32. I made a 32x32 grid of squares. In each one I drew concentric, rotated squares, a technique I’ve been playing with lately. They formed an interesting pattern at this scale. I timed them so that pattern formed an expanding ring.

Day 24 - Impossible objects

Your browser does not support the video tag.

Subtitle, “undecided geometry”. I think a lot of people did isometric stuff for this one. Thank you Monument Valley. I tried to make this so that it would force your eye to see it differently when it rotated. I can see both ways each time though, but it does kind of shift around.

Day 25

Long prompt for this one:

If you like generative art, you probably have some photos on your phone of cool looking patterns, textures, shapes or things that you’ve seen. You might have even thought, “I should try to recreate this with code”. Today is the day.

I had been on a flight a little while back. Took some photos from the plane on the runway, trying to catch a sunrise. But it was rainy and I mostly just got a picture of raindrops on the window. Tried to recreate that. It came out… OK, I guess.

Six days to go!

FOTD: Map

kp@bit-101.com (Keith Peters) — Fri, 26 Jan 2024 06:00:00 -0500

I’ve alluded to the power you get when you combine two or more useful functions. Today we’ll combine norm and lerp.

Map

The map function has a bit of a learning curve because it has five parameters to understand, but it’s one of my most used functions. Once you really get it, you’ll probably use it all the time. Best to see it, then walk through it.

func Map(srcValue, srcMin, srcMax, dstMin, dstMax) {
	t = norm(srcValue, srcMin, srcMax)
	return lerp(t, dstMin, dstMax)
}

Internally, it simply combines norm and lerp. But what a combo!

The first three parameters are the same as the parameters to norm. a value, minimum and maximum. In fact, those three parameters are fed straight into the norm function, giving back a normalized value, t. The first three parameters are the source value and range.

Then, this t value is used with the minumum and maximum values of the destination range.

Example(s)

Let’s walk through a simple one first.

x = map(7, 5, 10, 300, 400)

First, the function will normalize 7 in the range of 5 to 10. This is 0.4.

Then it uses 0.4 as a t value to lerp from 300 to 400. This gives us 340.

Now a more useful example. I’ve generated an array of 100 random numbers called nums. The way I’ve generated them, I don’t have any idea of the min and max values in the array. But I have a function that will go through and get these. Cool. Now I can loop through the list and get each number and map it to a value in the range of 0 to the height of the canvas, 300. This gives me the y value.

min = getMin(nums)
max = getMax(nums)

for (i = 0; i < nums.length; i++) {
    n = nums[i]
    y = map(n, min, max, 300, 0)
    x = map(i, 0, nums.length, 0, 800)
    lineto(x, y)
}
stroke()

For the x value, I can map the iterator, i with a source range of 0 to the length of the array, and a destination range of 0 to the width of the canvas.

Then I just draw a line to each new point.

Note also that instead of using the destination range of 0, 300 in the map for y, I’m using 300, 0. The great thing about norm, lerp and map is that you can flip the ranges without breaking anything. This lets the top of the canvas (zero) represent the maximum value and the bottom of the canvas (300) represent the minimum value, like you’d expect.

Without any math or knowledge of the values in the array, or even how long the array is, we walk away with a perfectly scaled graph of the values.

I’ll note that the map for x is trivial:

x = map(i, 0, nums.length, 0, 800)

I could have just as easily said

x = i / nums.length * 800

But as before, this requires me to think about math in my head, whereas using the function I just think about the input range and output range. Also, if I decide to change the dimensions or position of the graph, like putting a border around it, I can do so easily by just changing the output ranges…

y = map(n, min, max, 280, 20)
x = map(i, 0, nums.length, 20, 780)

Resources

I did a video on this function on my Coding Math Youtube channel. View it here: Coding Math Mini #3, Map

Photo by Markus Spiske

FOTD: Lerp

kp@bit-101.com (Keith Peters) — Thu, 25 Jan 2024 04:47:06 -0500

Lerp

The last Function Of The Day was norm, which mapped a value in a range to a value between 0 and 1. The lerp function does the exact opposite. It takes a normal value (something between 0.0 and 1.0 and maps it into a given range. That is to say it interpolates between the minimum and maximum of that range, in a straight line - linearly. Linear Interpolation = lerp.

The Code

function lerp(t, min, max) {
	return min + (max - min) * t
}

The t parameter (somewhat of a convention here) is the normal value, and min and max define the range.

Generally, t is considered to lie between 0 and 1. However, it can be smaller or larger. This will result in an output value that is outside of the range, which can actually be useful.

Once again we get the size of the range by subtracting min from max. Multiplying that by t puts you in the right place within that range. And adding that to min shifts the range so it falls in the right place. If t is zero, it will return min and if t is one, it returns max.

Example(s)

It’s often useful to think of the normal or t value as a percentage. It can be useful inside a for loop. Say you want to draw a number of shapes across the width of a canvas. You can loop from 0 to 20 and divide the iteration value by 20 to get your t. Then use that in lerp to get an x value and draw a shape at that point.

for (i = 0; i <= 20; i++) {
    t = i / 20.0
    x = lerp(t, 0, 800)
    circle(x, 50, 5)
}

Now this is a trivial example because it would be just as easy to say:

x = t * 800

But say you wanted the dots to go from some different points. Just change the 0, 800 to something else.

x = lerp(t, 100, 700)

No additional math required.

You could then lerp t to some other values like -PI to PI and then use that in a sine function.

for (i = 0; i <= 20; i++) {
    t = i / 20.0
    x = lerp(t, 0, 800)
    a = lerp(t, -PI, PI)
    y = 150 + sin(a) * 100
    circle(x, y, 5)
}

The lerp function is, all by itself, super useful, but becomes even more powerful when mixed with other useful functions. As we will see, eventually.

Resources

I did a video on this function on my Coding Math Youtube channel. View it here: Coding Math Mini #2, Linear Interpolation

Photo by Markus Spiske

FOTD: Norm

kp@bit-101.com (Keith Peters) — Wed, 24 Jan 2024 05:06:07 -0500

I’m going to do a series of short articles, labeled FOTD - Function Of The Day. These might be math functions, drawing functions, PRNG functions, color functions, etc. Mostly these will be useful for visual or creative coders.

Each post will be short. Just a single function, how it works, what it might be used for, and maybe an example or two. No particular schedule for these. As the mood strikes me. But between the two main libraries I’ve curated, blcairo and bitlib, I have close to 1000 functions, so there should be no shortage of material. Maybe worth noting that I’ll usually be using some kind of pseudocode for all the examples.

Norm

Today’s function is norm. This takes in three parameters: value, min and max. It returns a number between 0.0 and 1.0 based on where value falls between min and max. Actually, if value is less than min it will return a negative number, and if value is greater than max it will return something larger than 1.0. But in most use cases, the min/max range will be contrived so that value fits into it cleanly.

The Code

In pseudocode, norm looks like this:

function norm(value, min, max) {
	return (value - min) / (max - min)
}

The max - min part gets the range between those two values. Subtracting min from value puts value into that range (if it can be put into that range). Dividing the two give you zero if value equals min, one if value equals max and somewhere between if value is between the two.

Example(s)

There’s one use for norm that’s immediately useful. I often use Perlin or Simplex noise to generate patterns or what have you. In some libraries these noise functions return a value between 0.0 and 1.0, which is already normalized. But in most, they return a value from -1.0 to 1.0. I have color functions that take values for channels in floats between 0.0 and 1.0, so I need to convert between the two.

I could say:

colorValue = (noise + 1) / 2

// or...

colorValue = noise / 2 + 0.5

Adding one moves the noise range to 0.0 to 2.0. Dividing by two puts it into a normalized range. The second version does something similar.

With norm I can say:

colorValue = norm(noise, -1, 1)

It’s a couple more characters, but for me, less of a cognitive load. There’s no math involved, I’m just mapping from -1, 1 to 0, 1.

Norm is often used as just one step in a calculation. Once you have the norm, you might want to do some further calculations with that value.

Say I was using atan2 to calculate angles between two points, like a center point and another pointer. This gives you a number from -PI to PI. You can use norm to normalize that to a 0, 1 range. Multiply that value by the width of the canvas, in my case 400, and now you can use that to draw a kind of gauge showing the value. Like so:

angle = atan2(y - centerY, x - centerX)
value = norm(angle, -PI, PI) * 400
moveto(value, 0)
lineto(value, 20)

Your browser does not support the video tag.

Now this might seem a little bit contrived, because honestly it is. The fact is, I don’t use norm by its own all the much other than the noise example above. But it goes well inside other more complex functions, some of which we’ll visit later.

Resources

I did a video on this function on my Coding Math Youtube channel. View it here: Coding Math Mini #1, Normalization

Photo by Markus Spiske

Books

kp@bit-101.com (Keith Peters) — Tue, 23 Jan 2024 05:00:55 -0500

For a good part of my tech career, writing has gone hand in hand with coding. Early on, I was writing tutorials and have had very active blog for many years. I even wrote several magazine articles. But writing books was a big part of what I did too. I found myself looking at some of my old books recently and wanted to do a recap. Most of these are based on Flash and versions of ActionScript. This might explain why my output has slowed in recent years.

The following list is complete (I think), but very definitely NOT in chronological order. More in stream of consciousness order.

Flash Math Creativity

This is the one that started it all. 2001. I had written a few Flash tutorials that had gotten a lot of attention. I was contacted by Friends of ED, a prolific publisher of Flash and web related books. They asked me if I wanted to write a chapter for this book they were starting to work on. Each chapter would be some creative, math-based idea, with lots of imagery, included code, and a write up of the inspiration and mechanics behind the piece. I said yes of course. And that started everything.

Flash Math Creativity, Second Edition

This is not the next book I worked on, but I’ll put it next because it’s a follow up to the original. It’s actually basically the same book. The original was written for ActionScript, later to be called “ActionScript 1.0”. The second edition was rewritten for ActionScript 2.0. I did the entire rewrite. I converted all the code to AS2 and then went through all the text and updated the technical descriptions to reflect those changes. It was a bit strange rewriting the chapters that these other great authors (many of whom I looked up to and admired) had written. I limited myself to just changing the technical descriptions only and touching as little of the rest of the chapters as I could.

The Most Wanted Series

Flash MX Most Wanted Effects and Movies and Flash MX 2004 Games Most Wanted were part of a series of books Friends of ED put out where they got a bunch of authors to write a chapter or more on related subjects. To be honest, I’m not sure where the “most wanted” part came in. It was a long time ago, but as I recall they just asked me to submit some ideas, which I did, and wound up writing those chapters. I think I did two or three chapters each for these two books. I don’t remember what the other ones in the series were, but I’m quite sure there were more.

Other Chapter Books

Friends of ED loved the whole chapter book idea. And I guess it worked for them. These were a few other books I did anywhere from one to three chapters for.

Bite-Size Flash MX, Adventures in Design Optimization was early “size coding” I guess you’d say. I used have PHPBB forum on my site and each month I’d sponsor a “25 lines” contest. The idea was to write the coolest possible Flash piece in 25 lines or less of ActionScript. It was freaking amazing what people did. Of course eventually it got out of hand and people were really bending the concept of what 25 lines was and it got caught up in rules and became less fun. Anyway, this was along those lines. 4k or less.

Flash MX Studio was I guess some kind of professionally targeted book on how to do cool stuff in Flash. I don’t remember much more about it at this point.

Fresh Flash, New Design Ideas with Flash MX was um… who knows? New design ideas with Flash MX I guess. I wrote some chapters for it apparently. I couldn’t tell you what they were about.

Extending Flash MX 2004

Flash MX 2004 was an IDE that included an extensibility layer. You could write various extensions for it - plugins, basically. It was all in a language called JSFL - JavaScript Flash. You could create your own UIs within the IDE, create new drawing tools that let the user draw custom shapes, script actions that automated the whole IDE, etc. It didn’t get used all that much, which was sad because you could do some pretty amazing things with it. I created an extension that let you play Asteroids right in the IDE. And another one that gave you a drawing tool for creating 3D objects. Another Flash developer I knew at the time, Todd Yard, was one of the few other people who was into this stuff as well. So we pitched this book and wrote it. It was very cool, but hardly anybody ever used JSFL ever. And I think they removed the whole layer from the next version, Flash 8.

New Masters of Flash, Volume 3

Being invited to write a chapter for New Masters was, for me, at that time, the best thing that had ever happened to me. It was like winning gold at the Olympics. I had the previous two versions of this book and had poured over them for years. The people in those books were my heros, my role models, eventually many became good friends too. So when I got into volume 3 it was huge. I wrote a chapter on particle systems, which was an obsession I had at that time. And lots of background on myself and my journey as a creative coder. I re-read it recently, prepared to cringe. But it was actually not bad. I enjoyed it.

Making Things Move! The Quadrilogy

For some years I’d been writing tutorials, writing books, doing daily experiments, seeking out cool new techniques for coding all kinds of visual, interactive, physics- and math-based Flash stuff. I wanted to put this all in a single book. I knew it would be a great book. I pitched it to Friends of ED (by then shut down and reincarnated by Apress). They were not interested. I pitched it to a couple of other publishers. No deal. I pitched it a couple more times to Friends of ED and finally I got a YES!

This was the first book I wrote cover-to-cover by myself. It took months. It was an amazing process. It was the culmination of years of daily obsession with Flash, vomited into a single volume. It’s the professional accomplishment I am the most proud of.

Here are the opening paragraphs:

Though Flash is now very officially dead, I still keep a copy of this book on my shelf and regularly find myself looking up techniques I wrote about nearly 20 years ago. Others have told me the same. The techniques and formulas are timeless.

The book went over quite well. For a while it was the best selling book on the ActionScript language. It sold a lot of copies. The title “Making Things Move!” was my initial idea. The publisher wanted to put the book in its “Foundation” series, so the title became “Foundation ActionScript Animation”, with “Making Things Move!” a subtitle. But it’s always been “Making Things Move!” to me.

Funny story, one of the initial cover images they came up for the book was a group of zombies. Get it? Making things move? Luckily, they iterated a bit on that one and came up with something quite nice.

Like Flash Math Creativity, it was written for an earlier version of ActionScript, so eventually I had to write an AS3 version.

By then I’d learned and thought of some more stuff I wish I’d put in the book. I kept a list of subjects and when it became long enough, I wrote a sequel, AdvancED ActionScript Animation.

This was just more cool techniques. It fit in well with the original.

Some time later, when Flash was breathing its last breaths, I was asked to convert Making Things Move to JavaScript. I said no. I regret saying no, but it is what it is. They got another author, Billy Lamberta, to do the conversion. I endorsed it, got my name on the cover, and even got royalties for it. But I should have written it. Oh well. Billy did a great job.

ActionScript 3.0 Cookbook

This was the first book I worked on that wasn’t for Friends of ED / Apress. I wrote it with a couple of friends, Joey Lott and Darren Schall. We each wrote roughly a third of it, as I recall. Maybe Joey wrote more. It was cool I guess. I’m not a huge fan of the cookbook format. It was less fun to write than a less structured project.

iPhone User Interface Design Projects

This was also for Apress, but not part of the Friends of ED line. I was doing iPhone development for a while and got asked to write a chapter (maybe more than one?) for this book. I don’t remember what I wrote about. I also spoke at some iOS conference around that time (not WWDC or anything, just a smaller conference put on by some people I knew). I thought iOS dev was going to be my next big thing. I had a number one game on the early App Store - Falling Balls. But I got fed up with how Apple controlled everything and was already becoming developer hostile. I got an Android phone, moved to Linux for a while and that was that.

Playing With Chaos

This was the last actual book I wrote, more than ten years ago. And it was the first book I self-published. I’d done a talk by the same name in Cologne, Germany at the Beyond Tellerand conference, hosted by my good friend, Marc Thiele. It was all about coding Mandelbrot and Julia sets, strange attractors, IFS fractals, cellular automata, etc. It was very well received and I thought it would make a good book. I sat down and wrote it and published it on Amazon Kindle. I also published it on Gumroad. It still sells copies and people say pretty good things about it. So that’s nice.

Coding Curves

This is not really a book, but it might be some day. I had this as an idea as a follow up to Playing With Chaos and it was going to be called Playing With Curves. I started writing this book every year or two. And I would generally start from scratch each time and get a chapter or two before giving up.

In 2022, I decided to do it as a series of online chapters. Here it is: Coding Curves. It wound up 14 chapters long, written and published one by one, from November 2022 to February 2023. I’m glad I finally got this done and doing it as a series of blog posts was a good technique. If I hadn’t done it that way, it would probably still be something I planned to do someday.

As I said, I might eventually gather all the chapters together and publish it as a single book. Maybe even for free at this point. Another project on the pile.

Coding Colors

This is a future project. It will be along the same lines as Coding Curves. I have a table of contents, but there’s a lot to figure out on how to do this one, to make it language/platform agnostic, since different languages and graphics APIs represent color very differently. I’ll get there someday.

Making Some Points

kp@bit-101.com (Keith Peters) — Sat, 20 Jan 2024 15:00:46 -0500

Some Go Basics

A lot of what I do is with two-dimensional points. As covered earlier, I’m doing all my work in Go, so I have a Point struct that looks like this:

type Point struct {
	X, Y float64
}

And a construction method that looks like this:

func NewPoint(x, y float64) *Point {
	return &Point{X: x, Y: y}
}

In Go, you can them make “methods” for the Point struct that look like this:

// Angle returns the angle from the origin to this point.
func (p *Point) Angle() float64 {
	return math.Atan2(p.Y, p.X)
}

The (p *Point) part just marks this as a method that can be called with any point as a “receiver”. So I can say:

p0 := geom.NewPoint(100, 100)
dist := p0.Angle()

This is all in the geom package, so any time I’m calling any of this outside the package, I prefix it with the package name.

Quite often I’m dealing with a whole bunch of points. So I can put them in an array (aka a “slice” in Go). A point array would look like this:

points := []*geom.Point{}

// add some points to this list:
points = append(points, geom.NewPoint(100, 100))
points = append(points, geom.NewPoint(200, 100))
points = append(points, geom.NewPoint(100, 300))

fmt.Println(len(points)) // 3

I can also make a random point within a rectangle:

for i := 0; i < 20; i++ {
    points = append(points, geom.RandomPointInRect(0, 0, width, height))
}

In my actual drawing library I have a method that will draw a list of points like this:

context.Points(points, 5)

The second arg is the radius that each point will be drawn with. I made it large here so you can see it:

There’s also a method to draw a path with a point list:

context.StrokePath(points, false)

The boolean parameter sets whether or not the path should be closed. i.e. draw a line back to the first point.

And the result:

Moving on from there…

All this is pretty basic stuff. But recently I enhanced this by making a new PointList type. This is just an alias to an array of points. But once you make it its own specific type, a lot of opportunities appear. Here’s the type definition and one of the construction methods.

// PointList is a slice of Points
type PointList []*Point

// NewPointList creates a new point list
func NewPointList() PointList {
	return PointList{}
}

Typing geom.NewPointList() feels lot better than typing []*geom.Point{} even though it’s a few characters longer. At least it feels better to me, and it works better with auto-complete.

And now I can attach methods to this new type, like for adding new points:

// Add adds a point to the list
func (p *PointList) Add(point *Point) {
	*p = append(*p, point)
}

// AddXY adds a point to the list
func (p *PointList) AddXY(x, y float64) {
	*p = append(*p, NewPoint(x, y))
}

Now maybe I can do something like this:

points := geom.NewPointList()
for x := 0.0; x < 800; x += 10 {
	y := math.Sin(x*0.01)*300 + 400
	points.AddXY(x, y)
}
// here, I'll point AND stroke
context.Points(points, 3)
context.StrokePath(points, false)

Typing points.AddXY(x, y) definitely feels better than points = append(points, geom.NewPoint(x, y))

This gets us this:

Now points have other methods on them, many of which you surely expect. Methods for translating, scaling and rotating and some other fun stuff. If I want to, say, rotate all the points by some small amount, I can loop through the point list and rotate each one like so:

for _, p := range points {
    p.Rotate(0.5)
}
// then draw as before...

Now the whole path is rotated from the origin.

If I want to rotate the path from some specific point instead of the 0, 0, origin, I can used RotateFrom on each point:

for _, p := range points {
    p.RotateFrom(400, 400, math.Pi/2)
}

This rotates everything 90 degrees around the center for the image:

I have point methods for Translate, Rotate, RotateFrom, Scale, ScaleFrom, as well as things like Randomize and Noisify. And I used them fairly often. After a few thousand times recreating the above for-loop, I realized I could just add methods to PointList that do all that for me. So now PointList has all these same methods. They work exacly how I’d been doing it manually. Here’s an example:

// ScaleFrom scales all the points in a list using the x, y location as a center.
func (p *PointList) ScaleFrom(x, y, sx, sy float64) {
	for _, point := range *p {
		point.ScaleFrom(x, y, sx, sy)
	}
}

Now, in the above example, in order to rotate the whole list, I can just say:

points.RotateFrom(400, 400, math.Pi / 2)

Here’s a demo of all of the above. Ten scenes showing: translate, scale, scale from, rotate, rotate from, randomize, and a few manipulations of noisify.

Your browser does not support the video tag.

Summary

That’s about it. Just an example of iterating a code base to make complex things much easier. If you’re interested in the code itself it’s at my bitlib repo

In the off chance that you’re interested in the entire code for this whole last demo, here it is:

// Package main renders an image, gif or video
package main

import (
	"github.com/bit101/bitlib/blmath"
	"github.com/bit101/bitlib/easing"
	"github.com/bit101/bitlib/geom"
	"github.com/bit101/bitlib/random"
	cairo "github.com/bit101/blcairo"
	"github.com/bit101/blcairo/render"
	"github.com/bit101/blcairo/target"
)

func main() {

	renderTarget := target.Video

	switch renderTarget {
	case target.Image:
		render.Image(800, 800, "out/out.png", scene1, 0.0)
		render.ViewImage("out/out.png")
		break

	case target.Video:
		program := render.NewProgram(400, 400, 30)
		program.AddSceneWithFrames(scene1, 60)
		program.AddSceneWithFrames(scene2, 60)
		program.AddSceneWithFrames(scene3, 60)
		program.AddSceneWithFrames(scene4, 60)
		program.AddSceneWithFrames(scene5, 60)
		program.AddSceneWithFrames(scene6, 60)
		program.AddSceneWithFrames(scene7, 60)
		program.AddSceneWithFrames(scene8, 60)
		program.AddSceneWithFrames(scene9, 60)
		program.AddSceneWithFrames(scene10, 60)
		program.RenderVideo("out/frames", "out/out.mp4")
		render.PlayVideo("out/out.mp4")
		break
	}
}

func setup() geom.PointList {
	random.Seed(0)
	return geom.PoissonDiskSampling(400, 400, 3, 30)
}

func draw(context *cairo.Context, points geom.PointList) {
	context.BlackOnWhite()
	context.Points(points, 1)
}

func scene1(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 0, 100)
	points.Translate(t, t)
	draw(context, points)
}

func scene2(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 1, 2)
	points.Scale(t, t)
	draw(context, points)
}

func scene3(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 1, 2)
	points.ScaleFrom(200, 200, t, t)
	draw(context, points)
}

func scene4(context *cairo.Context, width, height, percent float64) {
	points := setup()
	points.Rotate(blmath.LerpSin(percent, 0, 1))
	draw(context, points)
}

func scene5(context *cairo.Context, width, height, percent float64) {
	points := setup()
	points.RotateFrom(200, 200, blmath.LerpSin(percent, 0, 1))
	draw(context, points)
}

func scene6(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 0, 50)
	points.Randomize(t, t)
	draw(context, points)
}

func scene7(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := easing.LinearEase(percent, 0, 6)
	points.Noisify(0.005, 0.005, 0, t)
	draw(context, points)
}

func scene8(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 0, 0.5)
	points.Noisify(0.005, 0.005, t, 6)
	draw(context, points)
}

func scene9(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := blmath.LerpSin(percent, 0.005, 0.001)
	points.Noisify(t, t, 0, 6)
	draw(context, points)
}

func scene10(context *cairo.Context, width, height, percent float64) {
	points := setup()
	t := easing.LinearEase(percent, 6, 0)
	points.Noisify(0.005, 0.005, 0, t)
	draw(context, points)
}

Each “scene” just creates the point list, alters it in some way, and draws it. The alteration is just a couple lines of code. Pretty neat.

Don’t judge the code too harshly, it’s just a demo I threw together for testing these methods.

Photo by InstaWalli from Pexels:

Genuary2024, Days 13-18

kp@bit-101.com (Keith Peters) — Thu, 18 Jan 2024 06:00:38 -0500

The next six days of genuary 2024!

Day 13 - Wobbly function day

Might have to start this one manually. Set it to loop and turn up the sound a bit.

Your browser does not support the video tag.

I didn’t quite follow the spirit of the prompt, but the strings do wobble. I wanted to practice synchronizing video and sound. Tones created in Supercollider with a synthdef I made last year that sounds sort of like strings.

Day 14 - Less than 1kb artwork

Didn’t want to do size coding. Wanted to see the best Mandelbrot image I could make in 1K. Had to JPG the heck out of it after it rendered.

Day 15 - Use a physics library

Your browser does not support the video tag.

Not quite a physics library, but it’s my easing library that I wrote last year and haven’t really used. It works pretty well. Been using it more since this, so that’s cool.

Day 16 - Draw 10,000 of something

Your browser does not support the video tag.

10,000 strange attractors rendered with 10,000 points on a 10,000 pixel canvas. Frame rate is 1 fps so the movie takes 10,000 seconds (around 2 hours 46 minutes).

Here’s a 60 fps version that takes 2 minutes and 46 seconds.

Your browser does not support the video tag.

Day 17 - Inspired by Islamic art

Your browser does not support the video tag.

Not Islamic art. You’ll have to take my word that it was “inspired by”.

Day 18 - Bauhaus

Your browser does not support the video tag.

Call back to a business card I once had from a company I worked for.

Creative Challenges

kp@bit-101.com (Keith Peters) — Sun, 14 Jan 2024 09:20:17 -0500

There are many community-based creative projects that people can participate in. Some of these are competitions with strict rules. Others are just prompts to inspire you over a given period of time. Some are for coding, others for music, writing, drawing or … whatever.

As for the links, I tried to find some that were the most informative if not completely “official” in every case. I’m happy to update any of the below info or make additions.

Time	Name	Theme	Hashtag	Link
January	Genuary	generative art	#genuary	genuary.art
January	Jamuary	music	#jamuary	jonathanmann.net
Feb 9-11	Lovebyte Party	size coding	#lovebyte	lovebyte.party
March-April	Watt-Wise Game Jam	energy efficient gaming	#wattwisegamejam	wattwise.games
May	May for Makers	misc	#mayformakers	plsmithed.com
October	Mathtober	math	#mathtober	fractalkitty.com
October	Inktober	drawing (ink)	#inktober	inktober.com
November	NaNoWriMo	writing	#nanowrimo	nanowrimo.org
November	Codevember	games or general code	#codevember	codevember.org/
December	December Adventure	anything!	#decemberadventure	eli.li
Weekly	Disquiet Junto	Music	#disquietjunto	disquiet.com

Header photo by Jeffrey Czum

My Creative Workflow - Libraries and Frameworks

kp@bit-101.com (Keith Peters) — Fri, 12 Jan 2024 17:17:05 -0500

In this article, I want to talk about using libraries to make creative coding faster, easier and more enjoyable.

Background

For more years that I’d like to admit, I engaged in reinventing wheels. I’d re-write the same code over and over. If I wanted to find the distance between two points, instead of using a distance function, I’d just write (pseudocode):

dx = p0.x - p1.x
dy = p0.y - p1.y 
dist = sqrt(dx * dx + dy * dy)

Another example: most drawing APIs supply you with an arc function that’s something like this:

arc(x, y, radius, startAngle, endAngle, counterclockwise)

So every time I wanted to draw a circle, I’d type it all out rather than creating a circle function:

arc(100, 100, 50, 0, PI * 2, false)

I could probably come up with dozens of other examples. If I wanted to draw a path through an array of points, I’d loop through the array and draw a line to each successive point, when I should have created a path function. I’d done all these things so many times that I didn’t even have to think about them, the code would just fall out of my fingertips. So one of the plus points of this approach was that I learned a lot of concepts really, really well.

And sometimes I wanted to do something slightly different - it was easy enough to change the code right there because it wasn’t hidden away in a function.

Or maybe in addition to needing to know the distance between two points, I’d also need to know the angle between them. In that case, I could re-use the intermediate dx and dy variables in the atan2 function, building it a bit of optimization. Calling a distance function and then an angle function would calculate these values twice internally.

Another plus point was that I never needed any dependencies. In a call back to a previous article, I could just open an editor and start typing, without needing to pull in a library or framework.

But as time went on and projects got more complex, I had to come with the grips with the fact that I was wasting a lot of time writing the same code over and over. And sometimes I’d figure out some complex idea, but it wasn’t something I’d use very often. Then later I’d need to do that thing again and I’d either have to figure it out again, or dig up the project where I’d done it before.

So I created a library that I called bitlib and started dumping all my functions in there. I started this in ActionScript and started it again, probably multiple times in JavaScript. I still wasn’t a total convert to this workflow though. I’d use the library sometimes, but not others. It was never very well organized and I kept recreating a new version rather than trying to improve an existing one. I think it was some time in 2016 that I got a bit more serious about curating this library - in JavaScript at the time. Purely for reference: https://github.com/bit101/bitlibjs - this is not something I use anymore.

In 2017 I consolidated this library even more while working on a new version of the BIT-101 Lab. In the source code for that project, you can see multiple versions of bitlib in the libs folder. As I did new experiments, I was adding new functions to the lib. This was good. Right direction.

Rust and Go

As mentioned, in late 2017 I started exploring Rust. On January 1, 2018 I blogged about a port of bitlib to Rust - bitlib-rs. https://www.bit-101.com/2017/2018/01/introducing-bitlib-rs-for-rust-and-cairo/. This was designed to work with the Cairographics bindings I was using to generate images and animations in Rust. Most of this was a direct port of what I had in JavaScript at the time.

A short while later, when I switched over to Go, I ported it all to that language. There’s been a few iterations and reorganizations of the code, but it’s followed a very direct, highly curated lineage for the past six years and I’m very proud of it. And that’s what I want to spend the rest of this article talking about.

My library currently consists of two main code bases.

BLCAIRO

blcairo is the main graphics library. It creates an image surface and a drawing context and beyond the Cairographics drawing API, it has over 100 custom drawing functions added to the context. So to draw a circle, I don’t have to use Arc:

context.Arc(100, 100, 50, 0, math.Pi, false)

I can just say:

context.Circle(100, 100, 50)

Each drawing function also has stroke and fill versions, so rather than saying:

context.Circle(100, 100, 50)
context.Stroke()

I can just say:

context.StrokeCircle(100, 100, 50)

These were trivial to code, but have saved me countless hours - not only in typing the extra line, but in trying to figure out why nothing got drawn because I forgot to type the extra line.

In addition to the core drawing API and extensions, there’s a render package that has all kinds of stuff in it for creating projects that generate animations, sprite sheets, videos, animated gifs, montages, etc. from multiple images. I’ll take a deeper dive into that shortly.

BITLIB

The other key part of my core libraries is the Go version of bitlib. This consists of all kinds of utilities that do not directly have to do with rendering (at least via blcairo). To avoid circular dependencies, I make sure that blcairo can depend on bitlib, but not the other way around. The packages in bitlib have, over the years, grown to the following:

bitmap is an alternate rendering target. blcairo is great for drawing vector shapes such as lines, curves, rectangles and paths, etc. But sometimes I need to iterate through every pixel of an image and set its color. The Cairo context is not great for that. I’ve often just drawn a 1x1 rectangle for each pixel, but that seems like overkill. You can get the image data from the surface and do the math to set each element of that data to the specific red, green, blue and alpha values and then put the image data back into the surface and then save it. That also seems like overkill. So I wrote my own bitmap encoder which works on an array of RGB color values. It’s very lightweight and pretty fast. So for doing something like Mandelbrot or other fractals, it’s ideal.

blcolor is a color library that integrates wonderfully with blcairo. You can create RGB or RGBA color values with integers (0-255) for each channel, or floats (0.0-1.0) per channel, a single integer (0x000000-0xFFFFFF for RGB, but also an alpha version), HSV, HSL, CMYK, or grayscale. You can create random colors or interpolate between two colors to create another color. There are a dozen and a half methods to compare colors (equals, straight diff, perceptual diff), evaluate certain aspects of colors (luminance, contrast), convert from RGB to HSV, HSL, CMYK, grayscale) or alter colors (scale, quantize, add, rotate, invert). There are methods for almost 150 standard named colors (Turquoise, Alice Blue, Olive, etc.), and a package for creating, manipulating and using color palettes.

blmath contains a lot of useful math functions beyond what exists in the Go math module. A few dozen functions and constants, always growing. The ones I use the most are those that map a number within a certain range to another range, or clamp it to that range, wrap it around that range, map it exponentially, etc. There’s one that I use constantly that I’ll cover later.

easing is an implementation of Robert Penner’s easing formulas. This lets you say I want the value of this variable to move from x to y over a certain time period or number of frames. It might start out fast and ease into position at some rate, or ease out of its original position slowly, or do both. Or bounce or spring around a certain point, etc. These exact equations are used in many animation libraries. One thing I’d like to add is some sequencing, so I could set a chain of different motions into effect.

geom has all kinds of geometry based functions and types. Points, lines, segments, rectangles, triangles, paths, circles, etc. Ways to create these objects and manipulate them. There’s also some collision detection routines in there. It’s not fully fleshed out, but you can see if most of these types are colliding with the same type of some of the other types, and often get the point(s) of collision.

noise contains Perlin and Simplex noise functions - in one, two and three dimensions.

random is a PRNG (pseudo random number generator). This uses Go’s built-in rand module under the hood, but creates an much improved and more useful (IMO) API on top of it. You can get random floats or ints, or a float or an int within a certain range, or an array of floats or ints. You can get a random boolean or a weighted boolean. There are methods that use normal distributions or powers. You can also generate a random string of a given length with given types of characters. You can generate random numbers directly through the module, i.e. x = random.Float() which is fine most of the time, but if you’re running multiple routines (threads) they will share the same PRNG, which can mess you up. In that case you can create a random number generator for each routine and use that safely:

rng := random.NewRandom()
x = rng.Float()

The Render Package

Rendering Images

Moving back to blcairo, I want to talk about the render package because I’m pretty proud of some of the stuff in there. None of the stuff in this package is required to use to create images or animations, but it makes things far easier. Without this package, it’s still pretty easy to create a still image. It looks something like this:

func main() {
	surface := cairo.NewSurface(400, 400)
	context := cairo.NewContext(surface)
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(1, 0, 0) // red
	context.FillRectangle(100, 100, 200, 200)
	surface.WriteToPNG("out.png")
}

When you run that, it creates the following image:

Using the render package, you can change that to:

func main() {
	render.Image(400, 400, "out.png", renderFrame, 0)
	render.ViewImage("out.png")
}

func renderFrame(context *cairo.Context, width, height, percent float64) {
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(1, 0, 0) // red
	context.FillRectangle(100, 100, 200, 200)
}

Here, you can just call the render.Image function, passing in the size of the image, a name for the output, a function that has the actual rendering code and a percent value. Forget about that last param for a moment. Now the surface and context are created for you and your renderFrame function is called with the needed params. You do the drawing and that’s it. There’s even a function that calls an installed image viewr to view the final image immediately.

Now that might not seem like all that much of an improvement. Same number of lines of code, and we’ve added an extra function. But it becomes pretty freaking awesome when you move into animation.

Rendering Animations

Say you want to do a two-second animation at 30 frames per second. You have to render 60 separate images to disk (or maybe memory). And you have to figure out how to change each image on each frame to create the animation that you want. Then you have to take all of those generated frames and assemble them into an animated gif or a video or a spritesheet or whatever.

That can be dozens of lines of complicated code: math, file io, calling external tools.

Look how simple this becomes:

func main() {
	render.Frames(400, 400, 60, "./frames", renderFrame)
	render.ConvertToVideo("./frames", "out.mp4", 400, 400, 30, 2)
	render.PlayVideo("out.mp4")
}

func renderFrame(context *cairo.Context, width, height, percent float64) {
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(1, 0, 0) // red
	context.FillRectangle(100, 100, 200*percent, 200*percent)
}

Run this, and within a second, you see this video playing:

Your browser does not support the video tag.

First we call render.Frames which takes a width and height, the total number of frames to render (2 seconds times 30 fps), a directory to store the frame images and a function to call to render each frame. The frames will be created as files named frame_0000.png through frame_0059.png in this case.

Then we call render.ConvertToVideo. This takes the frames directory, the output file name, the size, fps and total time of the video in seconds. Yes, there’s some redundancy here, but hold that thought.

Finally, there’s a function that you can configure to play the video with your video player of choice.

Note that the signature of the renderFrame function has not changed. So we could continue to use this with render.Image. In practice, I often switch back and forth between the two, as I’ll mention later.

Anyway, renderFrame now makes use of the final percent parameter. Each of the 60 times that renderFrame is called, percent will be slightly bigger, going from 0.0 the first time it’s called (first frame) up to just under 1.0 for the final frame. Here, I’m using the percent value to change the size of the square I’m drawing. Pretty cool.

Perfect Loops!

But there’s a problem with this animation. It does not cleanly loop. The square expands and then jumps back to its original size. I like my animations to loop perfectly. So we go back to the blmath package and one of my favorite functions, blmath.LerpSin. This is a combination of “lerp” for “linear interpolation” and Sin as in a Sine function. It’s a bad name because it’s not linear at all, but I’m sticking with it. What it does is map the 0.0 to 1.0 of percent to a new range of values that will loop from a minimum to a maximum using a sine wave. So you’d do something like this:

func renderFrame(context *cairo.Context, width, height, percent float64) {
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(1, 0, 0) // red
	size := blmath.LerpSin(percent, 0, 200)
	context.FillRectangle(100, 100, size, size)
}

Here, we’re mapping percent onto values going from 0 to 200. We save this in a size variable and use that to draw the square.

Your browser does not support the video tag.

Less than a dozen lines of code and we have a perfectly looping animation.

Making it even better!

As mentioned, there’s still some excess wordiness and duplication of parameters in there. A more recent update solves a lot of this as well. We can refactor the main function into:

func main() {
	program := render.NewProgram(400, 400, 30)
	program.AddSceneWithSeconds(scene1, 2)
	program.RenderVideo("./frames", "out.mp4")
	render.PlayVideo("out.mp4")
}

This introduces the concept of a “program” which contains a number of “scenes”. We create a new program with a size and fps.

Then we add a single scene, giving it a render method (which I renamed to scene1) and a number of seconds. We could also use program.AddSceneWithFrames to specify the length of the scene in frames rather than seconds.

Finally we just tell the program to render a video, giving it a place to save the frames and a name for the output file. Much simpler.

And now, as you can probably guess, we can easily build up complex animations using multiple scenes. Here’s an example, with fewer than 20 lines of code for a two-scene animation.

func main() {
	program := render.NewProgram(400, 400, 30)
	program.AddSceneWithSeconds(scene1, 2)
	program.AddSceneWithSeconds(scene2, 2)
	program.RenderVideo("./frames", "out.mp4")
	render.PlayVideo("out.mp4")
}

func scene1(context *cairo.Context, width, height, percent float64) {
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(1, 0, 0) // red
	size := blmath.LerpSin(percent, 0, 200)
	context.FillRectangle(100, 100, size, size)
}

func scene2(context *cairo.Context, width, height, percent float64) {
	context.ClearRGB(0, 1, 0)     // green
	context.SetSourceRGB(0, 0, 1) // red
	size := blmath.LerpSin(percent, 0, 100)
	context.FillCircle(200, 200, size)
}

Your browser does not support the video tag.

Bringing it all together

A lot of the stuff I’ve talked about in the last few posts is now coming together. I’ve shown you the language and platform I’m using and the libraries and frameworks I’ve created. You’ve seen the enhanced Cairo drawing routines, the blmath stuff, and the power of the render package. Now let’s bring it full circle and show you the template that I use to start a new project:

package main

import (
	"github.com/bit101/bitlib/blmath"
	cairo "github.com/bit101/blcairo"
	"github.com/bit101/blcairo/render"
	"github.com/bit101/blcairo/target"
)

func main() {
	renderTarget := target.Gif

	switch renderTarget {
	case target.Image:
		render.Image(800, 800, "out/out.png", scene1, 0.0)
		render.ViewImage("out/out.png")
		break

	case target.Video:
		program := render.NewProgram(400, 400, 30)
		program.AddSceneWithFrames(scene1, 60)

		program.RenderVideo("out/frames", "out/out.mp4")
		render.PlayVideo("out/out.mp4")
		break
	}
}

func scene1(context *cairo.Context, width, height, percent float64) {
	context.BlackOnWhite()
	context.Save()
	context.TranslateCenter()
	context.DrawAxes(0.25)
	r := blmath.LerpSin(percent, 50, width/2)
	sphere := cairo.NewSphere(0, 0, r, 1, 0, 0)
	sphere.SetShadowColor(0.2, 0, 0)
	sphere.Draw(context)
	context.Restore()
}

It seems like a lot, but it’s mainly stuff we’ve already seen. At the start of main I just create a renderTarget variable. It starts out with the value target.Video. But it could also be target.Image, or target.Gif, target.SpriteSheet, target.Montage. Then it will call one of the blocks there, which either renders an image or video. If you want to render one of the other targets you can add a block for that. I can easily change that render target to change what it renders without changing any other code.

Sometimes I’m working on a video piece which takes several seconds to render a whole bunch of frames. But I’m just trying to get a basic form looking right. So I’ll switch over to rendering it as an image, which is much faster and allows me to iterate a lot quicker. Then when I have it down, I jump back to rendering the video.

The scene1 function just draws an animated red sphere - I showed that in a previous post. Usually when I create a new project using my tinpig tool, I’ll make the animation just to ensure it’s working correctly. Then I’ll clear out most of the scene1 code and start doing what I want.

One other bit of bling in the render package is when you go to render a video it gives you nice feedback about what’s going on. During render, you get a progress bar and an estimated time of completion:

(I upped the number of frames to 6000 to give myself time to do the screenshot!)

And when it’s done, it gives you a bunch of information about what it did.

Summary

It might sound like I’m giving you a sales pitch here. I’m honestly not. I don’t know if a single other person in the world is using these libraries. And that’s 100% fine with me. I made them for me and I love using them. In fact, if a lot of people used these, I’d have to be more worried about maintenance, avoiding breaking changes, dealing with feature requests, etc. So I’m happy being the main (or only) user.

My point is more that you should iterate on your own workflow until it makes you as happy as my workflow makes me.

It might also sound like I’m bragging a bit here. And sure, I definitely am. I’m pretty proud of what I’ve created here. It’s the culmination of many years of tiny improvements though. And culmination isn’t the best word either because I’m 100% sure that a year from now it will be even better.

Header Image 'Old books in Sarah's house' by lungstruck is licensed under CC BY 2.0

Genuary 2024, Days 7-12

kp@bit-101.com (Keith Peters) — Fri, 12 Jan 2024 05:00:00 -0500

Another six days of genuary 2024 are done! Here they are, with some random ramblings.

Day 7 - Progress Bar

Your browser does not support the video tag.

I was thinking of a progress bar that was not just a line filling up from left to right, or a circle or arc doing the same. Went through a lot of iterations of this. Ended up on this progression from a simple boring left-to-right progress bar, to total chaos. It still gets the idea across though.

Day 8 - Chaotic System

Your browser does not support the video tag.

This is a pretty run-of-the-mill 2d strange attractor, but I just kind of made up the formula on my own. I usually have to look up a formula, but I just made some params and threw some sines and cosines at it, and tested some random values until I saw some nice attractors coming out of it and honed in on that range. Beyond that, nothing too special here.

Day 9 - ASCII

Your browser does not support the video tag.

Same formula as Day 8, but far fewer points, and I labeled each one and drew lines between successive generated points. Interesting effect I thought.

Day 10 - Hexagonal

Your browser does not support the video tag.

This started as something similar I’d done with a rectangular grid a while back - rotated and scaled shapes inside of each cell. But I added an offset to it as well which gives it an almost 3d feel.

Day 11 - In the style of Anni Albers (1899-1994)

Your browser does not support the video tag.

I was not familiar with Anni Albers. I looked over her work and was intrigued by a fabric piece called Intersecting. The basic concept here is borrowed from that. Each line occasionally jumps lanes and uses a dampened spring formula to settle into that lane.

Day 12 - Lava Lamp

Your browser does not support the video tag.

This grew out of a much simpler, much more boring piece I almost published as is. It was just multiple flat morphing blob shapes in a grid. But I realized I could do something similar to what I did with the hexagons in Day 10. Then added some colors and just tweened the hell out of every conceivable property.

My Creative Workflow - Language and Platform

kp@bit-101.com (Keith Peters) — Tue, 09 Jan 2024 14:08:08 -0500

The language and platform you use for creative coding is a huge deciding factor in how you will work, how much you will enjoy working, and even the kinds of work you will do. So it’s worth shopping around and coming up with something that lets you do the kind of things you want to do and makes it a good experience.

Beginnings - Flash and ActionScript

As readers of this blog will likely know, I started out my career in Flash. I’d wanted to do creative coding as far back as the mid-80s and had made some games and creative apps in BASIC on the Commodore Amiga, but never got beyond that until I discovered Flash around 1998. The workflow there just clicked for me and I was able to do all kinds of stuff that I only dreamed about before that. No, that’s not right. I was able to do far more than I had dreamed about.

What I loved about Flash was that it was initially a combination of coding and more traditional drawing and animation tools. You could draw something with the mouse or a tablet and animate it on the timeline, or you could animate it with code. Later versions of ActionScript gave us an actual drawing API where we could create shapes in code, and then animate them in code. At that point, for many of us, the Flash IDE became unnecessary and we moved on to code-centric tools like MTASC (Motion Twin ActionScript Compiler) and various code editors. Later there was FlexBuilder and a standalone SDK for creating ActionScript projects, and third party dev environments like FDT (Flash Development Tool).

What I liked about ActionScript was how easy it was to create art and animation. The drawing API was decent and a first class part of the tool. As was the display list and animation system. The enterFrame event let us code the timeline. The display list with nested sprites and movie clips made complex animation a piece of cake. Initially the language was loosley typed, but even when typing and classes were added, it remained very expressive. It never tripped you up with fussy syntax (though it did have its idiosyncracies). And compilation was pretty fast. Nearly instantaneous for simple things, and not bad for even more complex projects. The Flash Player was ubiquitous, so you could upload your content and the whole world could see it and interact with it.

Processing

When Flash’s days were numbered, Processing (or, as it was known a the time, “Proce55ing”) was an obvious replacement for many. It had a lot going for it, even back then. A very simple IDE, but you could also code using your own editor and tools. A super rich drawing an animation API, well beyond what Flash/ActionScript had. One drawback for me was that Processing was Java-based. And the output for the web was Java applets. These were not nearly as accessible as Flash movies. And by “not accessible” I simply mean that people could not access them because they didn’t have the Java plugin installed in their browser. But it’s great for installations or for generating movies, animations or graphics that would be used elsewhere.

Later, the JavaScript adaptation of Processing - p5.js brought much of the core functionality of Processing back to the web.

I used Processing off and on but it never made it to the top of my list of languages and platforms. The same with p5.js. I have a lot of respect for both. I just never went deep with them.

Both versions of this tool are very solid choices to this day for creative coders.

Cinder

Cinder was another platform that saw a lot of adoption by creative coders around the time of Flash’s demise. It was co-created by Robert Hodgin, a hugely influential artist in the Flash community at the time, initially known by his project, flight404, one of the original classic Flash experimental sites.

Cinder is C++ based, and was far more powerful than most other tools in use at the time, and came with a higher learning curve. It was, and probably still is, used for many professional projects.

Unity3D

Unity was another obvious choice for a lot of Flash developers looking for a new home. It was mainly targeted as a 3D tool, but eventually had capable 2D animation workflows as well. It had a serious IDE for creating visual content and animation as well as coding in C# and JavaScript (if I recall correctly). Towards the end of Flash’s life, 3D was a popular subject on the platform. There were multiple home grown 3D libraries like Papervision3D and Away3D, and eventually Adobe added native 3D to the tool. For the people who were into that, the switch to Unity was a no-brainer. And it remains a hugely popular platform for game development in particular.

JavaScript

Of course, for many of us, JavaScript was an obvious path. It was mostly the same language as ActionScript - based on ECMAScript - and with the release of HTML5’s Canvas and 2D drawing context, it had a very capable drawing api which was very similar to Flash’s. The requestAnimationFrame function was essentially the same as ActionScript’s enterFrame event. So transitioning from Flash to JS/HTML/Canvas was quite easy. Additionally, it had the same benefit of allowing you to upload your creation and having it be available to the world. This is the path I took.

As a side note, Canvas also eventually supported a 3D context using WebGL, so it supported native 3D as well.

Anyway, when I was starting out on this path in March 2011, I did a series of 30 blog posts - one per day - about my journey in learning JS. The first post is here. And while they aren’t indexed, you can still walk through them by clicking on the next posts links. The code formatting didn’t survive very well, but if you really want to read 13-year-old technical articles, you gotta expect some rough spots.

JavaScript/Canvas remained my weapon of choice for 6 or 7 years. And it’s still an occasional second place tool I reach for now and then.

What I like about JavaScript… again, very expressive. It’s not fussy and lets you do what you want. In fact, it probably gives you too much leeway. And has more idiosyncracies than ActionScript had. The drawing API for canvas is very capable and very extendable. Zero compile time. It’s interpolated. And you just upload and everyone can see and interact with your creation. You hardly need any tooling at all. A simple text editor will suffice. But you can install hard core editors and linters and code completion tools, etc. that will speed up your dev.

If you want, you can go with TypeScript, which gives you optional typing and a lot of other advanced language features. But this means you need Webpack or some other system for transpiling, which ups the complexity of your projects significantly.

Some of what I did not like so much about JavaScript:

While I’d done a lot of on-line interactive pieces for nearly two years, I found myself being a bit more interested in creating large, high-res images for various projects, and have wanted to do more with creating static images and even movies, combined with generated audio or music. The JS flow for this was not as good. Working through the browser to create an image or animation on a web page and then saving that to a file on the file system is a pretty horrible idea, but that’s what I was doing.

Also, while JS has made great strides, it is not the fastest platform out there, especially when you start thinking of creating hundreds, or even thousands of frames for a movie.

Finally, I’ve mentioned some of the idiosyncracies of JS. They really do get pretty crazy at times. Some very serious WTF moments with how JS interprets code. It’s all logical if you look deep enough. I guess. I mean, there are reasons why it does what it does. But those reasons don’t make much sense to the average human.

So in late 2017, I started looking around for my next language and platform.

Python

Python was an initial choice. Python has tons of libraries, and more than a few for drawing 2D graphics and doing animations, including pycairo. But while I’d done some work in Python and liked it well enough, I kind of wanted to learn a new language. So I skipped over this one at the time.

A couple of times later, as recently as late 2023, I thought about coming back to Python using pycairo. I ported some of my existing stuff to it, but found it painfully slow compared to what I’ve been used to over the last few years. So I again abandoned Python as a serious language for creative coding. Just my opinion.

Rust

In last 2017, two languages had caught my attention, Rust and Go. I decided to go for Rust first. I found some Rust bindings for CairoGraphics, which is a C-based graphics library - the same one pycairo is based on. I built out a capable graphics library and started doing some fun stuff using Rust. It’s a great language for creating apps and services, but I really struggled with it for doing any kind of creative coding. It’s extremely strict. It aims to prevent you from doing anything that could create a crash or memory issue. It just won’t compile until you do things the exact way Rust demands. Going back to my last post, it was extremely hard to stay in creative mode while fighting with the details of how Rust wanted things to be. It just wasn’t going to work for me. That said, others have done creative coding and even game programming in Rust. Power to them. It’s not for me.

Go

Click!

I dipped my toes into Go in early 2018 and haven’t really looked back since. Every once in a while I think about trying something else (see above about Python) and always come back to Go. I stuck with CairoGraphics using a library I found called go-cairo and started building my own library on top of that. For whatever reason, the language just clicked with me. Although strictly typed, it often doesn’t feel like it is. It gets out of your way and lets you code. The syntax is mostly easy to pick up. It’s FAST. It is a compiled language, but you can run it like a scripting language - go run main.go. This compiles your code to an executable in a temp folder and then runs it. The compilation is generally so fast that it feels instantaneous, like you’re just interpreting a file - with a very fast interpreter.

Cairo is pretty much a superset of HTML Canvas’s drawing API. Very capable and very extendable. The library I was using was not in active development or maintenance, and after submitting a few updates (which were merged in), I finally just forked it and made my own version which I maintain for myself at least - blcairo. Beyond just cairo bindings, it adds all kinds of custom drawing routines and an animation framework for creating still images, frame sequences, gifs, videos, sprite sheets, or image montages. Beyond that I created another library, bitlib, which has custom math, geometry, easing, random, color and noise modules. This is in constant development. Almost every project I do winds up with some update to one or both of these libraries. The two of them are my bread and butter and I’ll talk more about them in a future post (or two).

Beyond the language itself, what I like about my current workflow is that it’s code -> executable -> image or animation on disk. Liberating myself from the browser was one of the main things I was after in 2017 when I decided to move away from JS.

One of the things I really miss about Flash and JS though is that interactivity. I can generate and upload animated gifs or videos or images and people can see them, and it’s great. But sometimes I miss the days of making things people could play with and interact with and explore on their own, sometimes wasting hours on them. Maybe someday I’ll get back into that, but I feel like I’m on the right path for now.

Summary

As I said in the beginning of this post, finding the right language and platform is pretty important to your workflow and productivity and even your creative potential. Don’t just do what everyone else is doing. Explore and find out what works for you. Hopefully something will click.

Next up I’ll talk about another piece of my workflow, which I started to touch on here - libraries.

Header image 'Computer books' by simonov is licensed under CC BY-SA 2.0.

My Creative Workflow - Templates

kp@bit-101.com (Keith Peters) — Sun, 07 Jan 2024 12:59:33 -0500

Art and Code

Many years ago I did a conference talk - at a few different conferences - entitled “Art and Code”. A lot of it had to do with fostering a creative technical workflow. I discussed a bit about left/right brain modes. Although I don’t buy into that stuff quite as much as a lot of the pop psychology books and articles do, I think you can be in a creative mode or in an analytical mode.

When you’re forced into decisions about, say, where to create a project, what to name it, how to structure it, etc., when you have to create a bunch of different folders and files and list out dependencies, etc., you move into that analytical mode.

You have a spark of a great idea and you want to try it out in code. You’re in create mode. If it takes you five minutes to launch your editor and set everything up before you’re able to start coding it, you may be losing that spark you started out with. Sometimes just the thought of setting all that up is enough to make you say “forget it”.

The best possible situation is: click a button, start coding. This is close to how the original Flash IDE used to be back in the day. Start Flash, open the Actions Panel, write code. Hit control-enter and see your creation. The Processing IDE is much the same. Neither of these made you decide where to put your project or what to name it. None of them required any serious boilerplate project structures.

Since moving away from Flash, I’ve tried to replicate that fast-start project workflow in whatever technical platform I’m doing creative coding in. It’s a very different workflow from traditional app development, where it’s fine to spend 30 minutes or an hour setting up a project, because you’ll be working on it for weeks or months or maybe years. So most general dev tools do not cater to this kind of setup.

Project Maker

As we moved beyond Flash and Processing, projects have gotten more complex, with all kinds of structure and dependencies and configuration involved. There’s rarely just a blank slate that you can start from and generate an image or animation with a couple lines of code.

In this new world, I’ve found that project templates are a key tool to get coding as quickly as possible.

After I moved off of the Flash IDE, I worked with tools like Flash Develop or Flex Builder. As I recall, these both had some support for templates built in, and a way to create your own custom templates. But when I moved over to JavaScript and HTML for creative coding, I found that I was going to have to roll my own solution.

One of my first solutions for this was a plugin I made for Sublime Text back in 2012. ProjectMaker. It was taken over by someone else after I stopped using it, but I don’t know if any of the 36 forks are still active.

With the plugin, you just had to start Sublime, and select a project template. You still had to choose a name and location for your project, but after that it would build out the whole project based on whatever template you chose, and drop you into the main file, ready to code.

I used this a lot for JavaScript-based creative coding projects. My template had the HTML, JS and CSS files, and pulled in any dependancies I used. The main JS file was all set up for me to start drawing things on an HTML Canvas.

Tinpig

After I stopped using Sublime Text, I needed that functionality again, and didn’t want it tied to any particular editor. So I created tinpig. This has roughly the same functionality as Project Maker, but is a standalone app that can make projects for any platform, language or IDE. It’s bundled with a few sample templates, but it’s more powerful when you create your own. The templates can have variables, which you will provide values for when you create your project. It’s all interactive, so just running the command walks you through everything you need and the result is a ready to run project.

Here’s what it looks like:

Most of these are custom projects I’ve set up for myself. The top one, “BLCairo Project” is the one I use most often. When I hit enter on that, I get a few prompts I have to fill in:

And here’s a look at what that created.

It’s got a sample animation coded up, so I just have to go into the new project directory and type make and it creates an animation and immediately launches and plays it. This just lets me know that the project is set up correctly and that it runs and compiles and produces what it is supposed to.

Here’s a real time video of creating a new project from start to showing the rendered test animation.

Your browser does not support the video tag.

The template itself I’ll discuss in another post. Of course, having good templates is vital, but first of all you should set up some kind of project templating so you’re not thinking about all that when you have an idea and need to get it down in code.

Tinpig is created in node.js and is on npmjs as https://www.npmjs.com/package/tinpig. So you can install it with one line.

Can I do better?

I admit neither of these solutions are “hit a key and start typing”. Even with tinpig, you still have to choose where to create your project, give it a name and answer some other questions. But it does a LOT of the heavy lifting required on most current workflows, so I’m pretty happy with it.

I once had a system that would create a project in a predefined location with a timestamped name. This is roughly what Processing does. That was really great for near instantaneous coding ramp up, but I wound up with a bunch of project folders that gave no clue what was in them. It was good for one-off ideation, but usually I find that I WANT to name something if it’s going to stick around for more than a few minutes.

I also had a tool that I created in Go that I was using for a while. It was pretty similar to tinpig, but specifically tailored for creating Go projects with my libraries. But more recently I returned to tinpig, cleaned it up a bit and really love it.

Resources

Some previous posts where I talked about these things:

I remember when I created Project Maker, there was discussion around a couple other similar tools that were popular. I don’t remember what they were and can’t currently find anything else that does something like this. If you know of something similar, let me know (see Mastodon links below) and I’ll be happy to add it here.

I’m also just curious what other people do for this. I’m hoping people are not manually creating complex projects from scratch every time. Do you have a templating system and custom project templates? Does the IDE or editor you use have templates built in? Do you just copy an old project and delete stuff in it and start again? Do you have a bare bones project that you copy and paste?

Update

I did find a couple of other potentially similar projects. I’m not familiar with them, but if you’re looking for a solution, these might fit your needs…

Offhand, I’d say these are both a lot heavier than tinpig and use the term “code generation” a lot. So I think these go beyond the goals of tinpig.

Header image by Nati

Genuary 2024, Days 1-6

kp@bit-101.com (Keith Peters) — Sat, 06 Jan 2024 05:00:00 -0500

Well, genuary 2024 is underway and I’ve made it to day 6! Here’s what I’ve done so far:

Day 1 - Particles, lots of them

Your browser does not support the video tag.

Particles are old friends of mine. But I tried to do something a bit different here. I’d added some good old Robert Penner style tweening functions to my graphics library some time last year. But I never got a chance to use them. This was an opportunity. Thousands of particles tween between four different arrangements, with some jittering going on as well. The square and circle are just distributions within those shapes. For the words, I created another surface and drew the text onto that. Then randomly generated points, keeping them only if they hit the pixels of the text. Now I had four sets of points, just had to tween from one to the next, and so on.

Day 2 - No Palettes

Your browser does not support the video tag.

I was trying to think of a way to get some strange color combinations by morphing between multiple points of defined colored values with other formulas going on. didn’t really get to where I wanted it, but it’s fine.

Day 3 - Droste effect

Your browser does not support the video tag.

A silly one. A rare case of me actually using photographs in my work - a selfie even! OMG. A couple of things I could have done better on this. One is that I should have started with a higher res source image so I could start the first sub-image smaller and have it scale up to full size, rather than having it overscale and then snap into focus when it loops. The other one is to have smoother scaling. It’s smooth mathematically, but not perceptually. It’s a known formula when zooming in on, say, fractals. I’ve done it before but got lazy here.

Day 4 - Pixels

Your browser does not support the video tag.

An old technique that I love. Sample each pixel of a source image and draw it to a destination image which gets displayed. Before you sample, you apply some kind of distortion to the coordinate you’re about to sample, but you draw it to the undistorted position in the destination. In this case, the distortion is some Simplex noise with varying parameters.

Day 5 - In the style of Vera Molnar

Your browser does not support the video tag.

At some point in the last 5-10 years it became very trendy for generative artists to name Vera Molnar as an influence. OK. Nothing against her. Maybe something against trends. I liked this piece though, so I played around with some ideas and it turned into what I created here. A line of points is drawn at the top of the screen. The points are moved down a bit and randomized a bit. Repeat to the bottom of the screen. Intensify the randomization over time.

Day 6 - Screensaver

Your browser does not support the video tag.

Flying toasters was too obvious. I made a screen. And tried to save it. Perlin noise on top of Simplex noise. (Or vice versa? I forget.) And some cheesy skeumorphism. I do kind of like the screen reflection. :)

Creating a Hugo Theme Part 4

kp@bit-101.com (Keith Peters) — Tue, 02 Jan 2024 06:00:00 -0500

Posts in this series:

Part 1
Part 2
Part 3
Part 4

We’ve got a functional site now and we can add all the content we want to it and it should pretty much show up when and where it’s supposed to. Now we just need to make it look good.

Now styling a web site can be a very personal thing. And let’s face it, that’s where all the fun is. So I’m not going to build out some in depth style system for you to copy over. I’m just going to go through the mechanics and give a few examples and turn it over to you to build out.

I’m also going to use plain old vanilla CSS. It’s not that I’m against frameworks or anything. I’m just going to leave that choice up to you.

But before we even touch CSS, there’s one thing I want to take care of with code.

Date formatting

Right now, all of our dates look like this:

Not great. We can do better. Hugo has built in date formatting tools. Let’s use them.

Starting with single.html, which looks like this now:

{{ define "main" }}
<h2>{{ .Title }}</h2>
<p>{{ .PublishDate }}</p>
{{ if .Params.Tags }}
  Tags: {{ partial "tags.html" . -}}
{{ end }}
<div>{{ .Content }}</div>
{{ end }}

We can set a date format right there, like so:

{{ define "main" }}
<h2>{{ .Title }}</h2>
<p>{{ time.Format "January 2, 2006" .PublishDate }}</p>
{{ if .Params.Tags }}
  Tags: {{ partial "tags.html" . -}}
{{ end }}
<div>{{ .Content }}</div>
{{ end }}

Here, we call the time.Format function, passing in the string “January 2, 2006”, and then the date we want to format. Why “January 2, 2006”? The date format library takes specific days and months and years, etc. to serve as an example formula. It’s designed to be more intuitive than arcane time formatting strings like "%B %d, %Y". Here are the formatting components you can use:

You can read more about it in the Hugo docs

Now the dates should look like this:

Try other date format strings like “Jan 2, 2006” or “1/2/06” and choose a format that you like. You might even want to include the exact time a post was published.

And don’t forget to do the same thing in the listitem.html partial.

<li>
  <h3>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </h3>
  {{ if .PublishDate }}
    <p>{{ time.Format "January 2, 2006" .PublishDate }}</p>
  {{ end }}
  {{ if .Params.Tags }}
    <p>Tags: {{ partial "tags.html" . -}}</p>
  {{ end }}
  {{ if .Summary }}
    <p>Summary: {{ .Summary }}</p>
  {{ end }}
</li>

So your posts list will have the right format too:

As with many things in programming, there are multiple ways to do this. Here’s an alternate, even simpler syntax:

<p>{{ .PublishDate.Format "January 2, 2006" }}</p>

You might even want to go all in with the HTML <time> element. You can read about the time element here.

OK, enough about dates, onto styles!

CSS styles

As I said, I’m not going to do a deep CSS dive. I’ll just point you in the direction and send you off to do your magic.

Generally, you want to put your CSS in a folder called static at the top level of your theme. Anything that’s in static will show up in the root of your site. so if you have something like static/styles/style.css, you’ll find that at localhost:1313/styles/style.css.

Within static you might also want to put a js folder if there’s going to be JavaScript on your site, and maybe images for any theme-level images or icons you want to use. Anyway, at this point, create the folders and file for static/styles/style.css. Your theme should look something like this now (I also added js and images):

Now we need to make sure our styles load into every page we create. If you guessed we’ll be doing something in baseof.html, you are close. If you guessed the head.html partial, you are spot on. Just link your stylesheet there:

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <link rel="stylesheet" type="text/css" href="{{ absURL `styles/style.css` }}">
  <title>My site</title>
</head>

I used the absURL function to make sure the link is correctly referenced even if the site is not at the root of the web server. You can read more about absURL here.

Now we are ready to make some styles! The one thing that bugs me the most about the site now is the default bulleted lists. Let’s get rid of the bullets and just put a border around those elements to set them off.

In listitem.html, let’s add a list-item class to the list item element.

<li class="list-item">
  <h3>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </h3>
  {{ if .PublishDate }}
    <p>{{ time.Format "January 2, 2006" .PublishDate }}</p>
  {{ end }}
  {{ if .Params.Tags }}
    <p>Tags: {{ partial "tags.html" . -}}</p>
  {{ end }}
  {{ if .Summary }}
    <p>Summary: {{ .Summary }}</p>
  {{ end }}
</li>

Then in styles.css create that style.

.list-item {
  list-style: none;
  border: 1px solid black;
  border-radius: 10px;
  margin: 10px;
  padding: 0 5px;
}

That gives us:

Here’s some more styles I added:

html, body {
  padding: 0;
  margin: 0;
  font-family: sans-serif;
}

nav {
  background-color: #ccf;
  padding: 20px;
  font-size: 24px;
}

This removes any margin and padding from the page, sets a slightly better default font and styles the nav. Now we look like this:

It’s not amazing, but hopefully you agree it’s going in the right direction. Not bad for a few very simple lines of CSS. I know you can do better.

Summary

Well, that wraps up this series. If you learned half as much reading it as I did writing it, then it’s all good. If you’re just building a single site, it’s probably fine to just leave your theme as part of your whole site. If you wind up making a theme you want to re-use or share, it’s worth pulling it out into it’s own git repo and then adding it to your site using git submodules.

I’d also strongly suggest (now that you know how themes work) that you create your themes using the hugo new theme <theme-name> command line tool. This will create all the necessary folders and some of the boilerplate files, as well as some theme configuration files that we just kind of ignored here. This will be especially important if you start sharing or re-using your themes.

Header Image 'Binary code' by Christiaan Colen is licensed under CC BY-SA 2.0

Creating a Hugo Theme Part 3

kp@bit-101.com (Keith Peters) — Mon, 01 Jan 2024 06:00:00 -0500

Posts in this series:

Part 1
Part 2
Part 3
Part 4

What we’ve done so far:

Got a home page working (a bit empty, but it’s there)
Got list pages and single pages working and showing what they should
Got a base template working and pulling in all of the above
Enabled navigation between all of the parts of the site, including a persistent navigation at the top of each page.

Today’s goal is to flesh each of these things out a bit more, and work in a few more pieces of the theme for better organization.

Making a custom home page

Right now our home page is pretty bland. In your final site, you probably want it to talk about your site or yourself or your specific project. But all of that is content and shouldn’t really be in the theme. So what do you do? One strategy is to make a new home page outside of the theme that will override the theme home page.

In your project directory, you’ll notice that Hugo created a few other folders, including a top level layouts folder. This folder mirrors the theme’s layout folder. And anything you put in here will override whatever is in the theme.

Create a new index.html file inside the top level layouts folder. In that, define “main” like we did in the them home template. Within that, you can create any site-specific content you want. Like so:

{{ define "main" }}
<h2>Hello from Keith's new site!</h2>
<p>In here will be all kinds of personal stuff that won't be in the theme.</p>
{{ end }}

Now instead of using the theme’s home page template, your site will uses this custom one.

Partials

Partials are like importable chunks of HTML. You can think of them as components that you build the rest of your site with. They’re very useful when you need the same chunk of HTML in multiple places, but it can also just help in site organization to have different partials handle specific sections of HTML.

Let’s look at our current baseof.html file:

<!DOCTYPE html>
<html>
  <head>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>My site</title>
  </head>
  <body>
    <nav>
      <a href="/">Home</a>
      <a href="/posts">Posts</a>
      <a href="/tags">Tags</a>
    </nav>
    {{ block "main" . }}
    {{ end }}
  </body>
</html>

We have three main things going on there.

The head section
The nav
The main block

Let’s add a footer as well, just to make it even busier.

<!DOCTYPE html>
<html>
  <head>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>My site</title>
  </head>
  <body>
    <nav>
      <a href="/">Home</a>
      <a href="/posts">Posts</a>
      <a href="/tags">Tags</a>
    </nav>
    {{ block "main" . }}
    {{ end }}
  </body>
  <div>This is a footer</div>
</html>

We can take these sections and split them into individual partial files. Actually, the main section is already split out, so we’ll just do the head, nav and footer.

In your theme’s layout folder, create a new partials folder. In that, create head.html, header.html, footer.html.

Then take everything in the <head> section of baseof.html and move it to head.html:

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>My site</title>
</head>

Take the <nav> section and move that to header.html:

<nav>
  <a href="/">Home</a>
  <a href="/posts">Posts</a>
  <a href="/tags">Tags</a>
</nav>

And finally, move the footer <div> to footer.html:

<div>This is a footer</div>

This leaves our baseof.html file nearly empty:

<!DOCTYPE html>
<html>
  <body>
    {{ block "main" . }}
    {{ end }}
  </body>
</html>

But we can now pull in those external sections as partials. It looks like this:

<!DOCTYPE html>
<html>
  {{- partial "head.html" . -}}
  <body>
    {{- partial "header.html" . -}}
    {{- block "main" . }}{{- end }}
    {{- partial "footer.html" . -}}
  </body>
</html>

You just specify the name of the partial file you want to insert. You don’t need the path, Hugo knows to look in the partials folder. And don’t forget the dot (.) to pass in the site context.

You’ll also note that I threw some hyphens (-) in there. These serve to strip out any extra white space as you’re importing blocks or partials.

This is nice because our baseof.html file is now almost 100% pure template, pulling its content from other places. It’s all about the structure of the site. You can see that it all works fine, and we now have a footer on each page as well.

If you were to use Hugo’s automation to create a new theme (hugo new theme newtheme), the baseof.html file it creates looks very similar:

<!DOCTYPE html>
<html>
    {{- partial "head.html" . -}}
    <body>
        {{- partial "header.html" . -}}
        <div id="content">
        {{- block "main" . }}{{- end }}
        </div>
        {{- partial "footer.html" . -}}
    </body>
</html>

The <div id="content"> surrounding the main block is a nice idea. I’ll update my baseof.html to match that. So we’ve come to the same place as the automated tool, but step by step. Hopefully you understand exactly what’s going on here more than you would have if you just let the tool just do it for you.

Improving the list page

As it stands, the list page just shows a simple bulleted list of page titles. We can do better. Here’s what it looks like now:

{{ define "main" }}
<ul>
{{ range .Pages.ByPublishDate.Reverse }}
  <li>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </li>
{{ end }}
</ul>
{{ end }}

We’ll be adding stuff to the <li> list item section, which is already getting big. Good place for another partial. Move the <li> section into a new file, layouts/partials/listitem.html and replace it with a call to that partial:

{{ define "main" }}
<ul>
{{ range .Pages.ByPublishDate.Reverse }}
  {{- partial "listitem.html" . -}}
{{ end }}
</ul>
{{ end }}

Make sure everything still works.

Now let’s flesh out that list item. I’m just going to throw a bunch of code at you and then walk through it.

<li>
  <h3>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </h3>
  <p>Date: {{ .PublishDate }}</p>
  <p>Tags: {{ partial "tags.html" . -}}</p>
  <p>Summary: {{ .Summary }}</p>
</li>

Here, we keep the <li> tag and put the title in an <h3> tag.

Then we add the publish date in a <p> tag.

Another <p> tag holds the list of tags. This one gets more complex, so I put it in yet another partial. Partials within partials. Partialception! We’ll look at that in a moment.

A final <p> tag holds the summary, which can be one of three things:

The text up to a  tag.
A summary item in the front matter of the page itself.
The first 70 words of the post itself.

And I believe Hugo will look for those things in that order. So if you have a more tag, it will ignore the summary front matter.

Let’s look at that tags.html partial next.

{{ range .Params.tags }}
  <a href="{{ add (absURL `tags/`) (.) }}">{{ . }}</a>
{{ end }}

Here we range over the tags and create an anchor tag for each one. The code for getting the path is a bit involved. We want to get the absolute path of “tags/” first. And then add the tag itself (represented by .) to that. As I said at the start of this series, I’m not going to dive too deeply into Hugo functions and variables.

One thing to note here is that since we have nested quotes here, we need to use a different type of quotes to surround “tags/”. Normally I’d use double quotes around the larger block and single quotes for the inner quoted material.

"foo 'bar' baz"

But Hugo doesn’t like that, so you have to use backticks in this case:

"foo `bar` baz"

And with all that, here’s what our post list looks like (truncated):

The usual caveat that it doesn’t look so good yet, but all the pieces are in the right places, which is all we’re going for here.

But remember, the same list template is used for all lists. Including tags. What happens when we look at the tags list?

Ewwwww… not good.

Since tags don’t have publish dates, or other tags, or summaries, it doesn’t make sense to try to show them. You run into a similar problem if you have a post without any tags. I removed the tags from post five and this is what the list looks like:

It doesn’t make sense to show “Tags:” when there are no tags. One way around all of this is just to check if there is any content to display before displaying it. I updated the listitem.html partial to look like this:

<li>
  <h3>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </h3>
  {{ if .PublishDate }}
    <p>Date: {{ .PublishDate }}</p>
  {{ end }}
  {{ if .Params.Tags }}
    <p>Tags: {{ partial "tags.html" . -}}</p>
  {{ end }}
  {{ if .Summary }}
    <p>Summary: {{ .Summary }}</p>
  {{ end }}
</li>

This cleans things up nicely.

Looking back at the single.html file, it would be good to show tags here as well. That’s pretty easy to add now that we were smart enough to break out tags into a partial.

{{ define "main" }}
<h2>{{ .Title }}</h2>
<p>{{ .PublishDate }}</p>
{{ if .Params.Tags }}
  Tags: {{ partial "tags.html" . -}}
{{ end }}
<div>{{ .Content }}</div>
{{ end }}

And now a single page looks like this:

You might be thinking that both the single and list pages show the date and then the tags. And this might be a good place for another partial, maybe a “postmetadata” partial that displays date and tags. I’ll leave that for you to create if you think it makes sense.

Summary

We’re looking pretty good here now!

OK, actually, it looks like crap. But again, everything is where it’s supposed to be and working as expected. Structure and functionality are nailed down. In the final post of this series, we’ll look at adding some CSS and images to make something worth looking at.

Header Image 'Binary code' by Christiaan Colen is licensed under CC BY-SA 2.0

Running Recap

kp@bit-101.com (Keith Peters) — Sun, 31 Dec 2023 13:02:23 -0500

A personal one this time. Back in 2009 I started running pretty seriously. I’d run sporadically for some time, but for the previous ten years or so I’d been pretty lazy and had gotten pretty heavy. I lost some weight in 2008 and in 2009 started doing the “Couch to 5K” running program. It stuck and over the next few years I ran a bunch of 5K races, two half marathons, a 20 mile race and three full marathons. I was actually training for an 50K (31 mile) ultramarathon when I just totally burned out on running. I remember the last big training run I did a that time. I ran 3 miles to a park, then 4 miles on the trails there, turned around and ran back home, refilled my bottles and snacks, then headed out in the opposite direction for 5 miles, then home again. Twenty four miles total. I was done for a while.

In the years since then, I’ve run on and off. I’ve gotten some decent streaks going, but never kept at it. In April of this year, I started it up again, and for some reason, it just all fell back into place. I’ve run three 5Ks this fall, and I’ve been pretty happy with my times. Haven’t beat my previous 5K PR, but getting close.

I rounded out the year with 530 miles. Not too shabby for starting in April. December was my best month with over 83 miles. My longest runs were 10 miles this year, but I got in four of those.

Health wise, I think I’m doing pretty well. In 2008 I peaked at 235 pounds. During my last bout of serious running I held steady in the low 180s mostly. In the years not running, I was generally around 190-200. These days I’ve been holding steady at around 173-174. As a benchmark, I remember being 175 in high school. FWIW, my BMI says I’m not overweight, but I’m kind of in the upper ranges of healthy weight. I wouldn’t mind getting into the 160s.

For 2024, mostly more of the same. Keep up steady mileage, see if I can take a few more seconds off my 5K time, and I’m considering the idea of a fall half marathon. I’d like to do 1000 miles next year, but that means doing 83 miles every single month. I’ve done it before, but I’ll be happy with 800-900. If I get close near the end of the year, maybe I’ll shoot for it.

One thing that’s been really helpful and supportive is that a bunch of my coworkers run as well. When I was running earlier, I was virtually the only one of everyone I worked with or hung out with that ran. I made a few friends on a site, dailymile.com (sadly now long gone). It was one of the best social networks I have ever been a part of. These days most of my coworkers, as well as other friends are on Strava. It’s a similar vibe, though dailymile was a bit more homegrown and cozy where Strava is super slick and overly sponsored. But it’s still a great resource. If you’re a runner or whatever, hit me up on Strava.

Gear

Shoes: Brooks Ghost 175

I’ve been using Brooks for a long time. Mostly Adrenaline GTXs. Decided to switch it up for something lighter. Only had them for a week or so, but really like them!

Watch: Garmin Forerunner 255

This watch is amazing. I’ve had Garmins from 2009 on. Probably five different models. My previous running watch was a Forerunner 235. Also really good, but the 255 has some amazing new features I couldn’t pass up. I wound up giving my 235 to a coworker who just ran her first half marathon with it recently. She was running with an Apple watch that would crash every time it rained or got wet.

Creating a Hugo Theme Part 2

kp@bit-101.com (Keith Peters) — Sun, 31 Dec 2023 06:00:00 -0500

Posts in this series:

Part 1
Part 2
Part 3
Part 4

In Part 1 we got the basic functional parts of a Hugo theme in place. It didn’t pull in any actual content or style anything, but it was enough to verify that everything was working like it should. Now let’s start pulling in your own content.

Throughout this tutorial, remember that anything in your theme folder should ideally be generic. You might want to use the same theme on a different site for example. But… nothing is stopping you from building a single-use theme that’s very customized for just your site. You do you.

Fleshing out the single page template

Let’s start with the single page template in layouts/_default/single.html. Currently it looks like this:

{{ define "main" }}
this is a single page.
{{ end }}

Rather than displaying static text, we want to pull the content from the page/post/whatever that we’re displaying. This is simple. We can just use the built in Hugo page method, .Content! This returns the content of the current page, converted to HTML and inserts it into the template when it is rendered.

{{ define "main" }}
{{ .Content }}
{{ end }}

While we’re changing up the code, let’s go into the baseof.html file and remove this line:

<p>this is the base template</p>

That was just in there to show it was functioning as expected.

Now, in your browser go to localhost:1313/posts/first and you should see your content!

We can add more to this template. We probably want the title and the publish date at least. And we can start to mark up this content with relevant HTML.

{{ define "main" }}
<h2>{{ .Title }}</h2>
<p>{{ .PublishDate }}</p>
<div>{{ .Content }}</div>
{{ end }}

Yeah, now we have something that’s starting to look a bit like a blog!

We could do more here, but let’s leave it like that for a while and move onto the list template.

Fleshing out the list template

The list template is used any time Hugo needs to display a list of items as the content of the page. As we have seen, that initially includes lists of pages, tags, or categories.

To display a list, we need to get the list of things we want to display, then mark up those items with some HTML to display them. As you might expect, this is going to involve looping through some content and doing the same thing to each item it finds.

Go ahead and edit the list.html template file and remove the placeholder text there.

This is about the bare minimum of a list template:

{{ define "main" }}
{{ range .Pages }}
  {{ .Title }}
{{ end }}
{{ end }}

The .Pages method returns a list of pages on the site, and the range function ranges through these. For each one, it just outputs the title of that page with the .Title method.

When that’s in place navigate to localhost:1313/posts to see a list of your posts. For now, we only have one, but it’s showing up there!

At this point, you should go in and make a few more posts. Call them whatever you want and put whatever text in them that makes you happy. When you’re done, you should see something more like this:

Well, they’re showing up, but you probably don’t want them all on the same line. And while they are showing up in order, we never specified that explicitly. We should do so. In your list template, make an unordered list element, and put each title in a list item.

For the ordering, we can use the .ByPublishDate to sort them. Usually you want to display the most recent post at the top. So you can add on the .Reverse function at the end. In the end it should look like this:

{{ define "main" }}
<ul>
{{ range .Pages.ByPublishDate.Reverse }}
  <li>{{ .Title }}</li>
{{ end }}
</ul>
{{ end }}

Now we have something that looks like an index of posts.

Don’t worry about the styling yet. Just focus on getting the structure right.

Next, it would be lovely if we could click on one of these items and it would navigate right to that post. We can do that with an anchor tag. There are different methods of getting the URL of a page, but the .RelPermalink method does the job nicely.

{{ define "main" }}
<ul>
{{ range .Pages.ByPublishDate.Reverse }}
  <li>
    <a href="{{ .RelPermalink }}">
      {{ .Title }}
    </a>
  </li>
{{ end }}
</ul>
{{ end }}

Now we have links!

And each link will take us right to the correct post! Fantastic.

Just to show you how powerful this list template is, let’s see it working on tags. We don’t even have to write any more code to do that. It’s ready! We just need to add some tags. I put some random “foo”, “bar” and “baz” tags on my posts, like so:

---
title: "First Post"
date: 2023-12-29T12:01:19-05:00
draft: true
tags: ["foo", "bar"]
---

This is my very first post

When you’ve done that, go to localhost:1313/tags and behold your list of tags:

Click on any one of them, like “baz” and it will direct you to localhost:1313/tags/baz and will then display a list of all the posts with that tag. Fantastic.

It would be nice to have a menu, say across the top of the site that let us get to different parts of the site, like the home page, the list of posts, the list of tags. This is surprisingly easy with basic HTML.

We’d like this navigation to appear at the top of every page, so let’s put it in baseof.html.

<!DOCTYPE html>
<html>
  <head>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>My site</title>
  </head>
  <body>
    <nav>
      <a href="/">Home</a>
      <a href="/posts">Posts</a>
      <a href="/tags">Tags</a>
    </nav>
    {{ block "main" . }}
    {{ end }}
  </body>
</html>

This simply uses the native nav element with links showing each section and the path to get there. Here’s what it looks like on the posts page:

I’ll also mention that you can create a more powerful and dynamic menu system in your Hugo config. Read the documention on menus. You’d still probably wind up putting some code in the same part of your baseof.html file. So this is still relevant - especially when we get into “partials” in the next article.

Summary

At this point, the site is basically functional. We can navigate all around it, go from any page to any other within a couple of clicks. Our home page works, our list page works on different types of lists, and our single pages work on every post.

This is probably a pretty good place to take a break. In Part 3 of this series we’ll flesh out some of these pages a little bit more, and we’ll wrap up in Part 4 with some styling.

Header Image 'Binary code' by Christiaan Colen is licensed under CC BY-SA 2.0

Creating a Hugo Theme Part 1

kp@bit-101.com (Keith Peters) — Sat, 30 Dec 2023 06:35:00 -0500

Posts in this series:

Part 1
Part 2
Part 3
Part 4

Obviously I’m pretty new to Hugo, so it may seem presumptuous to start writing a tutorial about it. But the best way to learn is to teach, so this is mostly for me. But if someone else benefits from this, all the better!

I’m going to assume that you’ve got Hugo installed, or can figure out how to get it installed. That might be as far as you got, and you’re wondering where to go from there.

Or maybe you’ve installed a pre-built theme and got that working, but now you want a bit more control.

Or maybe you’ve tried to create your own theme and got confused, like I did.

This post is mainly about making a custom theme, so I’m not going to go too deep into Hugo functions and variables and syntax. Of course I’ll have to use that stuff at points, but I’m going to assume that either you are familiar with it already, or that you’ll be able to look it up and figure it out.

I also want to mention that there is a command (hugo new theme mytheme) that will create all the necessary theme folders and some of the files for you. I don’t use that here, instead choosing to build things up piece by piece for a fuller understanding of what each piece does.

Create a site and first page

Go to the directory you want to create your site and type:

hugo new site mysite

Hugo will create a new directory in that folder and within that will create a bunch of other folders and files. cd into that folder and take a look around. You should see something like this:

Now make a new post or page:

hugo new posts/first.md

The posts part can be anything. If you’re building a blog and want to have posts, then posts makes sense. If you’re building another kind of site that has more of a page paradigm, then use pages/first.md. Again, it can be anything and it has no meaning other than the fact that it’s going to define your navigation and URL paths.

Now let’s edit your post. It should be in content/posts/first.md and will look like this:

---
title: "first"
date: 2023-12-29T11:43:54-05:00
draft: true
---

All that stuff is called “front matter”. It’s the metadata of the post or page. Feel free to edit the title and even the date. Keep draft as true for now. And add some content below the final ---. This is a markdown file, so if you’re familiar with markdown, you should be all set. Save it when you’re done.

Launching the server

Now you probably want to see your awesome site. So back in the command line, in the root of your new project type:

hugo server -DF --noHTTPCache

The first part runs the server which will serve up your new site locally in development mode. The -D will publish posts that are still in draft mode, and the F will publish posts that have dates in the future. You probably want to use these two options when you’re testing your site in dev mode. But when you’re deploying your live site, you’ll usually omit those.

The dev server you are now running is live updating, so your site will refresh whenever you save anything in the project. The --noHTTPCache parameter just helps to make sure that all changes are picked up and nothing is cached.

You probably want to run the server in a separate terminal or tab from where you’re doing your development so it stays running, but available so you can stop and restart it easily as needed.

In a browser, go to localhost:1313. Voila! … um… “Page Not Found”. The server is running, but it’s not finding the content. If you’re familiar with Hugo or web servers in general, you might try going to localhost:1313/posts or localhost:1313/posts/first. Same deal.

You may also have noticed some warnings when you started the server, saying it couldn’t find layout files and telling you you should make some templates. Well, that’s what we’re here for!

Creating the home page

There are a few basic templates that Hugo supports out of the box. The home page, a list page, and a single page. Let’s start with the home page.

In your project, you’ll see a themes folder. In that, create a folder for your theme. I’m going to name it mytheme but feel free to be more creative.

Inside that, create a layouts folder, and within that create an index.html file. For now just put some text in there, like this is the home page.

You might be tempted to fill this page up with some kind of standard HTML markup. Hold that thought. Keep it simple for now.

OK. We have the barest, most minimal, bare-minimal theme ever. Let’s tell Hugo to use this theme. Edit hugo.toml in the root of your project. And add a line to point to your theme. It should look like this:

baseURL = 'http://example.org/'
languageCode = 'en-us'
title = 'My New Hugo Site'
theme = 'mytheme'

Make sure everything is saved. Although the server mostly live-reloads, it often doesn’t when you make major file changes. So you might need to refresh your browser, or even stop and restart the server. You should wind up with your new home page successfully displaying, “this is the home page.” (or whatever you filled it with).

We’ll flesh this out later, but let’s build out the files for single pages and lists first.

Creating list and single page templates

While your home page “works” now, if you go to localhost:1313/posts or localhost:1313/posts/first, you’re still going to get “Page Not Found”. That’s because those paths are looking for a list template and a single page template. So let’s make them.

In your template’s layout folder, create a folder called _default. In that, create list.html and single.html files. Again, keep the content simple, something like, “this is a list page” and “this is a single page”.

You might need to restart the server, but now you should be able to go to localhost:1313/posts and see your list page, and go to localhost:1313/posts/first to see your single page.

We’re getting somewhere!

A few more things to note here:

You can also see the list page for localhost:1313/tags and localhost:1313/categories even though you didn’t define any tags or categories. Those are built in.
If you go to a list that doesn’t exist yet, such as localhost:1313/pages or a specific page that doesn’t exist like localhost:1313/posts/second, you’ll get “Page Not Found” as you should expect.

Creating a base template

So far, our HTML pages just consist of bits of raw text. You’re probably itching to create DOCTYPEs and meta tags and html, head, and body tags, etc. And we could do that in each of our three templates if we wanted. But we’d probably be duplicating a lot of stuff, and would need to change that in all three places every time we wanted to update it.

There are a couple of obvious strategies to handle this.

One is to extract the repeating sections and include those pieces in each of the three templates.

The other way is to create a base template and include each of the three page templates in the base. Hugo does the latter. The base template is called baseof.html and lives in the layouts/_default folder of your theme. Create that now.

Now we can fill in all the HTML goodness in this file. Something like this:

<!DOCTYPE html>
<html>
  <head>
      <meta charset="utf-8">
      <meta name="viewport" 
          content="width=device-width, initial-scale=1.0">
      <title>My site</title>
  </head>
  <body>
    <p>this is the base template</p>
  </body>
</html>

Feel free to add whatever meta tags, style and script links, etc. here, but try to keep the content generic. Remember, this template will be used as the base of every page in your site. Later we’ll put a header and footer in here, which makes sense.

And now we need to insert the content of our other three templates into this base template. To do that, we reference a block. Now we get into our first taste of Hugo syntax. Our block looks like this:

<!DOCTYPE html>
<html>
  <head>
      <meta charset="utf-8">
      <meta name="viewport"
          content="width=device-width, initial-scale=1.0">
      <title>My site</title>
  </head>
  <body>
    <p>this is the base template</p>
    {{ block "main" . }}
    {{ end }}
  </body>
</html>

We reference a block called main within double curly brackets, and pass in the dot . to it, which in Hugo terms is like passing “this”. Note that we also need to end this block with the end tag.

Now we need to define this main block in each our our templates. In each of index.html, single.html and list.html, surround the text with a block definition tag, like so:

{{ define "main" }}
this is a single page.
{{ end }}

Now, Hugo will always use baseof.html. When you’re on the home page, it will insert the main block from index.html into the base. When you’re on a list or single page, it will include the main block from one of those.

Again, with all these major changes, you’ll probably have to restart the server and refresh your browser, but now when you navigate to localhost:1313 or the paths /posts or /posts/first, you should see the line, “this is the base template” and the text from the individual template shown in the browser.

Note that non-existent paths will still correctly serve “Page Not Found”.

In case you’re having any issues, here’s what your project should look like at this point:

Summary

OK, that’s enough for one post. It might be a bit unsatisfying since we didn’t actually theme anything or even show our single post yet. We’ll get there. But perhaps the most important thing about understanding Hugo theming is knowing all of the components of a theme, what each one does, where they live, and how they work together. We’ve covered most of that here today. Next time we’ll start displaying actual content, and then work on a bit of styling.

Header Image 'Binary code' by Christiaan Colen is licensed under CC BY-SA 2.0

Community Creative Coding

kp@bit-101.com (Keith Peters) — Fri, 29 Dec 2023 06:00:00 -0500

I love it when a community comes together to do something positive and fun. An example is when creative people form some kind of hashtag on social media and post images, animations, or other art. Often these are tied to a particular day like #silentsunday. Other times they extend in time, with a different theme for each day.

genuary

The header image on this post refers to genuary, a community generative art project that provides a prompt for each day of January each year. You create something based on the prompt and post it to your social network of choice, with the given hashtag of the day. It’s very loose, with no hard and fast rules. Follow the prompts… or don’t. Do some of them, skip others. It’s all about creating stuff and sharing.

I plan to participate in genuary this year. I’m shooting for doing every day, and I’ll post here now and then, with a few days worth of creations each time.

Last year I did about a week’s worth, but got busy with other things, so didn’t finish. Here’s a couple I did then:

Day 6: Steal like an artist.

Your browser does not support the video tag.

Day 7: Sample a color palette from your favorite movie/album cover.

Your browser does not support the video tag.

7 Days of Code

Another one is 7 Days of Code (no longer live). This is a monthly coding challenge with prompts given for just the first seven days of each month. I think it’s been on hiatus for a while, but it’s back for January 2024. Maybe bad timing as I think a lot of the same people who do 7 days will also do genuary. Here’s some stuff I did for 7 days in October 2021. I don’t recall the prompts.

Others

I’ve actually set up some of these on a very impromptu basis in past years. Search that social media site with the bird icon for hashtags #GulyIsForJifs and #MayContainGifs. These were daily challenges to create an animated gif every day of July and May. No particular theme or prompt on those ones. There are still quite a few up on the aforementioned site. I also have a folder full of gifs tagged #Awegif2021 tag for August 2021, but nothing comes up in searches for that. Maybe I’ll do something like that later in the year on Mastodon.

Here’s one I did for those #Awegif2021

Anyway, I’m looking forward to genuary. Ping me if you’re participating!

Hugo

kp@bit-101.com (Keith Peters) — Thu, 28 Dec 2023 12:00:00 -0500

As I mentioned the other day, I decided on Hugo to create the new version of this blog. A few reasons for that.

It’s Go and I love Go.
I’ve used Hugo before and got along well with it.
It’s pretty easy to work with, compared to some other systems I’ve worked with or attempted to.

Theme Woes

One thing that’s not fantastic about Hugo is the theme system. Or more to the point, the documentation around it. To be fair, there’s a fair amount of documenation on the Hugo site, and the initial installation and quickstart tutorials are great, but after that it’s just reference material mostly.

I installed a few themes and some were fine, but there were little things I wanted to change about them, and it was not clear at all where or how to do that. To figure it out, I had to find some other resources.

Creating a Hugo Theme From Scratch

This post really helped a lot. Although it is just a “copy this code block into this file, and this one into this other file”, it does create a bare minimum setup to work from. From there I could see which bits of code controlled what and was able to piece things together. Starting with that as a base, I ripped out all the external dependencies and little by little replaced most of what was left. And I learned a lot along the way. I can’t say I’d recommend starting out by creating your own theme from scratch, but it worked for me and now I have complete understanding and control over how the site looks and functions, and will probably keep iterating on it for some time.

Hugo video tutorial series

This series was also very helpful. It’s dated, but almost all still relevant. Mike tends to over-explain simple points repeatedly, which is fine, but I found myself saying, “yeah, yeah, yeah, I got it!” multiple times. :) I threw it on 1.5x speed and watched all 23 of them and learned a lot more.

After I got some of those basics down, I was much more comfortable going into the official docs and understanding what they were talking about.

Some others resources listed on the External Resources page of the Hugo docs site. I haven’t gotten into any of these yet, but will check them out over time:

Big Oops

Confession time.

In Hugo, you write your content in markdown, which is then converted to HTML and merged with various theme templates, then published as a collection of static HTML pages, assets and CSS. There’s a development server which is great for testing locally as you’re writing or developing your theme. But when you’re ready to publish, you just build the site and upload it to any web server. A common way to do this is with rsync. So I was doing that to make sure it looked ok live on the web.

My previous site was at https://bit-101.com/blog (where this site is now). So I figured it would be safe to rsync the new site to the root of the web folder. And it would have been safe, but I was using the --delete option, which deletes any existing files in the destination that aren’t in the source. And there went my two previous sites, and a whole bunch of other stuff I had up there.

Thankfully, my provider had a backup so I was able to get everything restored. And I now have a local backup of all that stuff.

Onward

I have to say with the theming stuff out of the way, I really, really, REALLY like blogging in markdown in neovim. And I love running my editor on one half of the screen and the browser on the other half, with the dev server running and hot loading my changes whenever I save. I’m sure I will continue to iterate on the theming and on the functional aspects of the blog as well, but for now I’m pretty happy where it’s at.

Hello Again!

kp@bit-101.com (Keith Peters) — Thu, 28 Dec 2023 08:00:00 -0500

Yes, it was time for a refresh. Honestly, I just got pretty tired of Wordpress.

I hate to be that guy on the internet who complains about software, but I’ve really been struggling with Wordpress’s Gutenberg engine for a long time. Maybe it’s just me. I use vim (neovim), which generally means that I just like to edit text when I’m editing text. Gutenberg has this whole visual block paradigm with dropdowns and popups and controls and side menus to control what’s in each block and what it looks like.

Maybe some people like that paradigm, but it just gets in my way. It feels slow and from what I can see, it is super buggy. Things seem to work one way one time and a different way the next. Generally never how I expect it to work.

Well, maybe it’s buggy, maybe it’s me, but I find myself constantly fighting it and getting frustrated, and I’m kind of done with it.

Beyond the editing experience, Wordpress is a big, beefy system. You have a database to worry about, hundreds and hundreds of PHP files with functions, and spam filters, and plugins to optimize the whole thing to make up for all the rest of it.

I’ve been thinking about this a lot, and a static site just sounds wonderful.

So, I’m moving over to Hugo, which means I can blog from neovim. I can create posts using markdown and check them into github.

The old site will still be available as an archive at https://bit-101.com/2017. And the earlier site has been revived and lives at https://bit-101.com/2003.

Let’s see how this goes! I’m hoping with some reduced friction, I just might post a bit more often.

About

kp@bit-101.com (Keith Peters) — Thu, 28 Dec 2023 00:00:00 +0000

Welcome to BIT-101. My home on the web since 2001. Initially this started out as a Flash based experimental “lab”, where I did a Flash experiment almost every day for several years. Usually some kind of interactive animation using math or physics. It was a lot of fun.

The blog was started in 2003. Earlier versions of the blog are still available.

Other sites of mine you might want to check out:

Art from Code
- Not a lot of writing here. Just images and animations. This went through a similar move from WordPress to Hugo in 2024. The earlier site is linked there.
keithpeters.org
- This is a personal domain that I grabbed early on. It’s gone through a few iterations and spent some time being blank. Currently it’s got a Faircamp install with the music I’ve been creating.

Books

I think I’ve written or contributed to 16 published books. You can find more about them here: https://bit-101.com/blog/posts/2024-01-23/books/

The main one I would suggest now is Playing With Chaos, which I still crack open personally to remember how certain techniques are coded. It’s all about fractals, strange attractors, chaos theory, cellular automata. Chock full of code to copy and run and alter.

Flash

Here’s a screenshot of the original Flash site in 2001! When I finally stopped maintaining it in 2005, it had around 800 open source experiments.

In 2003, that site won the Experimental category at the Flash Film Festival at the FlashForward Conference in NYC. My award was a rubber arrow (the conference logo), which I still have today!

Programming Professionally

That was my big moment. I got home from New York, and the next day I got fired from my crappy job in the estimating department of a mechanical contracting company. Best thing that ever happened to me. Got a new job as a programmer within a week. Twenty years later, I’m still working in tech. These days I manage two engineering teams and oversee another manager on two other teams. I don’t write much code at work anymore, but that’s OK. I let the kids do all that stuff.

Programming Recreationally

I still write code all the time, mostly for fun or artistic purposes, or the occasional side project. I love learning new languages, techniques, new formulas to make interesting images or animations. That’s a lot of what you’ll find me writing about here.

Writing

The original Flash site also lead to me to getting my first authoring opportunity - writing a chapter for Flash Math Creativity. Such an amazing book! Nobody publishes books like this anymore. Someone should.

Over the next many years I wrote or contributed to at least sixteen programming books.

I wrote a bit more about the history of this site and some of my professional experiences in my 20 Years post on the earlier site.

Personal

I live outside of Boston, Massachusetts in the USA. While I’ve spent up to a few months in other locations, New England has always been my home. I grew up south of Boston, lived in Boston and Cambridge for many years, then north of Boston for a while. Now I’m out in the western suburbs for the past fifteen years. I’ve got four kids from my first marriage, all grown up and scattered across the country, and one daughter still at home, in junior year of college. My oldest daughter just made me a grandfather recently!

I don’t take many selfies, so this is your chance to get a look at me. From age 21, 42, and 59.

I just keep getting more handsome! I’m about 60 pounds lighter in the third picture than in the second one. A lot of that is due to running. I ran three marathons in my late 40s. Took some years off after that, but I’ve been back into it and logged over 500 miles in 2023. I might shoot for a half marathon in 2024.

Header image 'To US 101' by Oran Viriyincy is licensed under CC BY-SA 2.0

kp@bit-101.com (Keith Peters) — Mon, 01 Jan 0001 00:00:00 +0000