In a recent post I shared some thoughts about art and included a few, somewhat tongue-in-cheek comments about the use of random number generation. Specifically,

Random is evil. Random is lazy. Random is OK when you’re starting a new piece. It’s OK when you have a formula and you’re searching for an interesting range of input parameters. But once you find something interesting, lock in those parameters and start focusing.

Interestingly, this was the part of the post that generated the most comments – only a few, but it was definitely the thing that several people latched onto. So I thought I’d expand my thoughts on it.

Now of course I don’t actually think random is evil. But it can be lazy, and it can definitely be boring. But yes, I use random all the time. In my blgo library I have a random module with all kinds of useful functionality that derives from Golang’s built-in pseudo random number generator (PRNG). In my bljs library, I had to build my own PRNG because one doesn’t exist in JavaScript. No, Math.random is not a useful PRNG since you cannot seed it.

So what, in my opinion, is a good use of random and what is not?

Random positions and sizes

Not a great use of random. One of the most boring. I once said something like, “If you can draw a shape, you’re an artist. Randomly draw 10,000 of the same shape, you’re a generative artist.” But honestly I think that’s the ultimate in boring, lazy generative art. Here’s 10,000 random points:

And 500 random squares with random sizes:

And 500 random lines:

I could go on. And I could generate 10,000 individual “art works” like any one of these with a few lines of code. Each would be different. But I wouldn’t call it art. Maybe in my very, very early days of experimenting with graphic code I thought this looked kind of cool. But seriously, I hope your skills and self-expectation rise above this level really quickly.

A Different Random?

So what might be better? Taking it slowly, let’s just try a different random. If you’re using Math.random or something similar that generates a random number, you’re probably getting a uniform distribution. This means that there is an equal probability that you’ll hit any given number in a given range. So it’s all pretty flat and boring. You could try a normal distribution. This favors numbers in the center of your given range, giving you something like this:

For this, you’ll either need a PRNG that has a normal distribution function, or you’ll need to come up with one yourself. Here’s a JavaScript example I found on StackOverflow that seems to do the trick:

// Standard Normal variate using Box-Muller transform.
function randn_bm() {
    var u = 0, v = 0;
    while(u === 0) u = Math.random(); //Converting [0,1) to (0,1)
    while(v === 0) v = Math.random();
    return Math.sqrt( -2.0 * Math.log( u ) ) * Math.cos( 2.0 * Math.PI * v );
}

Unlike a uniform distribution generators however, normal distribution generators will usually give you a range of numbers both negative and positive. Most of the generated numbers (around 99.7%) should fall into the range of -3.0 to +3.0. So you’ll have to do some transformations to get them into the range you actually want them.

Here’s an example that places 100 squares in a normally distributed random position, and uses a normal distribution for their sizes as well:

In this case, I took the absolute value of a generated number (which would generally wind up in the range of 0 to 3, weighted heavily towards 0) and multiplied it by 40 to get the size of each square. This skews the square sizes toward the smaller range. Of course, you could do different math to skew it to larger squares, or even towards medium sized squares, with smaller and larger ones occurring less frequently.

And some random lines with normal distribution:

If you find normal distribution interesting, you might want to look at one of the many other probability distributions.

We’re still in the realm of random, but even so, I think the results are much more interesting.

Noise

Another strategy that is more interesting than uniform random distribution is using noise. This usually means Perlin or Simplex noise (which I’ve discussed here in depth). Here, I chose 10,000 random points, but only accepted them if the noise value at that location was above 0 (in a range of -1 to +1):

Things get more interesting when you throw size into the mix. This one does the same thing, but draws squares. The size of each square is in proportion to the value of the noise at that point.

Other

Other things to try are various types of attractor formulas, chaos games, iterated function systems (IFS) and other fractal-related formulas. These can often be just as chaotic and unpredictable as PRNGs, but at the same time give some kind of underlying structure that is much more interesting than uniformly distributed random numbers.

Random Colors

I know next to nothing about color theory, and tend to do things in black and white as long as I can get away with it. Big fan of monochrome. But now and then I’m forced to use color under extreme duress. My first naive attempts to fly under the radar with color usually go along with a function I created called randomRGB, which does what you would guess.

Here’s the above normally distributed squares (a few more of them this time) with random RGB color values.

Well, it IS colorful. But it evokes absolutely nothing. From there, I usually move over to HSV colors, which at least sends me in the right direction. Here, I’m using a random hue from 0 to 60 and a saturation and value of 1.0:

At least this has something going for it. A kind of pixelated, MineCraftian fire maybe. Or, we could keep a single hue and vary the saturation:

Or a random value:

These at least have something going for them. If you want to go all out, you can start creating color palettes. Adobe Color Wheel is a great resource. Here’s a split complementary palette I generated:

To be fair, I’m still just picking one of the five values from the palette randomly, so it’s still pretty lame. I refer back to my first statement of this section. I think it would probably be better to use the reds in one area and the blues in another and the green as a highlight… or something.

Random is Not Evil

As you can see, in just about all of my “non-random” examples, I’m still using random to some degree. But I’m using it secondarily, piping it through something else, which gives it some structure. Once you have some kind of predictable structure, then you start tweaking values to move that structure a bit this way or that way and start adding some of your own taste and judgement into what you are creating. Then it starts to become more of your work, not just the work of a PRNG.

Often, I will have some formula that generates some kind of fractal or attractor. It can produces quite different images depending on what parameters you give it. But that means there can be millions or even billions of different potential images (infinite really, but many of them will be very similar). So I’ll create a loop that just makes a bunch of thumbnails with random parameters and generate a grid of images. On each thumbnail, I’ll print the parameters that created that image.Here’s a sample from last year:

When I see one I like, I’ll hard code those parameters into the code and generate a larger version. Then I’ll tweak them one way or the other to create something that I really, really like. This use of random allows me to explore way more possibilities in a much shorter time than manually trying to find a good set of parameters.

Summary

I could probably go on about this subject a lot longer, but hopefully I’ve clarified my thoughts a little bit and maybe given you some ideas to try out.

Circle inversion is a mathematical technique that holds a lot of opportunities for the creative coder.

Circle inversion, or circular inversion, or geometric inversion is a method of transforming a point within a circle to a new point outside the circle or vice-versa. By transforming multiple points, you can transform entire shapes, which become warped and curved in interesting ways, as if the original shape was reflected onto a curved surface. You can even do a circle inversion chaos game with multiple random circles to create interesting fractals. We’ll dive into that after we cover the basics.

The All Important Formula

So we start with a circle centered at x0, y0 with a radius of r. And a point x1, y1 within that circle. We want to invert that point to x2, y2, outside of the circle. That’s the point we need to find.

To describe it a different way, we can easily get the distance from the center of the circle to point x1, y1. We’ll call that dist0. We want to get the distance from the center to x2, y2, which we’ll call dist1.

The formula is simply r^2 / dist0. That’s if the first point is inside the circle. If you want to transform a point outside the circle to a point inside it … well, actually, it’s the exact same formula, which is nice and handy.

I’m not going to go deep into the geometry and math that you use to work out that formula, but it’s a good idea to work it out if you’re into that kind of stuff. There are plenty of pages that will show you exactly what’s going on here.

And the code that will make it happen:

function invert(cx, cy, r, x1, y1) {
  const dx = x1 - cx;
  const dy = y1 - cy;
  const dist0 = Math.sqrt(dx * dx + dy * dy);
  const dist1 = r * r / dist0;
  const angle = Math.atan2(y1 - cy, x1 - cx);

  return {
    x: cx + Math.cos(angle) * dist1,
    y: cy + Math.sin(angle) * dist1,
  }
}

This takes the center x, y and radius of a circle and the x, y of another point. It calculates the distance from the point to the center using standard Pythagorean theorem, and finds dist1 using the formula I just described. It then finds the angle from the circle center to the point and uses some trig to find the new point. And here it is in action:

You can move the circle’s center, change its radius, and move point p0. Point p1 will then move in accordance. Things to note:

• As p0 gets close to the edge of the circle, p1 also moves in close to the edge of the circle.
• As p0 gets close to the center of the circle, p1 moves out to infinity.
• When p0 goes outside of the circle, p1 crosses over to the inside.

There’s a bit of an optimization you can make to this code to remove all the trig. I’ll just put it here and leave it to you to figure out how it’s working:

function invert(cx, cy, r, x1, y1) {
  const dx = x1 - cx;
  const dy = y1 - cy;
  const dist0 = Math.sqrt(dx * dx + dy * dy);
  const dist1 = r * r / dist0;
  const ratio = dist1 / dist0;

  return {
    x: cx + dx * ratio,
    y: cy + dy * ratio,
  };
}

In fact, you can go even one step further. Since we’re dividing by dist0 twice, we’re actually dividing by the square of dist0. So we can use that to get rid of the square root and both distances:

function invert(cx, cy, r, x1, y1) {
  const dx = x1 - cx;
  const dy = y1 - cy;
  const ratio = (r * r ) / (dx * dx + dy * dy);

  return {
    x: cx + dx * ratio,
    y: cy + dy * ratio,
  };
}

Shapes

Once you can invert a single point, you can start inverting entire paths of points, meaning you can invert shapes. To give you a feel for this, here’s a little app that lets you draw on the canvas with your mouse. It will draw both the exact path you are drawing, as well as the inverted path.

There’s no clear button, so you’ll have to refresh the whole page if you want to restart a drawing. If you want to spend some more time with it, you can open up the app in a fresh page.

I’m going to stop here for now. The core takeaway is that invert function. Play around with it and see what happens when you draw different shapes. I’ll come back later with that circle inversion chaos game concept I mentioned earlier. That’s where things get really fun.

Note on diagrams.

The drawings above were done on my Onyx Boox Note Air eInk tablet. I had talked about sketching in eInk in an earlier post. I was about to code up some graphics to show the principles, or maybe break out some kind of drawing application, then decided to give it a go just sketching on the tablet. I then exported the drawings and added them to the post. It’s also the first time I used color in sketching on eInk. Although the Note Air is not a color tablet, you can choose different colors to draw with. They render in black and white on the eInk screen, but when exported, they are indeed color drawings. Not super important to the post. It was just a fun little workflow that worked out pretty well from my viewpoint

Controls: MiniComps

Graphics: BLJS

Support this work

Source for the final piece.

One of my tricks when coming up with new generative ideas is to take two (or three) completely unrelated techniques and combine them. I even created an app once where I had a long list of concepts and it would randomly combine them and come up with things like, “circle packing with diffuse limited aggregation”. (I just made that up and I’m not sure how it would work, but it’s intriguing, isn’t it?) I can’t say that the app ever sparked any projects that I ended up using, but I’ll probably go back to it some time and see if I can make it better.

This post was one of those successful combinations that evolved on its own and produced something that I have never seen anyone explore before. It produces a visualization of the forces behind a strange attractor that I find fascinating and beautiful. It feels like putting on special glasses that let you see the invisible workings of the universe.

Palettes

In a recent post, I shared making animated gifs with ffmpeg. I hadn’t been doing it very long so I wasn’t sure how it would work in the long run. Lo and behold, I ran into a problem. I was making black and white (actually monochrome grayscale) gifs and for the most part it was going well. But then I saw some yellow getting in there!

None of the still images had anything but grayscale values. So how was I getting yellow? To be honest, I’m not sure of the details 100%, but it’s got to do with palettes. Gifs generally have just 256 available colors. There are tricks to make animated gifs use more than that, but let’s stick with the basic case. 256. The frames you create for your animations will likely be pngs, which mean they can have millions of colors. Somehow, ffmpeg needs to take all those millions of colors and choose just 256 for the gif.

Last time, I posted this command:

ffmpeg -framerate 30 -i frames/frame_%04d.png out.gif

This was working pretty well for my grayscale gifs, but if you tried using it for full color animations, there’s a good chance you ended up with a mess. Because you basically got a random palette. I don’t know how the palette is chosen in that case, but there’s a damn good chance it’s not going to be right. So you need to create a decent palette. That’s done with a palettegen filter. First you run ffmpeg to generate a new 16×16 pixel image (256 pixels) that contains the palette it thinks it should use. That looks like this:

ffmpeg -i frames/frame_%04d.png -vf palettegen palette.png

I’ll keep the same flow of going through each parameter:

-i frames/frame_%04d.png – chooses the frames that contain the palette data.

-vf palettegen – a video filter. The filter is palettegen which generates a palette.

palette.png – the output image holding the palette.

The result, as I said, is just a 16×16 png image. You can open it up and see your palette. This has been done by analyzing all the images in the sequence and determining which 256 colors can be used across the entire animation to make things look decent.

Now you use this palette image in another call to ffmpeg:

ffmpeg -framerate 30 -i frames/frame_%04d.png -i palette.png -filter_complex paletteuse out.gif

Once again, I’ll break it down.

-framerate 30 – the fps just like before.

-i frames/frame_%04d.png – the input frames, just like before.

-i palette.png – yet another input, this time, the palette image.

-filter_complex paletteuse – this is where the magic happens.

out.gif – the output image, just like before.

So the filter_complex one is pretty complex, especially if you try to look up the documentation or examples. You’ll find examples like this (IGNORE THESE!):

ffmpeg -i input.mp4 -filter_complex "[0:v]scale=iw:2*trunc(iw*16/18), \
boxblur=luma_radius=min(h\,w)/20:luma_power=1:chroma_radius=min(cw\,ch)/20: \
chroma_power=1[bg];[bg][0:v]overlay=(W-w)/2:(H-h)/2,setsar=1" output.mp4

or…

ffmpeg -i bg.mp4 -i video1.mp4 -i video2.mp4 -filter_complex \
"[0:v][1:v]setpts=PTS-STARTPTS,overlay=20:40[bg]; \
 [bg][2:v]setpts=PTS-STARTPTS,overlay=(W-w)/2:(H-h)/2[v]; \
 [1:a][2:a]amerge=inputs=2[a]" \
-map "[v]" -map "[a]" -ac 2 output.mp4

You’ll generally find these in Stackoverflow, with instructions like, “Just do this…” and no explanations at all on why you should JUST do that.

If you’re lucky, you’ll at least find something like…

 -filter_complex "fps=24,scale=${SIZE}:-1:flags=lanczos[x];[x][1:v]paletteuse"

I stripped away everything but the filter_complex part there. This one actually came from a good friend, Kenny Bunch, who saw my last article and happened to be digging in to the exact same stuff at the same time, but with full color animations.

This one was still more complex than I needed, but it was the simplest example I found, so I was very thankful.

So back to basics. filter_complex is a way of defining, a… well, a complex filter. You define the filter in a string and can chain together multiple actions to do all kinds of fancy things. The way it works is a series of filter actions that result in an output, and then you can use that output in further actions. Like this, broken down per action:

do first filter action[a]; does the first filter action and stores the output in a variable a.

[a] do another filter action[b]; feeds the output a into the next action, and saves that as b.

[b] yet another action[x]; you get the idea.

So in the above example:

fps=24,scale=${SIZE}:-1:flags=lanczos[x]; Sets the fps to 24, scales the gif to a given size on x and keeps the aspect ratio on y (the -1 param), uses Lanczos resampling to do the scaling. Stores the result in x.

[x][1:v]paletteuse Takes the data in x and uses input 1 (the palette image) as a palette using the paletteuse filter. In short, uses the palette image as the palette for the animation.

In my case, I’d already set the framerate. And I wasn’t scaling anything. so I could get rid of that whole first action. And apparently, ffmpeg was smart enough to figure out that input 0 was the frames and input 1 was the palette, So I could shorten the entire thing down to :

-filter_complex paletteuse

Magical.

Using these two commands, I was able to get a correctly paletted animation, with no yellow:

I still don’t know exactly why I was getting yellow, since all the source frames were completely grayscale. I guess it was just the complexity of calculating the values of all the channels of all the pixels from all the frames. Somewhere along the line, it wound up with a bit less in the blue channel for some reason. And once it started, it just multiplied.

Anyway, couldn’t help thinking of this…

Speed and Size

Other considerations when deciding whether to use ffmpeg or ImageMagick are rendering speed and file size.

ffmpeg will create gifs way faster than ImageMagick. A quick test:

Input: 300 frames, 500×500 pixels each.

ImageMagick: 29.985 seconds

ffmpeg: 4.301 seconds

And the ffmpeg test included generating the palette as well as the animation.

File size is not so good a story though. Same animation:

ImageMagick: 13 Mb

ffmpeg: 27 Mb

Other examples weren’t quite that bad, but ImageMagick always wins handily in the size category.

Of course, the other thing I mentioned last time was that ImageMagic will consistently use so much memory that the process just crashes and fails, whereas ffmpeg never has a problem in that area.

Just Say Yes

One last trick if you’re scripting these commands and you’re constantly having your script stop and ask you if you want to override your previous output gif with a new animation. Just add the -y parameter into your command and it won’t bother you anymore.

I create a lot of animated gifs and I’m getting into creating videos as well. All of these are initially created by rendering a series of png files into a folder called frames. Over the last few years I’ve figured out some pretty good recipes for converting these frames into animated gifs and videos. So I’m sharing.

Both ImageMagick and ffmpeg are extremely complex programs with tons of parameters that allow you to do all kinds of image, video and audio manipulation. I’m not any kind of an expert on any of these. And this post is not meant as thorough documentation on these programs. It’s more of a cookbook of commands that handle specific use cases well. I’ll also try to explain what each of these parameters does, which might help you to come up with your own commands.

Nothing beats thoroughly reading the official docs, but if you just want to get something done quickly, these recipes should help you.

ImageMagick

It confused me at first, but ImageMagick is not a single executable. It’s more of a suite of graphics utilities. Maybe under the hood it’s all one executable, but it presents as different commands you can call. For generating an animated gif, you’ll be using the convert command.

As I said, I keep my frames in a folder called frames. They are named sequentially like so:

• frame_0000.png
• frame_0001.png
• frame_0002.png

I use four digits, which gives me up to 10,000 frames. At 30 fps, that gives me about 333 seconds worth of video, or about five and a half minutes. Good enough for my needs now. If you go with three digits, you’ll only get 33 seconds worth. If I ever need to do more than five minutes in a single clip, five digits will give me almost an hour.

convert uses simple wildcard matching, so my input is simply frames/*.png and the output can be something like out.gif

Next you need to figure out the frame rate. I usually go with 30 fps. You don’t enter frame rate directly with convert though. You need to specify a delay between each frame. And here’s where things get really weird. The unit you use for the delay parameter is hundredths of a second. So if you want 30fps, you’d specify 100 / 30 or 3.333… Not exactly intuitive, but you can get used to anything. Put it all together and you get:

convert -delay 3.333 frames/*.png out.png

But there’s one more parameter I put in there, which is -layers Optimize

convert has a whole ton of stuff you can do with it to affect how the gif is put together. -layers Optimize combines a few of the most useful ones into a single command. I found that before I started using this, I’d get an occasional animation that just totally glitched out for a frame or two. Actually, it was more than occasional. Not every time, but often enough to be a problem. Using this parameter completely solved it, so I highly recommend it. If you want to read more up on it: https://legacy.imagemagick.org/Usage/anim_opt/#optimize

So the final command I use is:

convert -delay 3.333 -layers Optimize frames/*.png out.png

This has been my bread and butter command for a long time now.

But lately, I’ve been running into problems using ImageMagick to create longer, larger format gifs. This hasn’t been an issue at all when I was just posting to Twitter, which has something like a 15mb size limit. But as I’ve been posting gifs in other places that can be up to 100mb, I’ve started making larger and longer animations. And ImageMagick has been crashing pretty regularly while generating these. I’ve watched a graph of memory use and I’m pretty sure that it’s just running out of memory. It uses up all my physical memory, then uses up all my swap, then crashes. On at least one occasion I got it to work by closing down every other program that was running on the computer. That freed up just enough memory to get through. But it doesn’t always work. So that’s… less than ideal.

ffmpeg for gifs

[Update] Before using the command below, read this post, which has to do with color palettes: More ffmpeg Tips

I’ve been using ffmpeg for a while now to create videos, and I knew it could create gifs as well, so I’ve recently given it a try and it seems to do just as good a job as ImageMagick. And it can handle those high res, long gifs as well. So I’m pretty happy so far. Creating gifs with ffmpeg is also pretty straightforward, except for the input filename specification, which I’ll cover next. Here’s the command I use:

ffmpeg -framerate 30 -i frames/frame_%04d.png out.gif

Breaking that down, the framerate param is just the number of frames per second. Simple. -i is for input, and you point it to your sequential frames. and finally, the output filename.

The only complexity is the frame_%04d.png part. This is a printf type of string formatting convention. The %04d part means that you have four digits (4d), and they will be padded with 0s. So frame 37 will be 0037.

As I said, this has been working really well for me so far, but I haven’t been using it very long, so if there are any issues, I might come upon them later.

Videos

Video creation is what ffmpeg is usually used for. It’s an amazingly powerful and complicated tool. Even just figuring out all of the possible parameters can be daunting. So I’ll go over some of the common use cases I have and what each parameter does.

Mostly I’ve been optimizing the videos I create for use on YouTube. This uses h264 encoding and a specific pixel format. Here’s the basic command I use for creating a video from a sequence of frames:

ffmpeg -framerate 30 -i frames/frame_%04d.png -s 1920x1080 -c:v libx264 \
 -preset veryslow -crf 20 -pix_fmt yuv420p out.mp4

(That should be all one line. I wrapped it to make it more readable.)

Yeah, a lot to take in there. So let’s go through it param-by-param.

-framerate 30 is just the fps setting.

-i frames/frame_%04d.png specifies the input files as described earlier.

-s 1920x1080 sets the size of the video in the format of widthxheight. I believe that if you leave this param off, it will create the video based on the size of the source images. But you can use this to scale up or down as well.

-c:v libx264 OK, this will take a little background. ffmpeg can be used for creating or processing both audio and video – or both at the same time. Some of the parameters can be applied to audio or video. In this case, -c is letting you specify what codec to use for encoding. But is it a video codec or an audio codec? By adding :v we’re specifying that it’s a video codec. Fairly obvious here because there is no audio involved in this case, but explicit is always good. I’ve seen examples where people use -s:v to specify the size of the video, which seems overkill to me. But ffmpeg can also handle things like subtitles, and maybe those can be sized? Again, explicit is good. Anyway, here we are saying to use the libx264 codec for encoding the video. Which is good for Youtube.

-preset veryslow Here we have a tradeoff – speed vs size. These are the presets you can use:

• ultrafast
• superfast
• veryfast
• faster
• fast
• medium – default preset
• slow
• slower
• veryslow

The faster you go, the larger your file size will be. It’s recommended that you use the slowest preset you have patience for.

-crf 20 is the constant rate factor. There are two ways of encoding h264 – constant rate factor (CRF) and average bit rate (ABR). Apparently CRF gives better quality generally. This parameter can go from 0 to 51, where 0 is lossless and 51 is crap. 23 is default and anything from around 17 to 28 is probably going to be generally acceptable. The CRF setting also affects file size. Lower CRF means a higher file size. So if you need the lowest possible file size, use a high CRF and a slow preset. It will take a long time and look like crap, but it will be small! More info on all this here: http://trac.ffmpeg.org/wiki/Encode/H.264

-pix_fmt yuv420p is the pixel format. You might be more used to RGB888 where you have red, green and blue channels, and 8 bits per each of those channels. YUV is just an alternate way of formatting colors and pixels. Mind-numbingly detailed description over at the old wikipedia: https://en.wikipedia.org/wiki/YUV . But this is a good format for Youtube.

out.mp4 is the file that is created after applying everything else.

So there you go. my bread and butter video creation command.

But there are a few more tidbits I use:

Fades

I usually try to make my animated gifs loop smoothly. But for longer form videos this is not always what you are going for. Still, you usually don’t want the video to just get to the end and stop. A fade out to black is a nice ending effect. You could build that into your source animation code, or you could do it in post of course. But ffmpeg has fades built in. Here’s the altered command:

ffmpeg -framerate 30 -i frames/frame_%04d.png -s 1920x1080 -c:v libx264 \
 -preset veryslow -crf 24 -pix_fmt yuv420p -vf "fade=t=out:st=55:d=5" out.mp4

Here, I’ve added the param -vf "fade=t=out:st=55:d=5"

The vf is for video filter, I believe. This takes a string in a very compact format. It starts by specifying the type of filter and its definition: "fade=...". After the equals sign is a list of parameters in the format: a=x:b=y:c=z. We’ll step through those:

t=out means the type of fade out.

st=55 tells the fade to start at 55 seconds into the video.

d=5 means the fade will last 5 seconds.

In most cases the st and d parameters will add up to the total video length, but you can do all kinds of things here. Like fading out mid video and then fading back in later.

Obviously t=in would execute a fade in. You might want to do that at the start of your video.

Although I have not had the need for it, I assume you can use -af to create an audio fade.

Adding Audio

One of the big benefits of video over animated gifs is the possibility of adding music or sound effects to the animation. As mentioned, ffmpeg can do that too.

I believe it’s possible to add the soundtrack to the video as you are creating it from the source frames, but this does not seem like a good workflow to me. I might generate dozens of videos before coming up with one I decide to go with. So it makes more sense to me to add the audio when I’m done. Here’s my command for adding audio to an existing video:

ffmpeg -i video.mp4 -i audio.wav -map 0:v -map 1:a -c:v copy output.mp4

Again, let’s go through each param.

-i video.mp4 is the video you’ve already created that you want to add audio to.

-i audio.wav is the sound file you want to add to the video. Yes, we have two inputs here. That’s how it works.

-map 0:v The map parameter tells ffmpeg which input is what. Here we’re saying the first input (0) is video (v).

-map 1:a As you can probably guess, this says that input 1 is audio.

-c:v copy As covered before, this is the video codec. In this case though, we don’t want to encode the video all over again. We just want to copy the video over and add the audio to it.

output.mp4 is the final combined audio/video file.

So that’s pretty straightforward and works great. But this kind of assumes that your video and audio are the same size. They might be, which is nice, but they might not be, which raises the question of how long to make the final video? Do you make it the same length of the video or the length of the audio?

I believe by default, it will choose the longest option. So if your video is one minute long and you throw a five minute song on top of it, you’ll wind up with a five minute video. Honestly, I’ve never tried this so I don’t know what happens. I assume you get four minutes of blackness. Or maybe four minutes of the last frame of the source video. Or maybe it loops?

On the other hand, if your video is longer than your audio, you will probably wind up with some amount of silence after the audio finishes.

So the other option here is the shortest param:

ffmpeg -i video.mp4 -i audio.wav -map 0:v -map 1:a -c:v copy -shortest output.mp4

All this does is make the output video the same length as the shortest input.

I’m sure you can figure out all the different use cases here. And you probably want to think about fading your audio out along with a video fade so the sound/music doesn’t just suddenly stop.

Here’s a tutorial on a really neat and powerful math art technique that I first came across years ago in the book, Computers, Pattern, Chaos and Beauty by Clifford Pickover. It’s described in Chapter 14, entitled Dynamical Systems. More recently I’ve seen it described as “Popcorn”. I’m not sure where that name came from, but there’s a good chance it originated with Paul Bourke, who used the term back in 1991 in this article.

The technique is based on some very simple formulas – just using a couple of trig functions (sine and tangent in this example) to transform an x, y point repeatedly. But it creates some amazingly intricate, complex and beautiful images. It’s also open to a nearly infinite amount of hacking by changing the few constants used or swapping out which trig functions you use or how you compose them.