BIT-101

In yesterday’s post I covered the bare bones basics of creating audio with the Web Audio API. In this post, I’ll demonstrate one way to start creating some interactivity.

One of the simplest and most dynamic ways to capture interactivity on a computer is by simply reading the mouse position. The strategy for this next experiment is to use the mouse’s y-position to control the frequency of a single oscillator. The code for this is super simple:

[php lang=”javascript”]
window.onload = function() {

var context = new window.webkitAudioContext(),
osc = context.createOscillator(),
h = window.innerHeight;

osc.connect(context.destination);
osc.start(0);

document.addEventListener(“mousemove”, function(e) {
osc.frequency.value = e.clientY / h * 1000 + 300;
});
};

[/php]

Create an AudioContext and an oscillator, get the window dimensions, connect and start the oscillator and listen for mousemove events.

In the mousemove handler, e.clientY / h will be a number from 0 to 1. Multiply this by 1000 and add 300 and you’ll have a frequency from 300 to 1300. This gets assigned to the oscillator’s frequency value. Move your mouse around the screen and get different pitches. Simple.

Remember, the above has only been tested in the latest version of Chrome at the time of this writing. Other configurations may work; some may require some changes.

Now what about they x-axis? Yesterday you had two oscillators going. Let’s try to hook the mouse’s x-position to that second oscillator.

[php lang=”javascript”]
window.onload = function() {

var context = new window.webkitAudioContext(),
osc = context.createOscillator(),
osc2 = context.createOscillator(),
gain = context.createGain(),
w = window.innerWidth,
h = window.innerHeight;

osc.frequency = 400;

osc.connect(context.destination);
osc.start(0);

gain.gain.value = 100;
gain.connect(osc.frequency);

osc2.frequency.value = 5;
osc2.connect(gain);
osc2.start(0);

document.addEventListener(“mousemove”, function(e) {
osc.frequency.value = e.clientY / h * 1000 + 200;
osc2.frequency.value = e.clientX / w * 30 + 5;
});
};

[/php]

This is much the same as the final code from yesterday, but now you’re using the mouse x- and y-positions to control the frequencies of both oscillators. Move your mouse all over the screen now and you’ll get all kinds of science-fictiony sounds. I picture a 1950’s flying saucer taking off, or maybe an alien ray gun. Mess with the frequency ranges of both oscillators and try changing the oscillator types for both – mix and match square, sawtooth and triangle waves for all kinds of interesting results.

This is something I’ve wanted to play with for a long time. So the other day I buckled down and started searching around.

Yes, you can create sound in JavaScript. In some browsers. Supposedly, it works in Chrome 14, Firefox 23, Opera 15 and Safari 6. Not IE. But, I’ve only tested this in Chrome. So for now, consider this something experimental and fun to lay with, not for your super awesome works-in-every-browser game.

I found several sites that had build up complex libraries based on the Web Audio API. These weren’t the greatest things to learn the basics from, but I eventually was able to pare some of the code down to the bare minimum needed to create some sounds in the browser. There’s also the MDN documentation here: https://developer.mozilla.org/en-US/docs/Web_Audio_API, which is a great reference, but not a step-by-step tutorial for creating sounds. There are a few tutorials linked to from there as well, but none really covered what I was interested in.

So, to get down to it, let’s create some noise.

First you need to create an AudioContext object. This is similar in concept to HTML5’s 2D graphics context for canvas. The context is the overarching object that you’ll use to create all the pieces that will create the sound you’re going to make. For Webkit-based browsers, you get an AudioContext like so:

[php lang=”javascript”]

var context =
new window.webkitAudioContext();

[/php]

The AudioContext has a few properties, the most important one being “destination”. The destination is basically the output of the context, where the sound goes. You can think of it as your speakers.

The next thing you need to know about the Web Audio API is that it is a node based system. You use the AudioContext to create various nodes that are used to create and shape sounds. Nodes have inputs and outputs that you can use to hook various nodes together into different configurations.

The most direct way to create a sound is to create an oscillator node. An oscillator node has zero inputs and one output. You can hook that output to the destination of your context. You’ll also need to specify a frequency for the oscillator. 440 hz will create the musical note, A. Here’s the code:

[php lang=”javascript”]
var context = new window.webkitAudioContext();

var osc = context.createOscillator();
osc.frequency.value = 440;
osc.connect(context.destination);
osc.start(0);

[/php]

And here’s how this looks from a node view:

Yeah, it says “frequency: 500”. My goof. Not changing it. Live with it.

You have an oscillator node with a frequency of 440 connected to the destination of the AudioContext. Call start(0) and you should get an annoying sound coming out of your speaker.

The oscillator node has a couple of other properties. One is “type”. This is the type of wave it uses to generate the sound. It defaults to “sine”. But, you can try “square”, “sawtooth” or “triangle” and see how they sound by doing this:

[php lang=”javascript”]
osc.type = “sawtooth”;
[/php]

There’s also a “custom” type, but that involves creating and setting a custom wave table. If you’re into it, go for it.

Anyway, wasn’t that easy? Let’s expand on it and create another oscillator that messes with the first one.

To do this, you’ll create two new nodes, an oscillator node and a gain node. A gain node is usually used to change the volume of a sound, but you’ll be using it here to alter the frequency of the original oscillator node. You’ll also create another, slower oscillator node. This new oscillator node’s output will be connected to the gain node. A gain node has a single input and a single output. As the new oscillator goes up and down at a frequency of 1 hz (once per second), it will affect the output of the gain node. A gain node also has a value property. If you set that to 100, then the gain node’s output will cycle from +100 to -100 as the new oscillator slowly cycles.

Now you need to hook this +/- 100 gain node output to the original oscillator. But remember that oscillator nodes don’t have any inputs, so you can’t connect it directly to the oscillator node. Instead, you connect it directly to the frequency property of the node. Now that gain output will change the original frequency of 440 hz + or – 100 hz as it cycles. A picture is worth several hundred words.

One oscillator is connected to the gain node, which is directly connected to the other oscillator’s frequency. That oscillator is connected to the destination. Here’s the code:

[php lang=”javascript”]
var context = new window.webkitAudioContext();

var osc = context.createOscillator();
osc.frequency.value = 440;
osc.connect(context.destination);
osc.start(0);

var gain = context.createGain();
gain.gain.value = 100;
gain.connect(osc.frequency);

var osc2 = context.createOscillator();
osc2.frequency.value = 1;
osc2.connect(gain);
osc2.start(0);
[/php]

Run that and you should hear a siren like sound.

You can also change the type of the second oscillator. Try making that a square wave:

[php lang=”javascript”]
osc2.type = “square”;
[/php]

Now, instead of cycling smoothly from +100 to -100, the gain’s output will be exactly +100 for half the cycle than jump to exactly -100 for the second half. The result is more like a siren you’d hear in Europe or the UK. It also drives my dog crazy.

Anyway, that’s the bare minimum. Check out the API at the link mentioned previously. There’s a lot more you can do with it from here.

At long last, the book is available for sale. It took a few days to get through all the copy edits and then a few last minute tweaks, then I hit the big old publish button. A few hours later, it was live … except in the US, Japan, India and Canada. I’m not sure if there is just some lag in the system for certain countries. I contacted support, they said they’d look into it and I never heard back, but the book soon went live on the US and Canadian stores. As of this writing it’s still listed as unavailable for purchase in India and Japan. It may resolve itself, but I’ll keep pushing on it anyway.

As mentioned previously, the whole process was a great experience and feels like a big accomplishment to have pulled it off from inception to being in the store, all under my own steam.

Anyway, go get it if this is the kind of thing you’re interested in. I hope you enjoy it. If so, it would be really nice to start seeing some good reviews stacking up there. 🙂