JavaScript Audio Synthesis Part 3: Making Music

First of all, I am not a musician by any stretch of the imagination. But that fact will become obvious all too soon. But if you’re going to make sound with code, you wind up either making sound effects or music. So let’s start with music. My goal here was to create a simple “tracker” application. You program in notes, it plays those notes back in progression to form a song. Of sorts.

I’m going to keep the song very simple. Mary Had a Little Lamb. It’s a song that you can play every note with one beat, so you don’t have to mess around with different length notes. Here’s a simple version of the song transcribed into notes and rests:

g f e f
g g g –
f f f –
g b b –
g f e f
g g g g
f f g f
e – – –

A quick search on the ‘net gives you the frequency values for the notes b, e, f and g, the only ones you’ll need for this song. Code that into an object:

[php lang=”javascript”]
scale = {
g: 392,
f: 349.23,
e: 329.63,
b: 493.88
}
[/php]

And then you can just code the song as a string.

[php lang=”javascript”]
song = “gfefgg-fff-gbb-gfefggggffgfe—“;
[/php]

Now you can create an AudioContext and an oscillator and set an interval that runs at a certain speed. In the interval callback, get the next note, find its frequency and set the oscillator’s frequency to that value. Like this:

[php lang=”javascript”]
window.onload = function() {

var audio = new window.webkitAudioContext(),
osc = audio.createOscillator(),
position = 0,
scale = {
g: 392,
f: 349.23,
e: 329.63,
b: 493.88
},
song = “gfefgg-fff-gbb-gfefggggffgfe—“;

osc.connect(audio.destination);
osc.start(0);

setInterval(play, 1000 / 4);

function play() {
var note = song.charAt(position),
freq = scale[note];
position += 1;
if(position >= song.length) {
position = 0;
}
if(freq) {
osc.frequency.value = freq;
}
}
};
[/php]

Now this actually works and you should be able to recognize the melody somewhat. But it leaves a lot to be desired. The biggest thing is that there is no separation between notes. You have a single oscillator running and you’re just changing its frequency. This ends up creating a sort of slide between notes rather than distinct notes. And when there’s a rest, well, there is no rest. It just keeps playing the last note.

There are various ways to try to handle this. One would be to call stop() on the oscillator, then change its frequency, then call start() again. But, when you read the documentation, it turns out that start and stop are one time operations on an oscillator. Once you call stop, it’s done. That particular oscillator cannot be restarted. So what to do?

The suggested answer is actually to create a new oscillator for each note. Initially, this sounds like a horrible idea. Create and destroy a new object for every single note in the song??? Well, it turns out that it’s not so bad. There are some frameworks that create a sort of object pool of notes in the background and reuse them. But the downside to that is that every note you create and start continues playing even if you can’t hear it. It’s your choice, and I suppose you could do all sorts of profiling to see which is more performant. But for Mary Had a Little Lamb, I think you’ll be safe to create a new oscillator each time.

To do this, make a new function called createOscillator. This will create an oscillator, specify its frequency and start it. After a given time, it will stop and disconnect that oscillator. You can then get rid of the main osc variable in the code and call the createOscillator function when you want to play a note.

[php lang=”javascript”]
window.onload = function() {

var audio = new window.webkitAudioContext(),
position = 0,
scale = {
g: 392,
f: 349.23,
e: 329.63,
b: 493.88
},
song = “gfefgg-fff-gbb-gfefggggffgfe—“;

setInterval(play, 1000 / 4);

function createOscillator(freq) {
var osc = audio.createOscillator();

osc.frequency.value = freq;
osc.type = “square”;
osc.connect(audio.destination);
osc.start(0);

setTimeout(function() {
osc.stop(0);
osc.disconnect(audio.destination);
}, 1000 / 4)
}

function play() {
var note = song.charAt(position),
freq = scale[note];
position += 1;
if(position >= song.length) {
position = 0;
}
if(freq) {
createOscillator(freq);
}
}
};
[/php]

This sounds better already. Each note is distinct, and when there is a rest, no note plays. But you can do even better.

The notes are just flat tones at this point. You can improve this by giving them a quick and dirty sound envelope. In real life, a sound doesn’t usually just start at full volume and cut out to silence when its time is up. The volume usually ramps up a bit first (the attack) and fades out quickly or slowly (the decay). Actually, sound envelopes can be much more complex than this (http://en.wikipedia.org/wiki/ADSR_envelope#ADSR_envelope) but a simple attack and decay to 0 will do fine for now and give the notes a much better sound.

As a sound envelope controls the volume of a sound, you’ll need to create a gain node. Do this inside the createOscillator function:

[php lang=”javascript”]
function createOscillator(freq) {
var attack = 10,
decay = 250,
gain = audio.createGain(),
osc = audio.createOscillator();
…
[/php]

The attack and decay values there are in milliseconds. This means that the sound will ramp up from 0 to full volume in 0.01 seconds and then back down to 0 in .25 seconds.

Next, the gain node will need to go between the oscillator’s output and the destination in order to control the volume of that oscillator.

[php lang=”javascript”]
gain.connect(audio.destination);
gain.gain.setValueAtTime(0, audio.currentTime);
gain.gain.linearRampToValueAtTime(1, audio.currentTime + attack / 1000);
gain.gain.linearRampToValueAtTime(0, audio.currentTime + decay / 1000);
…
[/php]

First you set the gain to 0, which in essence sets the volume to 0. You do this with the setValueAtTime method, passing in the current time from the AudioContext object. In other words, set the volume to 0, NOW.

Then you use linearRampToValueAtTime to set the attack and decay. This says go from whatever value you are currently at and interpolate to this new value so that you arrive there at the specified time. Note that the values you pass in here are in seconds, so you’ll need to divide by 1000 when using millisecond values.

Finally, you connect the oscillator to the gain, set the frequency and start it. And of course, when you clean up, you’ll need to disconnect everything as well.

[php lang=”javascript”]
osc.frequency.value = freq;
osc.type = “square”;
osc.connect(gain);
osc.start(0);

setTimeout(function() {
osc.stop(0);
osc.disconnect(gain);
gain.disconnect(audio.destination);
}, decay)
}
[/php]

The final code is below:

[php lang=”javascript”]
window.onload = function() {

var audio = new window.webkitAudioContext(),
position = 0,
scale = {
g: 392,
f: 349.23,
e: 329.63,
b: 493.88
},
song = “gfefgg-fff-gbb-gfefggggffgfe—“;

setInterval(play, 1000 / 4);

function createOscillator(freq) {
var attack = 10,
decay = 250,
gain = audio.createGain(),
osc = audio.createOscillator();

gain.connect(audio.destination);
gain.gain.setValueAtTime(0, audio.currentTime);
gain.gain.linearRampToValueAtTime(1, audio.currentTime + attack / 1000);
gain.gain.linearRampToValueAtTime(0, audio.currentTime + decay / 1000);

osc.frequency.value = freq;
osc.type = “square”;
osc.connect(gain);
osc.start(0);

setTimeout(function() {
osc.stop(0);
osc.disconnect(gain);
gain.disconnect(audio.destination);
}, decay)
}

function play() {
var note = song.charAt(position),
freq = scale[note];
position += 1;
if(position >= song.length) {
position = 0;
}
if(freq) {
createOscillator(freq);
}
}
};
[/php]

Now the notes have a more bell-like sound. It’s not awesome, but it’s a whole lot better. Mess around with trying to create different envelopes to see how that changes the sound. Code in your own song strings too. Don’t forget to add any additional note frequencies that you might use. You might even want to do something with allowing more than one-beat notes, though that starts to get a bit more complex.

Anyway, this is pretty rough and dirty code. A whole lot that could be improved upon here, but it’s good for demonstration purposes and hopefully gives you a better understanding of the underlying fundamentals than a more complex structure would. Have fun with it.