Audio Visualization in JavaScript with p5.js

This time I thought of visualizing audio from JavaScript. There’s no reason for why I wanted to do this in the first place! but, I thought it would be interesting to do and in the process, learn something new. Here’s what I learned.

Prerequisites -

1. Basic understanding about JavaScript and node.js
2. Basic mathematical knowledge is an advantage.

You can clone the project from here, and follow along.

nishanc/audio-visualization

git clone https://github.com/nishanc/audio-visualization.git

Project setup with p5.js

We’re doing this from p5.js, advertised as ‘a JavaScript library for creative coding, with a focus on making coding accessible and inclusive for artists, designers, educators, beginners, and anyone else!’. Well I must say that this really is the case. You can do graphical manipulations with less lines of code and they are easy to understand.

You can get started pretty easy, following snippet shows a sample HTML page with p5.js

sketch.js looks like this.

You can learn more about this at p5js.org/get-started. But in this tutorial we will use a node development server(with express) to run our code (install Node and Git if you haven’t already.). To get started, you can clone the first commit of the above github repository by executing following commands in order. Open a new folder from your terminal/cmd and execute,

git init
git remote add origin https://github.com/nishanc/audio-visualization 
git fetch origin 6c0f21158014b5bc8b21350db3fd9a5422447cd3
git reset --hard FETCH_HEAD

This will give you following directory structure.

In the same terminal, restore npm packages with,

npm i

Now you can run the server with,

node server.js

Navigate to http://localhost:3000 and you will see something like this. Which is an ellipse that follows your mouse pointer.

createCanvas() creates a canvas element in the document, and sets the dimensions of it in pixels. The background() function sets the color used for the background of the p5.js canvas. ellipse() draws an ellipse (oval) to the screen. An ellipse with equal width and height is a circle. By default, the first two parameters set the location, and the third and fourth parameters set the shape’s width and height. The system variable mouseX always contains the current horizontal position of the mouse, relative to (0, 0) of the canvas. The system variable mouseY always contains the current vertical position of the mouse, relative to (0, 0) of the canvas.

Okay. Now we know what we’re dealing with. Let’s move on.

Getting microphone input.

Before visualizing any audio, let’s see how we can get mic input.

We can get user microphone using p5.AudioIn() and overall amplitude/ volume from getLevel(). Then we draw an ellipse in the center of the canvas(height/2, width/2) with the radius of vol*500 , because the volume is a float between 0–1, so we need to convert it into a large number.

Get amplitude from audio file.

Alright! now we will try to do the same for an audio file. Create a new folder named audio inpublic folder and add an mp3 file. Then create a function name preload() and read the song.

function preload() {  
    song = loadSound('audio/the-alphabeat.mp3');
}

Then in the setup() start playing the song and get it’s amplitude using p5.Amplitude() which measures volume between 0.0 and 1.0. Listens to all p5sound by default, or use setInput() to listen to a specific sound source.

function setup() {  
    createCanvas(600, 600)
    song.play();    
    amp = new p5.Amplitude();
}

Now instead of mic.getLevel() we can say amp.getLevel() in the draw() function.

Graphing amplitude.

Now we will graph the amplitude on the canvas. To create a graph we need history of the amplitude. So create an empty array in line 2.

let volHistory = [];

In draw function, push each amplitude value to this array. Then we loop through the array and draw a point() in x and y coordinates, x is given by index of the array and y is a map() created to map the amplitude (0–1) to values between canvas height and 0

let vol = amp.getLevel();
volHistory.push(vol);
for (let x = 0; x < volHistory.length; x++) {
     stroke(255)
     let y = map(volHistory[x], 0, 1, height, 0);
     point(x, y);
}

Instead of a point, now we will draw a line. Using the beginShape() and endShape() functions allow creating more complex forms. beginShape() begins recording vertices for a shape and endShape() stops recording. All shapes are constructed by connecting a series of vertices. vertex() is used to specify the vertex coordinates for points, lines, triangles, quads, and polygons. So replace the point() with vertex(). Completed draw function is as follows.

Notice the if condition. You might have noticed that when we draw the point graph, it goes till the end of the canvas and stops. That is because our volHistory array is lengthier than the width of the canvas. So what we do is if the volHistory is lengthier than the width, at position 0, remove 1 item using splice(0,1) , first item of the array will be removed making space for new item at the end.

Radial graph.

Now let’s modify the linear graph to a radial graph. To achieve this you need to know the relationship between radius, angle of a circle with Cartesian plain.

You know that we need x and y coordinate to draw a vertex. If you know the r-radius, in our case is the amplitude and the angle — θ, this is an easy task.

As you know we may not use the amplitude as it is because that is too small (between 0–1), so we will map 0 to 10 and 1 to 300 . The maximum number of items that the volHistory array can have is also 360, corresponding to the total number of degrees in a circle. translate() specifies an amount to displace objects within the display window. The x parameter specifies left/right translation, the y parameter specifies up/down translation. We also want to set the current mode of p5 to degrees using angleMode(DEGREES); (default mode is RADIANS) in the setup() function.

Visualizing amplitude for each frequency.

To do this, we will use p5.FFT() . FFT stands for Fast Fourier transform. Fourier analysis converts a signal from its original domain (often time or space) to a representation in the frequency domain and vice versa. This operation is useful in many fields, but computing it directly from the definition is often too slow to be practical. A fast Fourier transform algorithm manages to reduce the complexity. That’s all you need to know.

As a start, let’s rename amp to fft and instead of p5.Amplitude() we will use a p5.FFT() . This takes two arguments, new p5.FFT([smoothing], [bins]) smooth is a value between 0.0 and 1.0, we will set it to 0.9 (used to smooth FFT analysis by averaging with the last analysis frame). bins is basically the number of frequencies / bands that we want to take out, default is 1024. We will set it to 128. Also, we will be using the analyze() function to get the frequency spectrum. It returns an array of amplitude values (between 0 and 255) across the frequency spectrum.

[75, 73, 74, 63, 54, 61, 71, 97, 122, 121, 116, 117, 127, 142, 151, 160, 182, 189, 215, 254, 255, 245, 183, 144, 136, 132, 124, 120, 127, 137, 140, 129, 126, 136, 133, 135, 151, 180, 184, 197, 203, 183, 145, 129, 118, 105, 106, 100, 104, 117, 131, 126, 131, 123, 144, 150, 148, 149, 166, 191, 197, 176, 129, 117, 117, 106, 115, 120, 115, 118, 125, 127, 127, 135, 135, 136, 138, 137, 168, 202, 206, 181, 137, 134, 141, 131, 128, 137, 140, 151, 153, 145, 154, 148, 145, 147, 148, 152, 198, 229, …]

What we’re trying to do is draw a line for each frequency band. Height of each line is equal to the amplitude of that band. Using map() function as before, we will map this value to the height of the canvas. And I have added a button to Pause and Play the audio. Modified sketch.js is as follows.

It’s stacked together right? Let’s get some space between them. If we want to expand this 128(bin size we set earlier) line across entire width of the canvas, the space between each line should be width/128.

Create a new variable and set it to width/128 in setup() function.

space_between_lines = width / 128;

And in draw() replace line ,

line(i, height, i, y);

with ,

line(i * space_between_lines, height, i * space_between_lines, y);

Now you will get something like this.

Instead of lines you can draw rectangles using rect()

Add colorMode(HSB);to the setup() function. Then in draw() replace,

line(i * space_between_lines, height, i * space_between_lines, y);

with,

fill(i,255,255); //remove stroke(255);
rect(i * space_between_lines, y, space_between_lines, height - y);

Output will look something like this.

If you want to flip the spectrum on x axis, change the above line as follows.

rect(width - (i * space_between_lines), y, space_between_lines, height - y);

Try to change the values and come up with a creative design of your own. With a bit of tweaking try if you can create the following effect.

Code for this output is in branch symmetric-spectrum, Clone the repository and switch branch using git checkout symmetric-spectrum

Well that’s it. I hope you learned something! Stay safe!.