Music Visualizer


Music Visualizer (Processing)

A challenging and changing time we find ourselves in with the emergence of Covid-19. Live music gatherings have been largely cancelled for the foreseeable and social distancing has become a social norm. I want to experience a new environment within my isolation room, with this in mind I have set out to design my code.

This image has an empty alt attribute; its file name is 5-2.jpg

Design

“How do we visualize music?”, I ask myself; what colors, shapes or forms do we see? It is challenging to envision such displays with only the mind, this is where visual displays of real time auditory playback comes in handy. Auditory displays such as a music visualizer allows the user to see a visual representation of the audio content they’re listening to. When combining both forms of media synchronization becomes a matter of great importance. The most common forms of sound are speech and music, both heavily used in the field of multimedia, much development has been done in the representation of both visually.

A code that I found profoundly interesting and appealing to me uses Perlin noise to display a mountain/sea -like texture which possesses the user with the illusion of flight. I want to get this same effect from my own code, both in the nature of a seemingly living/breathing and never ending visualization. When visualizing music I like to think bass to the left and treble to the right, with respect to this my code will feature low frequency to the left and high frequency on the right.

This image has an empty alt attribute; its file name is wave-1.png
Example code from Daniel Shaffman

Implementation

When designing a music visualizer which would be effective with any mp3 file, I wanted a design which allows the user with as much input as possible, making the user feel in control. I will keep frequency horizontal (low end frequencies to left high end to the right). I have also added color to emphasize amplitude (green = low, red = high). These will happen simultaneously as the code runs. The user can then control the “play/pause” button and the “q” button as a navigation of exit.The song will begin to play.

 if ((key == 'q') || (key == 'Q')) // check if user hits Q, exit  {    exit();

I decided to add 2 sliders and a button to my code using the GUI Builder. The GP4 GUI Builder provides a visual form editor which provides easy to use assistance when adding buttons, sliders and text boxes. I first added a button for select file, once file has been selected the song and visuals will appear similtaniously after a file has been imported. The sliders used allow the user to manually adjust the visuals displayed by moving the xSlider and ySlider values, the range of values for xSlider are 0.01 to 1.0 and ySlider is 125 to 200. Below is the GUI Builder

This image has an empty alt attribute; its file name is gui.jpg
GUI Builder

In order to generate the frequency display I used void.drawspectrum. inside here I added a loop in respect of the height and width of the canvas, i also added a void.makespectrum to move the array initialized in the previous void function mentioned.  I also added bands*slider so that I could stretch my visual along the x-axis. The stroke(spectrum[j]*255, 255-spectrum[j]*255,0); picks a color depending on the energy in the band. I used RGB color system along with a spectrum and multiplied it by the variable j; to achieve a different color depending on the energy in the band.

 stroke(spectrum[j]*255, 255-spectrum[j]*255,0 ); // pick a color depending on energy in the bandColor
 vertex(width*log(i+1)/log(bands*slider), (height - sonogram[i][j]*height - log(j+1)*yoff));

I also added a void.makespectrum to move the array initialized in the previous void function mentioned above. This is done by ading another for loop to The Fast Fourier Transform (FFT) analysze calculates the power spectrum of an audio file, once it has a call for fft.analyze.

  fft.analyze(spectrum); // analyse the file. The fft is normalised 0..1  for(int i = 0; i < bands; i++) // show the analysis

Try my code here.

This image has an empty alt attribute; its file name is 5.jpg

Evaluation

“Thinking aloud may be the single most valuable usability engineering method.” (Neilson, Usability). For this reason I used this method of evaluation on my users as they carry out my task list mentioned below as they use my code for the first time. This cheap and roubust method is perfect for evaluating my design, the steps involved in the thinking aloud method:

  1. Recruit relevant user
  2. Give tasks to perform
  3. Stay quiet and encourage thinking aloud

Recruiting relevant users for my design was difficult as Covid-19 guidelines of social distancing meant I were to use my housemates of whom I was in quarantine with. I wrote down a list of tasks and gently navigated them towards the Processing icon on my desktop. I then instructed them the name of my application and where it is located in Processing folder. At this point it was up to the users and I would remain quiet while taking notes based off the participant responses.

Thinking aloud session

Task list below:
a) Start application
b) Open a music file
d) Listen to the music and look at the visualization.( How does it make you feel?)
e) Pause the music
f) Resume playing the music
g) Quit the application

As the Users began to carryout my task list i took down notes based of responses and action by the User A, B and C which are detailed below. I remained quite during this and as the User thinks aloud i jotted down some notes detailing the users use of interaction with the code itself and the visuals produced. Here i am looking for weaknesses in my design so i can amend them if necessary.

User A is fortunately on my course so is familiar with Processing. User beings by reading over the code and GUI window and launches the code, The User then clicks select file to launch a mp3 file and selects a familiar song with heavy treble and bass. Once the song plays user notices the color changing during heigth pitch sounds and comments on it. User then manipulates the form of the code using the siders. The user then notices the use of sliders over knobs and the effectiveness of them regarding the usability of the code. User then moves the ySlider to the max value and views the single line frequency bands interacting with the music and comments on the principle/foundation of the code. User then presses “q” to exit and close the window. The User then comments on the ability to exit using a key instead of navigating to the exit tab.

User B has little to no previous knowledge of Processing but agrees to participate in the talk list. User begins by selecting file to load an mp3 file, the user expresses their concern over my “lack of good songs”, user then selects a random sound file. User appeared dazzled by the visuals commenting on the distance achieved in the sketch. User then uses the command prompt and pauses the visualizer and resumes shortly afterwards. User then closes the application by pressing “Q” proceeds to load another song which produces another sketch, user comments on the difference in the visuals and comments on the color change as the music produces greater pitch. The user then commented on how the visuals could be effective behind a live electronic performance and refers back to a previous event that they experienced and wondered if the visuals could have been made using the same software.?

User C has a small amount of knowlodge of Processing from a previous experience. The user clicks. User Cbegan by using the sliders on the side of the frame but realized it does nothing while the song hasn’t began to play. Userthen clicks select file and locates a mp3 file located in the processing folder. Once the song begin to play the user is curious about the visuals and comments on the resemblance to a heart rate monitor. The user then goes back to the sliders once more as the visuals are running and interacts with the sliders to first stretch along the y-axis and along the x-axis. The user then requests to pause the song and fails to notice in the code the action needed to pause.  The user then closes then sketch.

A problem I noticed while carrying out my talk list to the users I noticed User A fails to notice in the code an action needed to pause the visual, I addressed this issue later in my code and included a command prompt for using a text box in the GUI Builder. I found that when locating a suitable place for the text boxes they suited the visualizer more when found at the top of the page to limit interference with the display being projected.

Since doing this assignment lock down still continues, who knows how long we will be kept in isolation. As i learn more and more about the seemingly unlimited possibilities in Processing i become profoundly more interested in the application. A very useful technique i discovered while doing this assignment is the use of GUI Builder and the value of a task list using thinking aloud to understand new possibilities to make my code greater. I look forward to making new visuals in the future as i think this medium of artwork can be massively effective.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s