Painting, data and music

I felt like I was in over my head taking this traditional painting course at OCAD. Fortunately, our instructor has a more open definition of painting.

I’d spent a lot of time learning a Mendelssohn song on the piano, practicing it again and again to focus on each layer of information in the sheet music (notes, volume, pedal, etc.) I thought it would be simple to translate the sheet music into a data set on excel. It ended up being more complicated. How do you put right hand and left hand notes in the same table when there are different amounts of each? How do you show markings that refer to groups of notes or that occur in the spaces between notes?

In Processing, I took data from:

  • pedal markings
  • note pitch
  • note length
  • volume (eg.: mp, p, f)

and turned them into:

  • line continuity
  • line curve
  • line length
  • line thickness and colour

Screen Shot 2016-11-15 at 10.25.29 AM.png

(An early test)

Instead of creating a line graph to represent music, the program “paints” a line one segment at a time, and the line curves and twists according to the relationship between each note and its predecessors. To me, this method of showing the data is more representative of the way we listen to music.  We usually don’t have perfect pitch, so we use the relationship between one note and the next to understand what’s being played. We pay attention to this relative pitch as much as we pay attention to the global pitch.

Screen Shot 2016-11-15 at 9.19.44 AM.png

This image is generated from one line of music, divided into the melody and the harmony. The shapes remind me a little of photographs of microscopic organisms.

Screen Shot 2016-11-24 at 11.16.15 AM.png

I adjusted the code to be able to include more accurate information. In both, you can see that when the line curves very smoothly, notes are progressively going up or down. Repeating patterns in the music can be see in through patterns in the line.

Screen Shot 2016-11-17 at 1.15.24 PM.png

The chunky line above on the left is something I generated after having figured out how to choose notes that would create interesting shapes in the line. It does not use data from a real piece of music.

It may be interesting to use a keyboard and generate these graphics in real time to see how someone might improvise when they have this added visual feedback.