Painting, data and music

I felt like I was in over my head taking this traditional painting course at OCAD. Fortunately, our instructor has a more open definition of painting.

I’d spent a lot of time learning a Mendelssohn song on the piano, practicing it again and again to focus on each layer of information in the sheet music (notes, volume, pedal, etc.) I thought it would be simple to translate the sheet music into a data set on excel. It ended up being more complicated. How do you put right hand and left hand notes in the same table when there are different amounts of each? How do you show markings that refer to groups of notes or that occur in the spaces between notes?

In Processing, I took data from:

• pedal markings
• note pitch
• note length
• volume (eg.: mp, p, f)

and turned them into:

• line continuity
• line curve
• line length
• line thickness and colour

(An early test)

Instead of creating a line graph to represent music, the program “paints” a line one segment at a time, and the line curves and twists according to the relationship between each note and its predecessors. To me, this method of showing the data is more representative of the way we listen to music.  We usually don’t have perfect pitch, so we use the relationship between one note and the next to understand what’s being played. We pay attention to this relative pitch as much as we pay attention to the global pitch.

This image is generated from one line of music, divided into the melody and the harmony. The shapes remind me a little of photographs of microscopic organisms.

I adjusted the code to be able to include more accurate information. In both, you can see that when the line curves very smoothly, notes are progressively going up or down. Repeating patterns in the music can be see in through patterns in the line.

The chunky line above on the left is something I generated after having figured out how to choose notes that would create interesting shapes in the line. It does not use data from a real piece of music.

It may be interesting to use a keyboard and generate these graphics in real time to see how someone might improvise when they have this added visual feedback.

 

Advertisement

Data Visualization

I worked a lot with data visualization a while back. It has been really fun to work with big data sets and to try to make them understandable. Here are some images of a visualization project I made in Processing with Sally Luc.

Sally was the master coder and UI designer behind the practical side of the program. Users can organize movie data in ways that are most relevant to their interests, and everything is presented cleanly.

I was more involved with the weirder side of the program, where popular movies are visualized by the dominant colour of their poster. You can see how this changes throughout the years by basically drawing the data onto the screen… or you can just calm yourself down by making some pretty circle art. I particularly like the purple trend around 2008.

The program still needs a bit of fine tuning (e.g.: that glitch on the left), but people enjoyed playing with it.

Playing with APIs and sentence structure

I was going through my files when I found a project I had been playing around with early last school year.

The program, created in Processing, takes famous quotes from a website, breaks them down into words and allows you to recombine them into “misquotes”. I like games that play with words. This program is my first step into developing an idea about a co-operative game where players sabotage each other’s attempts at communication.

Lock System

Here’s a combination lock system prototype I made. It has 4 modes to it and gives light and sound feedback.

Full Feather

I’ve been working on a game in Unity for what has probably been my longest project at University. Since my semester has been so busy, I had to squish working periods into a few free spaces in my schedule. The process of making the demo was really stressful, but I made sure to go with an idea I really cared about and I’m excited to continue working on it past the stage of it being an assignment.

I worked mostly alone on this one, which gave me a lot of flexibility. It also meant I would have to learn a lot of things I had never done seriously before. This included coding in Unity, designing repeating patterns (patterns are an integral part of the game) and 3D modelling/rigging/animation. The game has a long way to go. It has been a great learning experience and I have a lot of ideas of how it can grow. You can check out the development blog here.