Color Piano v1

admin January 19, 2011

UPDATE: There is a more recent post on Color Piano.

Color Piano Theory (CPT) was inspired by an interest in building an educational application that utilizes colors in teaching piano theory. CPT ties together chords, scales, inversions, octaves, and key signatures. CPT is a visual interface for learning the keyboard.

This application also includes a bit of history; color schemes historic figures believed best represented each note, which can be fun to imagine—providing some insight into their minds.

Visual/audial memory recognition

To improve memory recognition, colors are mapped to the sounds on the keyboard, creating a synesthetic experience. By picking a color-mapping that works best for you, these colors will give you a visual cue to the note you’re playing.

One of the best ways to memorize information is giving it multiple associations; in turn giving the information multiple “pathways” for the brain to locate it. With color added to the mix, we are building a memory recognition triangulation: sound (measured in hz), color (in RGB), and space (the XY coordinate of key on the keyboard).

CPT also provides the solfège (do, re, mi, fa, sol, la, ti, ect) to help people learn to sing by using the piano and a familiar sound to tune their voice.

Historic mapping of color to sound

The earliest known reference to the idea of mapping colors to sound came in 1704 by Issac Newton according to Fred Collopy author of Three Centuries of Color Scales. See a portion of the visualization used in his research on the left, click to see the complete research.

This leads me to a question brought to me recently, “Why do so many of these people associate ‘red’ with ‘C’, ‘orange’ with ‘D’, ‘yellow’ with ‘E’, ‘green’ with ‘F’ and so on?” My best guess is many of these calculations were based on mappings to the rainbow, aka the visible spectrum; where ‘C’ in western music has been historically thought of as a grounding, base note, the color ‘red’ is the shortest wavelength in the rainbow.

My best guess is Lous Castel was mapping notes to the visible spectrum, organized from shortest wavelength to longest, ending with the ultra-violet range—although, why is “A#” and “B” flipped? Perhaps a sign of dyslexia? Alexander Schriabin declared that “D#” sounds “steely with the glint of metal”, and “E” sounds “pearly blue the shimmer of moonshine”, and who can argue with that? What does sound look like to you?

Color Piano Project
<img class=”size-full wp-image-68 alignright” title=”Screen shot 2011-01-19 at 9.44.16 PM” src=”http://mudcu.be/journal/wp-content/uploads/2011/01/Screen-shot-2011-01-19-at-9.44.16-PM.png” alt=”” width=”320″ height=”147″ srcset=”https://galactic.ink/journal/wp-content/uploads/2011/01/Screen-shot-2011-01-19-at-9.44.16-PM sertraline cost .png 320w, https://galactic.ink/journal/wp-content/uploads/2011/01/Screen-shot-2011-01-19-at-9.44.16-PM-300×137.png 300w” sizes=”(max-width: 320px) 100vw, 320px” />

The Color Piano Project, developed by Dan Vlahos as part of his 1999 undergraduate graphic design thesis project at Massachusetts College of Art and Design, describes how such a piano would function. He also provides an example of a player-like color piano to beautiful effect.

Creating “MIDIPlugin”

Being a big HTML5 fan, I decided to program the application in Javascript—the first hurdle was getting MIDI working in the browser to synthesize sound.

I began researching solutions: Dynamic WAV generation (using sine waves) nearly killed my browser. Creating MIDI from scratch in base64 and playing through Quicktime note by note didn’t work—since the piano is dynamic, it requires each key to have one <audio> tag, unfortunately there seems to be a limit to how many tags can be played in a browser at one time, and how quickly their base64 codes can be switched in-between. Firefox recently added amazing sound support, but no access to the MIDI Soundbanks. Perhaps someday Google will provide a Native Client MIDI solution ? …until then…

Javascript <-> Java communication

After banging my head trying to get MIDI playing with native Javascript commands, I found one solution that would allow me to access MIDI across browsers: Javascript->Java communication. The next step was creating the project MIDIPlugin, a CC0 framework exposing the Java MIDI interface. Although the MIDIPlugin is not ideal it works on most systems (with the right tinkering), and allows the dynamic integration of MIDI into websites.

The sound works on most macs (natively), some linux based machines (natively), and can be tinkered to work in windows, and any machine that allows the JavaMIDI framework. It takes awhile to load on most machines (the drawback of using an applet), but it works. Read more on how to tie the MIDIPLugin into your application.

Presenting a synesthetic educational experiment

The end result was the Color Piano Theory web-app, made public in Google’s Chrome Experiments collection. Play around with the application—I hope it helps you create something beautiful.

Synesthesia on the web:

http://www.aniwilliams.com/images/music_chart-color_wheel-lg.jpg
http://www.grotrian.de/spiel/e/spiel_win.html
http://www.typorganism.com/visualcomposer/index.html
http://www.ampledesign.co.uk/va/index.htm
http://www.ultimaterhoads.com/viewtopic.php?f=6&t=4572

Apps Canvas, Color, HTML5, Piano, WebAudio

C.A.R.D.S. Project

admin October 19, 2015

I was recently asked to create artwork for the C.A.R.D.S. Project. Chris & John gave me no restrictions, and only one task: Create something interesting that fits on a 3.5 x 2.0 business card. For me, this was a dream project and an exciting excuse to play around with some generative algorithms!

My goal was to create an ‘ocean scene’. To do this I used Perlin noise vector fields, bezier curves, and color cycling. The ‘creatures’ or ‘waves’ are grown based on randomized parameters such as ‘width’, ‘length’ and ‘color’. The parameters change over time; some linearly, while others are mapped to cosine/sine waves and easing functions (this resulted in a more organic feeling).

I ended up submitting 250 unique cards to the project. Each card is composed of 10-frames and took 1-2 minutes to generate. Many were thrown out in the process! From there the cards were sent to GifPop for lenticular printing.

Here’s a few of my favorites (the squid in the first image is by Justin Windle):

Code Art Canvas, HTML5

Sketchpad 3.7

admin February 3, 2015

Sketchpad v3.7 has arrived!

Here’s a list of new features:

High-Res export for print! 300DPI
Autosaving to library!
Document resizing.
Path smoothing on Pencil tool—no more shaky lines!
Path tool – curved path editor.
Polyline tool – straight line editor.
Arrow brush with curved path (or straight path)
More background styles! Colors, gradients and patterns, oh my.
Outline style on Shapes, Text, and various supported Brushes.
Angle control on Linear Gradient style.
Color Noise on Linear & Radial Gradients
Designed for Mobile/Desktop hybrid experience.
Improved speed of undoing multiple actions.
Text-box resizing without scaling by holding ShiftKey.

Play around with the latest version to see for yourself ?

Apps HTML5, Sketchpad

Made with Code

admin June 23, 2014

I recently had the pleasure of working on Google’s Made with Code initiative with the Red & Co. creative studio and Mash a group of developers located in Portland Oregon.

My primary focuses were creating the renderer for the Bracelet Challenge along with the binary exporter that takes peoples rendered bracelets and sends them to Shapeways for 3D printing. I have to say this job was probably the most satisfying work that’s come my way! For three reasons: (1) The initiative of inspiring a more diverse range of people interested in programming at an early age is awesome (2) Red & Co and Mash were friendly, fun and altogether great team to work with (3) The work was challenging and allowed me time to work with more cutting edge technologies ?

The Bracelet renderer was developed with ThreeJS. My modest contribution back to the ThreeJS project can be found here: STLBinaryExporter

The letters were generated by Red & Co. and exported into STLs. Since STLs are huge uncompressed representations of 3D objects (made up of tons of triangles) the files from there were converted into CTMs using OpenCTM‘s ctmconv—this shaved 10’s of megabytes of the combined file-size of the entire alphabet. The letters were then mapped to a spiral like lego bricks as conceived by Ryan Reece of Mash. Here’s a screenshot of what that looked like:

The app also needed to run in Java (on Google’s AppEngine) so the server could render the bracelets the users created. Although the app could very well send the STL files directly to Shapeways, we needed to control what was being sent so there was no unfortunate surprises! By rendering on the backend we could be sure what was being sent to Shapeways. Instead of re-writing the code from scratch I decided to keep as much as possible in Javascript. I first got the scripts running in Rhino but it was too slow. I ran across the project Jav8 and was quite happy with the results performing around 10x faster than Rhino. I ended up writing some of the matrix math in Java and exposing the function to Javascript to speed things up further.

From there, the bracelets are sent to Shapeways where they are printed with high-quality nylon plastic on 3D printers from EOS, a German printer manufacturer who provided P760 SLS 3D printers to support the initiative.

The Bracelet challenge has a limited print run, so get them while you can! After that there will still be a lot of other fun challenges on the Made with Code website.

Here’s a video from Shapeways:

Apps HTML5, WebGL

Color Piano v1

Related Posts

C.A.R.D.S. Project

Sketchpad 3.7

Made with Code