Dynamic MIDI generation in the browser

admin August 19, 2010

UPDATE: This project has been cancelled, the MIDIBridge (which takes this idea much further) is what you’re looking for; http://www.abumarkub.net/abublog/?p=505

Have you ever wanted to use MIDI in your Javascript or Flash project? It’s been a long time dream of mine. After a long search I came up with one solution that would provide the best support for the most computers (at present time) — utilizing Java’s javax.sound.midi interface through an . Introducing the MIDIPlugin providing the fundamental functions to create dynamic music & sound effects in your browser.

The MIDIPlugin requires that Java and a MIDI Soundbank be installed on your computer. Some computers have these installed by default, others do not. More on computability later.

Piano Theory was built utilizing the MIDIPlugin. Although the MIDIPlugin works in Internet Explorer my piano webapp does not (at this time) — apologies to IE users.

My pitch for MIDI support becoming a W3C standard

Just as color and native primitives are the building blocks of graphics MIDI is an essential building block of music — we have for graphics, we’re lacking decent support for dynamic music generation.

The benefits of dynamic music generation is substantial — saving bandwidth, opening up a whole new realm of sense (the sense of audio) to dynamic content, allowing developers to create more interactive & immersive projects. MIDI is a well defined framework that could be implemented into the W3C standards, as long as there is the support behind it. With the motion that the web has been moving forwards recently, the possibility is ripe.

The plugin is easy to use

Include the MIDIPlugin in your projects to create a dynamic audio experience:

<applet archive="MIDIPlugin.jar" code="MIDIPlugin.class" height="1" id="MIDIPlugin" name="MIDIPlugin" width="1"><applet>

Test to see whether the browser supports the MIDIPlugin (has Soundbank installed and supports Java) with the following function in window.onload:

MIDIPlugin = document.MIDIPlugin;
setTimeout(function () { // run on next event loop (once MIDIPlugin is loaded)
    try { // activate MIDIPlugin
        MIDIPlugin.openPlugin();
    } catch (e) { // plugin not supported
        MIDIPlugin = false;
    }
}, 0);

When MIDIPlugin is false the user is either missing the MIDI Soundbank or Java — prompt the user to install the appropriate software. If the MIDIPlugin isn’t set to false… well then, the fun starts ;)Functions available to you within the MIDIPlugin object:

MIDIPlugin {
    setChannel(int)
    setMono(boolean)
    setMute(boolean)
    setOmni(boolean)
    setSolo(boolean)
    getInstruments(null)
    setInstrument(int)
    setBend(double) // 0.5 = default
    setPan(double) // 0.5 = center
    setPressure(double)
    setReverb(double)
    setSustain(double)
    setVelocity(double)
    setVolume(double)
    playNote(int) // 0 is low C - 1 is C# and so on
    stopNote(int) //
    playChord(string) // for instance [0,5,7].join(",")
    stopChord(string) //
    stopAllNotes()
    openPlugin()
    closePlugin()
}

Remember to unload the plugin onbeforeunload, or your will have a loose connection hanging, which over time (multiple reloads) will result in the MIDI sound not working until you restart your browser:
```
window.onbeforeunload = function() {
    MIDIPlugin.closePlugin();
};
```

Licensed to use in your projects ?

Released as CC0 — this means you can use the MIDIPlugin in your project, and modify it to your hearts content without giving recognition, be that commercial or non-commercial.

Git me

Easily forkable on GITHub or download the precompiled .jar applet.

Supported OS/Browsers

MIDIDPlugin works on Macs out of the box. Linux works out of the box depending on distribution, other times it requires Java SE to be installed. Windows requires Java SE as well as the MIDI Soundbanks to be installed (the Windows version of Java SE doesn’t ship with MIDI Soundbanks included). On the browser front the plugin works across the board: Chrome, Opera, Firefox, Safari and IE.

Other fun things in the name of music
http://rhythmiclight.com/archives/ideas/colorscales.html
http://www.alexisisaac.net/products/flashMidi/
http://homepage3.nifty.com/sketch/flash/flmml050.swf
http://code.google.com/p/abcjs/
http://www.vexflow.com/
http://en.wikipedia.org/wiki/General_MIDI

C.A.R.D.S. Project

admin October 19, 2015

I was recently asked to create artwork for the C.A.R.D.S. Project. Chris & John gave me no restrictions, and only one task: Create something interesting that fits on a 3.5 x 2.0 business card. For me, this was a dream project and an exciting excuse to play around with some generative algorithms!

My goal was to create an ‘ocean scene’. To do this I used Perlin noise vector fields, bezier curves, and color cycling. The ‘creatures’ or ‘waves’ are grown based on randomized parameters such as ‘width’, ‘length’ and ‘color’. The parameters change over time; some linearly, while others are mapped to cosine/sine waves and easing functions (this resulted in a more organic feeling).

I ended up submitting 250 unique cards to the project. Each card is composed of 10-frames and took 1-2 minutes to generate. Many were thrown out in the process! From there the cards were sent to GifPop for lenticular printing.

Here’s a few of my favorites (the squid in the first image is by Justin Windle):

Code Art Canvas, HTML5

Sketchpad 3.7

admin February 3, 2015

Sketchpad v3.7 has arrived!

Here’s a list of new features:

High-Res export for print! 300DPI
Autosaving to library!
Document resizing.
Path smoothing on Pencil tool—no more shaky lines!
Path tool – curved path editor.
Polyline tool – straight line editor.
Arrow brush with curved path (or straight path)
More background styles! Colors, gradients and patterns, oh my.
Outline style on Shapes, Text, and various supported Brushes.
Angle control on Linear Gradient style.
Color Noise on Linear & Radial Gradients
Designed for Mobile/Desktop hybrid experience.
Improved speed of undoing multiple actions.
Text-box resizing without scaling by holding ShiftKey.

Play around with the latest version to see for yourself ?

Apps HTML5, Sketchpad

Made with Code

admin June 23, 2014

I recently had the pleasure of working on Google’s Made with Code initiative with the Red & Co. creative studio and Mash a group of developers located in Portland Oregon.

My primary focuses were creating the renderer for the Bracelet Challenge along with the binary exporter that takes peoples rendered bracelets and sends them to Shapeways for 3D printing. I have to say this job was probably the most satisfying work that’s come my way! For three reasons: (1) The initiative of inspiring a more diverse range of people interested in programming at an early age is awesome (2) Red & Co and Mash were friendly, fun and altogether great team to work with (3) The work was challenging and allowed me time to work with more cutting edge technologies ?

The Bracelet renderer was developed with ThreeJS. My modest contribution back to the ThreeJS project can be found here: STLBinaryExporter

The letters were generated by Red & Co. and exported into STLs. Since STLs are huge uncompressed representations of 3D objects (made up of tons of triangles) the files from there were converted into CTMs using OpenCTM‘s ctmconv—this shaved 10’s of megabytes of the combined file-size of the entire alphabet. The letters were then mapped to a spiral like lego bricks as conceived by Ryan Reece of Mash. Here’s a screenshot of what that looked like:

The app also needed to run in Java (on Google’s AppEngine) so the server could render the bracelets the users created. Although the app could very well send the STL files directly to Shapeways, we needed to control what was being sent so there was no unfortunate surprises! By rendering on the backend we could be sure what was being sent to Shapeways. Instead of re-writing the code from scratch I decided to keep as much as possible in Javascript. I first got the scripts running in Rhino but it was too slow. I ran across the project Jav8 and was quite happy with the results performing around 10x faster than Rhino. I ended up writing some of the matrix math in Java and exposing the function to Javascript to speed things up further.

From there, the bracelets are sent to Shapeways where they are printed with high-quality nylon plastic on 3D printers from EOS, a German printer manufacturer who provided P760 SLS 3D printers to support the initiative.

The Bracelet challenge has a limited print run, so get them while you can! After that there will still be a lot of other fun challenges on the Made with Code website.

Here’s a video from Shapeways:

Apps HTML5, WebGL