I worked on this project as Lead Engineer at Tellart

Partners: mono, Adam Florin

“A ten-day audio experiment that listened to more than just the audible noise of a New York City street corner, SoundAffects took the form of a wall on the corner of 5th Avenue and 13th Street that collected data input including temperature readings, color analysis of video feeds, precipitation measurements, noise levels, cellular phone activity/interference, and proximity sensors.

All of this raw data was fed into the SoundAffects generative music algorithm and visualizer system. We worked with mono for client Parsons The New School for Design, and the goal was to encourage engagement with the city environment. The music, and the corresponding visuals and video, were available in near real-time through the browser or your phone.

The project was completed using HTML5, JavaScript, Max/MSP/Jitter, Ableton Live, and various other software tools. On the hardware side, Tellart rigged industrial sensors to an Arduino micro-controller, and also captured live video throughout the ten days. The music composition algorithm was composed by Tellart’s in-house computer-music specialists.”

-Description courtesy of  Tellart

As the lead engineer, as well as the lead music composer on this project, I was always busy. Some days, I was stress-testing a cluster of Mac Minis that would eventually upload the piece to the internet in real time. Other days, I was tweaking synthesizers and arpeggios.

My favorite part about this project was hearing it all in context after months of work. There was just something magical about how the music changed throughout the day–bustling, clicking , and chirping through mid-day, then careening through an eerie chromatic twilight modulation, until finally, in the middle of the night, it opened up into a low, slow, reverberant drone. And the chance occurrences were hilarious. I remember once I heard something funny and new in the music, so I looked over, and there was a bright red falafel truck jerkily rolling off the curb to head back to queens for the evening.

I had programmed the colors in the street to vary the melody, but this falafel truck was taking red to a new level.

SoundAffects received a Core77 Design Award