Generative music and conceptual sounds(?)

frozenman on 19/12/2007 at 18:42

This is an area I've been spending a lot of my free time (i.e. time at work) reading about and thinking about, and I wanted to start a thread to maybe get some ideas here because no one I know really gives a shit about tape loops and band-pass filters, plus I know there's a couple of you folks on here who are in audio engineering schools and what not. PLUS I think the impetus that made me want to learn more in this field came from this message board a few years back when someone posted the link to the (http://instruct1.cit.cornell.edu/courses/eceprojectsland/STUDENTPROJ/2002to2003/lil2/) hamster-controlled MIDI synthesizer.

The hamsters really got my gears turning, and a lot of this deals with the same basic pattern: You take one natural system, one real world event or process, and apply a certain mapping or translation that converts the system's degrees of freedom into musical parameters. Obviously not any process will yield results as interesting as the hamsters, but something tells me that if you map naturally beautiful events (the flocking of birds, the way trees grow, the blinking of car-lights at an intersection as they are obstructed and revealed by passing cars) to sound, the results may be just as beautiful. Perhaps not 'beautiful' in the musical sense (Xenakis) but it will at least allow you to gain a new perspective on things.

So I'd like to share a couple of ideas I've had, part of the reason being is that these are all just wafts of thought that come to me, and I have very little idea of how feasible, original, or useless any of them really are or could be.

Inline Image:

http://image.versiontracker.com/scrnsht/127683/365089/t939FEaT110Sgram3DSurfBlkBkgnd.jpg

Topographic Sound (?)

A spectrogram takes a given audio signal and displays a 3d representation of it (aphex). To me its always sort of looked like mountain ranges and canyons. What would happen if you could import topographic maps of mountains and hills and valleys into some audio analysis software, and synthesize sound from that?

This is rather similar to programs like Metasynth where sound is extrapolated from an image, but I feel like in general images are a bit too chaotic to get anything but noise out of them. However, a range of mountains or hills ((http://www.davestravelcorner.com/photos/unitedstates/california/north/San-Jose-Hills.jpg)) I feel could make a really interesting sound, and sharp crags of mountains may even resemble the (http://www.zainea.com/su5.gif) formants of human speech.

Once you have these topographic maps, I imagine it would be possible to develop models of erosion that you could apply to the sound, so that ridges of frequencies get worn out and those formants dulled out.

TREES

For a while I've been trying to answer the question 'How does a tree sound?' and here's one way I've thought up (but, naturally, I'm not even near implementing).

So imagine you could take the cross section of the whole height of a tree, taking half-centimeter slices of the tree from the trunk (one big slab in the center) all the way up the branches (dozens of little sticks), and orient the whole system such that the trunk is at the origin of an x-y plane. I've done some lookin' around on the internet and this isn't a very easy task in itself, so it may have to be substituted by a model of a tree, but supposing that you could plot all of the information about the position and posture of each branch in a tree....The z-axis, the height of the tree is taken to be the time axis, and each slice, each cross-section is a frame, an audio snapshot, a grain, whatever. I suppose that an ideal length for a whole tree would be on the order of a few seconds, say 5, multiplied by however many frames per second you want to hear, let's say 30.

For each frame:
-The area of the tree section controls the width of a bandpass filter that's being applied to some sound. For the trunk, or the biggest part of the tree, the filter would be wide open, for the tiny branches, it would be sharpened to a comb.
-The x coordinate of the tree section you're examining adjusts the offset of the bandpass filter.
-The y coordinate of the tree section you're examining adjusts the pan setting.

So for example, you start with the trunk at it's widest, at the ground. As you progress up the tree the trunk begins to taper, slightly closing the bandpass filter, until you reach the first branch (let's say this branch is idealized and is pointing along the line y=x). After a few frames the branch will be completely seperated from the trunk, and so now you'll have two bandpass filters, one is the trunk getting thinner, the second is the branch which is thinner still. Since the branch is pointing along the line y=x, as you progress each frame, the bandpass filter corresponding to it will a) pan to the right ear, b) become thinner and thinner, c) translate up the frequency spectrum higher and higher. Now I'm taking the science way of doing shit: that's how it would work for a single branch, now generalize it for the whole damned tree.

These are all just basically filter settings. So maybe that's not what a tree sounds like, but the way a tree FEELS. You'd still have to pick a suitably 'tree-ey' sound to be filtered, and it'd have to have enough harmonic content so that the very thin, distant branches can still be heard.

Anyways, I'd love to hear some comments from anybody. Not nescessarily pertaining to what I've babbled on about, but about any related topic. I can't wait to play Spore when it comes out.

Jason Moyer on 19/12/2007 at 18:53

The idea of taking 2-d and 3-d images and translating them into music is a good one, but I've never found software that did it very well. If someone made something that generated midi data or something out of it then it might be interesting, but everything I've used just basically generates audio files of random noise.

I can't remember all of the ones I've fiddled with, but there's Coagula and IIRC Fruityloops has a built-in instrument plugin that does it.

Volca on 20/12/2007 at 11:28

You reminded me of this book:
(http://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detective_Agency) Dirk Gently's Holistic Detective Agency

Anyway, you could try producing the mountain sound quite easily given there are dozens free (http://en.wikipedia.org/wiki/USGS_DEM) DEM files on the net. Only the usage inverse Fourier transform is needed (numerical, together with some stretching, correct me here), and you're practically done. I'd be interested to hear the result :)

frozenman on 20/12/2007 at 15:57

Thanks for the tip Volca,

Interesting, I'll take a look at these files. The caveat...and it's a big one...is that I have no real programming knowledge. I'm fairly familiar with Fourier Transforms and Analysis, primarily from undergraduate classes in physics, i.e. largely from a mathematical level, but I have no clue how to implement one numerically/computationally...er whatever. I've been meaning to look into CSound as I imagine this is probably the ideal environment to be working in.

Also, how did I remind you of the book, I'm not familiar with it...

Volca on 20/12/2007 at 16:06

I was thinking a bit about it, and you could actually do without any programming at all. DEM files can be converted to heightmap image (tga file or such - I saw dem2tga somewhere, but did no try). This image could be then played with something like this:
(http://www.webcenter.ru/~vsoft/BitmapPlayer.htm) http://www.webcenter.ru/~vsoft/BitmapPlayer.htm

About the book:
The character named Richard MacDuff is a programmer. He written a program that could convert any type of data into music. This was the link :)

demagogue on 20/12/2007 at 16:45

Here's my contribution: (http://musicalgorithms.ewu.edu/algorithms.html)

When you mentioned "what does a tree sound like?", it reminded me of the old saw that trees, like many things in nature, respect the Fibonacci algorithm, which got me thinking about this site. The idea is you select or input some algorithm, from math or economics or biology or whatever, then give some basic rules for how it's used ("normalizing" it to the pitch and duration), and then it mechanically "constructs" the piece.

While it might not seem like it at first, it actually is a lot like your idea, taking important and beautiful patterns from real world and representing them as musical patterns. And one nice thing about it is you can import your own numbers or algorithm; so you can take a pattern you want to represent and if you can reduce it to an algorithmic sequence, then the program takes care of musicalizing it in the way you'd like. It might not be exactly what you were thinking, but it's more effective than you might think at first. Try it out; it's fun.

You also really should listen to some of the compositions already (http://musicalgorithms.ewu.edu/archive.html) uploaded ... which musicalize things like Redwood tree DNA (very elegant imo), bull trout DNA, pi, the golden mean, a star chart, a rabbit population over time, etc, etc, to get an idea of how things can turn out.

Volca on 20/12/2007 at 20:37

I've tried the Bitmaps&Waves program with this image:
(http://www.travelnotes.de/rays/grandcan/grandcan.gif) http://www.travelnotes.de/rays/grandcan/grandcan.gif
(Grand Canyon. I didn't inline it, too big)

I rotated it clockwise 90 degrees, then inverted. The result was surprising to listen to. A sort of sci-fi sound you hear in the dark passages of some movies. It took a minute or two to play whole. Interesting, thanks for your idea :)

Now I'm gonna read and understand the Tree idea of yours ;)

frozenman on 23/12/2007 at 00:03

Quote Posted by demagogue

Here's my contribution: (http://musicalgorithms.ewu.edu/algorithms.html)

Yeah that website is pretty interesting. I've messed around with it a little bit, but, eh, I dunno, the results never quite struck me. (Although that Redwood DNA is pretty cool)

When applying algorithms for any sort of generative music, certainly the source system should have a sense of elegance or beauty, but it's also important that the algorithm that maps it into music must be equally elegant. Simply taking the numbers in the decimal expansion of phi is a rather, I dunno, obvious application. I do want to play around a bit more with the population growth one, however I feel like the evolving state of the system (#_population_1, #_population_2) could be better represented than just converting the numerical values into pitch and duration. Perhaps the size of a population could control the harmonic content of a long sustaining sound...as the population grows more harmonics get added and swell, and be panned accordingly (i.e. a very small population 1 will be in the left channel, as the population grows, harmonics are added and it spreads out more into the left channel, so the sound kind of devours population 2 which is in the right ear.)

Also, there's always a sort of edge-trimming involved when you use straight numerical data from a system to directly control pitch or duration, at least when you use midi, because you'll always have to perform modulo operations or loop around scaling values when you get huge numbers like 12304005.544.

Does anyone know of any other websites like this though? Because they are fun as hell.

demagogue on 23/12/2007 at 01:07

I know exactly what you're talking about. It's like the people that made that website had a great insight but didn't approach the full potential.

First, like you noticed, it deals pretty much just with melody, with no good input into harmony or rhythm.

For harmony, though, probably it'd have to be added by the user, overlaying a 12-tone harmony by hand (what the recorded versions did). I don't know if this is beyond the capability of a program or not ... serialist (12tone) harmonic composing is tricky even for persons who have studied it for a while.

But for rhythm, certainly they could have thought of something better than a 1-1 mapping with pitch and duration. Using a related or synergistic pattern is an interesting idea I had. I was sort of thinking along the lines that you were actually, something like the concurrent populations of two species that depended on each other, like predator/prey or host/parasite, etc, one rhythm the other pitch (so the rhythm starts to heat up as the prey is under attack, and slows as the predators struggle). Or another interesting way to play with that idea is to make a fugue with the related patterns being the different voices, so you can literally hear the relationship of predator and prey over time. :D

The "edge-trimming" is just normalization to pitch. I wouldn't actually think of it as an unfortunate side-effect. IMO it's core to how the thing works ... whatever pattern you want, you always have to normalize it to its new medium, so you preserve the pattern in a meaningful way. I think the site could do a better job scaling it though. Right now, I think every "big" number just gets tagged as the highest pitch, when it might be better if it telescoped it on something like a log scale to keep them manageable. But usually, if the pattern is shooting off to infinity it's probably not a good pattern to use anyway. The best kinds are chaos equations with strange attractors that keep the number hovering within a range and having a kind of pattern to it. And I'd agree with you that random sequences, like pi and sqrt(2), are no better than any random sequence as far as the sound you get, hardly worth the pizazz of "listening to pi".

But anyway, my more general thought is that this website is probably best used as a tool for composers that go on to use the melody tracks in their own compositions, without expecting too much from the raw output itself.

frozenman on 3/1/2008 at 16:13

(http://tones.wolfram.com/)

Here's something that might be extremely interesting - I've yet to have a chance to really explore it, but it's in a similar vein to the musicalalgorithms.ewu.edu link you provided demagogue.

This one generates music based on computational models of cellular automata. The computer I'm on is shit, so I can't listen to them unless I have them sent to my cellphone (which is really cool).