Contour Grammar vs. Fractal Cutting

Finally, having completed a basic implementation of Contour Grammar, I am getting to see what the idea sounds like in musical form. Indeed, some very interesting effects have been achieved thanks to the functional phrase language as well as the use of coherent random sources (which I will discuss at some point in another post). However, even this idea is lacking the coherence of Fraccut, and it bothers me enormously that such an elaborate idea as Contour Grammar can't even beat a relatively simple implementation of random fractal cutting, a rudimentary mathematical process.

Of course, it's very obvious why Fraccut still holds the record for best melodies. Since the cutting engine essentially starts with the chord progression as a seed, coherence is almost guaranteed. It's a powerful effect. Yet, the reality is that Fraccut is hardly aware of what it is doing. In a sense, the coherence generated by Fraccut is nothing more than a cheap trick stemming from the clever picking of the seed. Nonetheless, it undeniably works better than anything else. In the end, I am concerned only with the audible quality of the results, not with the route taken to achieve said results.

That being said, the big question now is: how can Contour Grammar pay closer attention to context in order to achieve greater contextual coherence? It would be nice to somehow combine the internal consistency of Contour Grammar with the external, contextual coherence of Fraccut.

Grammatical fractal cutting, anyone?

40.000

Today, the program hit 40,000 lines of code, counting the framework and all plugins. That's a pretty big landmark for mGen. Thirty thousand seems so long ago.

The most recent ten thousand lines have not pumped out as much sample quality increase as previous thousands. Rather, this batch of code seems to have been more focused on internals, setting the stage for more rapid development in the near future. I feel as though this focus on internal development will result in an explosion of high-quality samples sooner or later. In fact, I'm rather certain that it will be sooner, not later, since contour grammar is coming along nicely and should be fully-operational soon.

I look forward to many more such landmarks.

Contour Grammar Process Flowchart

The Contour Grammar engine, though still young, is already evolving into a complex system of data handling that will, if expectations are met, provide a great deal of both creativity and coherence in future melodic plugins.

The data structure, in brief, works as follows:

  • Words comprise an array of offsets
  • dWords comprise a word dedicated to pitch offsets and a word dedicated to rhythmic blocks
  • Dictionaries comprise many words or dWords
  • Phrases comprise a pointer to the base dictionary of the language in which the phrase is "spoken," a word that provides the rhythmic structure of the phrase (think: rhythmic shells technique), a dWord array that contains the words within the phrase and an array of lengths for the words; this last piece of data is only necessary if the structure word is omitted
  • Polystreams comprise separate arrays for the different elements of a musical note

It is far easier to understand the process through visualization:

Note that, if the engine is instructed to use the accent list (i.e. use the array of lengths for phrase rhythmic structure rather than a word), then this technique requires random or chaotic sources in two places. If the plugin uses words for rhythmic structure, then the technique requires a total of three such sources. This requirement provides ample opportunity for creativity, as we could try different combinations of random sources and observe the results.

Though work on Contour Grammar is progressing at a rather sluggish pace due to an unusually busy week, I anticipate a very productive weekend.

An Original, Realistic Algorithm for Terrain Erosion

Yes, another non-music digression. I can't stand not to write about this though, considering the fact that I've had a serious breakthrough today with a new algorithm.

Here's the problem: erosion algorithms tend to produce a sort of homogeneous soup of terrain without recreating some of the more true-to-life effects such as fractal coastlines and such.  The solution? A terrain density map.  As far as I know, this is a completely original algorithm.  I've seen nothing like it in the literature, so I'm quite excited about having invented it.  The initial test runs also produced exciting results!

Here's the basic concept: real terrains are not homogeneous.  Not all dirt is created equal. So, along with a 2-dimensional array representing the heightmap of a terrain, we should also create an equally-sized 2-dimensional array representing the density (or integrity, if the term is preferred) of the terrain at each given point. The density array can be filled with a low-octave perlin noise function (I found two to three octaves to give optimal results, with a persistence of between 1.5 and 2).

Now, we perform an erosion algorithm as usual, except that we use the density of the terrain at each point as the threshold value for erosion. That is, if the amount of potential erosion at a point is less than the density of the terrain at that point, then the point will resist erosion. Ideally, this algorithm erodes terrain in a more coherent, less uniform way than typical thermal erosion algorithms. For example, coasts display some fractal dimension now since some areas of the terrain erode more easily than others.

A sample heightmap using the new algorithm:

The difference is most notable around the rivers, where the coasts clearly display some fractal coherence absent in thermal erosion algorithms. Notice that "noise" of the coastlines is clearly more coherent than single-octave noise, thanks to the perlin-based density map.

I am still trying to work out how to make the features appear larger (that is, make the coast even more jagged), since the heightmap, although nice, isn't drastically different from previous ones. I am quite confident, however, that there's a lot of potential in this new density erosion algorithm. Who knows, maybe this will be the future choice algorithm for procedural terrain erosion!

Functional Phrase Language for Contour Grammar

Unfortunately, most of my work over the past week has been devoted entirely to the internals of Contour Grammar.  The format of the data structures will completely govern the quality of the plugin's future, so I wish to spend a great deal of time and thought on the layout of the grammar engine.

A minor breakthrough (which, oddly enough occurred in an airport) did provide a boost of inspiration. One of the current problems with Contour Grammar lies in the ambiguity in defining phrases. How should phrases be picked to maximize coherence? My newest solution to the problem entails a sub-grammar that I call functional phrase language, since it uses functions to indicate derivations of new words from parent words. In this way, rather than filling phrases with a random array of words, the program will fill the phrase with a random array of functional instructions that should ensure greater coherence.

Here's an example of what a line of such instructions might look like:

1,2,1.up.up,2.invert,1.maxdown

And here's how the Contour Grammar engine would read and interpret the instructions:

  1. Draw a random word from the active dictionary, label it "1," and add it to the active phrase
  2. Draw a random word from the active dictionary, label it "2," and add it to the active phrase
  3. Take the first word, translate all pitches up 1 step, then translate all pitches up another step, and add the resulting derived word to the active phrase
  4. Take the second word, invert the pitches (horizontal reflection), and add the resulting derived word to the active phrase
  5. Take the first word, determine the maximum pitch, then translate each occurrence of the maximum pitch up 1 step, and add the resulting derived word to the active phrase

If everything goes according to plan, Contour Grammar should be off the ground and getting cold, hard results by the end of the week. As I mentioned before, I really hope to have some unique samples up for February before it comes to a close. Audible progress is real progress!

Techniques for Heightmap Synthesis: II

I have made a great deal of progress today in my work with virtual terrains!

In the last post I mentioned bucket fill algorithms and how they could contribute to realism by creating isocontours on the heightmap. I have successfully written and applied my own tolerance bucket fill algorithm. Unlike most such algorithms, however, my version actually preserves a set amount of the original surface variation, so as not to create a completely flat contour - which, of course, would be unrealistic.

I have also written a perlin noise generator to give variation to the heightmaps. Perlin noise is simply a linear combination of noise functions with varied frequencies and amplitudes. In practice, perlin noise generators often use a coherent noise function, such as a high-frequency sinusoid. I chose, however, to use only a simple rand() function. By mapping each point on the heightmap to a point on the (increasingly large) noise maps and performing 2-dimensional linear interpolation, one can achieve the desired effect without having to use trigonometric functions.

Now, using an additive combination of Brownian random deposition walks, bucket fills, perlin noise, and softening, I am able to create much more realistic terrains.

Below are a few sample heightmaps created with the aforementioned methods:

A quick summary of what's going on in these heightmaps:

  • Random deposition walks create long, winding rivers and mountain ranges
  • Bucket fills level out parts of the terrain to create isocontours, which give the appearance of defined features like plateaus
  • Perlin noise randomizes the space in between features with coherent noise
  • Softening passes remove rough spots created by deposition and bucket fills

Techniques for Heightmap Synthesis

And now, for another deviation from algorithmic music.  As part of my research in building virtual worlds, I am exploring effective methods of creating interesting and detailed heightmaps for procedural terrain.

My earlier posts showed off some landscapes that were created using spectral synthesis and terrain deposition. I am now exploring methods that I hope will bring greater realism to the heightmaps. Having developed a set of algorithms that use Brownian motion (random walks) as well as certain erosion techniques to create features in the land, I began testing various heightmaps by walking over the resulting terrains.

Brownian random-walking deposition seems to create much more believable terrains than spectral synthesis or particle deposition alone.  Application of erosion and smoothing algorithms then reduces the sharpness of the terrains appropriately.

Below are a few of the heightmaps that were generated by my algorithms (note that the colors were chosen arbitrarily by the program):


All of the heightmaps were generated using custom-built random walk, erosion, and smoothing algorithms with varied parameters.  A "randomize" filter was then applied by a third-party texturing program, which helped create isocontours (plateaus, essentially) for added realism.  Though the filter is very nondescript, from what I can tell, it uses a technique much like a tolerance bucket-fill to "randomize" the image.  I will try to write my own version of this algorithm so that it can be applied automatically to the heightmaps.

While these heightmaps represent a great step forward in creating believable terrains (personally, I love number 4), they are still far from realistic.  Real land has far more intricacy and fractal dimension.  I will continue looking for new and better algorithms to create more realistic worlds.

Contour Grammar: Initial Results

It's been a long, bumpy road to the first functional mGen plugin written in c++. Nonetheless, tonight has seen preliminary results of the work-in-progress contour grammar system embodied in c++.

Contour grammar is, essentially, a marriage of two previously-brainstormed ideas: rhythmic shells and curve splicing. The name refers to the fact that, at heart, the engine is grammatical, using the method of rhythmic shells to handle words and phrases in context, but that words are defined using basic contours like those described in the curve-splicing method.

Unlike the grammar engine of GGrewve or GrammGen, contour grammar is functional in form, meaning that a word is not necessarily limited to a certain length. Rather than using an array of pitch or rhythmic offsets to characterize a word, contour grammar uses an array of coefficients that determine the properties of an infinite contour. Rhythmic shells then give precise instructions for transforming sets of these infinite contours into concrete, finite note streams.

Some expected advantages of contour grammar:

  • Arbitrary note size makes words extremely flexible (any given word can fill up any given amount of space, so no length-checking is necessary)
  • Rhythmic shells preserve rhythmic integrity of phrases
    • Shells still allow for variation by providing direct access to the words upon which the shell is built
    • Tweaking a single word's coefficients will change a specific part of the shell while preserving everything else, allowing for great coherence
  • Object-oriented data structures provide means of easily manipulating high-level variables that result in subtle, low-level changes
  • Very little conversion necessary between rhythmic shell and raw pattern
    • Rhythmic shell -> Polystream
    • Polystream -> Contextual Polystream (snap to key, duration *= quanta, etc.)
    • Contextual Polystream -> Pattern

So, in summary, contour grammar provides a flexible, coherent, and easily-variable data structure while still remaining concrete enough to require only a few minor steps to convert to pattern format.  In general, abstractions such as those made by grammar engines suffer from either a lack of coherence or a difficulty in de-abstraction (returning them to the base format).  Contour grammar, it seems, circumvents these drawbacks with a well-designed data structure.

Preliminary results with contour grammar already display some degree of coherence, a great deal of variability, and, most importantly, the potential for continued development and improvement.  February will surely bring some new, interesting samples!

Revisiting Artificial Neural Networks (ANNs)

Though I researched and toyed with artificial neural networks for a few weeks last summer, the speed drawbacks of AHK prevented me from pursuing a generative module that uses the technique of neural networks to generate output. Now, however, with the completion of the core pieces of a c++ library, I am able to leverage the speed of c++ in designing plugins. That being said, the time has come to revisit ANNs...this time, wielding roughly 100x the speed!

The basic theory of artificial neural networks is simple: create layers of "units" that perform single-function calculations, then create numerous connections between said units that carry input/output signals between them. Finally, sandwich the layers of units with layers of inputs and outputs that carry signals between an external interface and the internal unit layers. ANNs essentially rely on the ability of function compositions to quickly become chaotic when plotted against a varying input. One can, however, shape the output with user-defined parameters or any sort of evolutionary optimization run against a "training set" of data.

I spent the past two days writing a basic artificial neural network library in c++ to experiment with the applicability of neural networks to music. Currently, I am only playing with settings and observing graphical output in order to get a feel for the capabilities of ANNs. When I feel that I understand their behavior, I will map their outputs to music and see what happens!

Here are some output graphs from different neural networks. Notice that many interesting effects, including discontinuities, can be achieved with the composition of relatively simply functions (breaks in the curves are possible with either ternary or modulus functions).


The neural networks each contained 100 units (neurons) and 100 to 500 neural connections (synapses) between the units. A single input node and a single output node were used to communicate with the network.

MainBlock Importing Now Functional!

The c++ mGen Library is taking shape very rapidly. I am writing hundreds of lines of code each night in a furious attempt to get back on track with developing new and better plugins, since the month of January saw no sample clip activity. The MainBlock structure, set up to mimic the identically-named central data structure handled by the mGen framework and used to pass data to external plugins, now has full loading capabilities. In other words, output from the main framework can be read into this extensive c++ structure.

One might ask why on Earth managing to read one's own data structures should be applauded. The answer lies the differences between the AHK and c++ languages. In AHK, which the main framework uses to manipulate and write the data, the information is stored, essentially, as one huge string. I created special functions that encoded the strings in certain ways to allow them to function more like objects, but these did not actually extend the functionality of AHK, they only faciliated the organization of data within strings. Bringing the information contained within MainBlock into a c++ structure requires decoding a large, messy data string and parsing it, using the parsed data to create real data objects within structures that can be directly manipulated without string hackery.

The main point here is that MainBlock importing represents the first step in allowing external modules written in alternate programming languages to communicate with the central mGen framework, written in AHK. The next task will, naturally, be the ability to save a c++ MainBlock structure to the encoded string format that the framework expects to receive back from modules.

A visual of the exciting moment in which a full .gds MainBlock was imported into a c++ program: