Generating Good Dictionaries

Glide has come under a lot of fire recently for lack of both creativity and coherence. After extensive testing tonight, I came to the conclusion that randomly-generated dictionaries have very little uniqueness. In a series of many renders using specific dictionaries, I was unable to distinguish between compositions that used different dictionaries. However, upon constructing a custom dictionary, the difference immediately became apparent. There can be only one conclusion: the dictionary generator needs a lot of improvement.

But how does one create good dictionaries? That's really the matter at hand. The generator used by Glide was never meant to be the final version - in fact, as I recall, it was simply put there as a placeholder, but I ended up relying on it anyway to make dictionaries.

This problem is really just another manifestation of the "big" problem in algorithmic composition: how can we create data that is both original and coherent? A balance struck in the dictionary-creation process will be a balance struck during runtime and, by extension, a balance struck in composition.

Here are a few ideas for adequate dictionary generation:

  • L-systems with random production rules
  • Fractal cutting spaces
  • Random permutations of a base dictionary (wait...do I smell recursion?)
  • Root entropy

I hope to test each of these methods in the coming weeks in order to build a successful dictionary-synthesis tool for use with Glide.

Serial Strings

I made an important addition to the MainBlock data structure today: a composition serial string variable. It's a very minor addition, but it will provide important functionality in the future. The serial string is really just a unique ID by which a composition can be identified. The string is alphanumeric, weighted in favor of characters so as to make the strings easier to type and remember. Currently, strings are 10 characters long. Here are some examples:

FREJ2SM197
VHOA5T4VEJ
H9F8EO96LJ

And now the question: so what? Well, here's the idea behind serial strings:

Many plugins make random decisions at runtime, such as weighting probability distributions. These decisions change between each composition, so the same plugin will act differently even if the configuration stays the same. In this way, each composition has a unique "personality." Using the serial string, plugins can create detailed databases that catalogue the settings that were used to generate a certain composition. It's easier to understand the practicality of this by example.

Let's say the user has a future version of mGen, which includes some kind of "wrapper" module that renders many compositions repeatedly and then, after accumulating about an hour of music, burns them to a disc, so that the user has a full CD of random music. Now, suppose the user is listening to the CD in his or her car and stumbles upon a really nice composition. Obviously, he or she will want to go back and figure out what was running through the program's "mind" when the composition was made. What generative modules were used and with what confgurations? More importantly, what unique decisions were made by each plugin at runtime that made the composition so great? By taking the serial string (which could be part of the composition's file name on the CD) and asking individual plugins or the general framework to pull up history files, the user could then see exactly how the composition was generated and, perhaps, discover the settings that produced such a piece.

User feedback-based evolutionary modules constitute an even more exciting application of serial strings. If the user likes a composition, he/she can simply enter the serial string into some kind of feedback system (which may be part of mGen as a whole or of individual plugins) to let the computer know that it did a good job with that composition. The computer can then try to figure out what settings result in good feedback, ideally creating an evolutionary system that gets better with time.

So many possibilities are centered around a single string! But the serial string is more than just an array of characters. It is the only precise key by which compositions can be remembered and identified - the only unique trait, other than sound waves, that separates a composition from thousands of others.

Long-Term Ideas

Multi-Module

  • A plugin that has its own plugin slots (nested plugins)
  • For example, a drum/bass multimodule could have the ability to load two plugins (one percussive, and one bass), and be able to have tighter control over coordinating them than the main framework would have if they were separate plugins
  • Think post-processing, except during the rendering process
  • Acts as an overseer

Post-Renderer

  • A macro that does something with rendered mp3s
  • Adds the song to iTunes library, for example

Unified Core Module

  • Alternative to structure -> coordination -> progression workflow
  • Provides a unified core interface
  • Could be more coherent

Unified Production Module

  • Overarching module that replaces the unified core module as well as all generative modules
  • Decides (algorithmically) which plugins to call and how to call them
  • This is the final step in making mGen truly random - no more user configuration of modules
  • Does NOT replace the renderer/post-renderer

Problem with MainBlock Data Structure

Well, I've encountered a rather nasty problem with the MainBlock structure tonight during my attempts to get the new renderer working.  Here's how the structure works at the moment:

  • Each generative module is allowed to output an unlimited number of generative tracks, each with its own patch setting
  • The main structure keeps track of how many generative modules are loaded
  • The main structure keeps track of how many generative tracks have been outputted (>= the number of generative modules)

Unfortunately, this (poorly-thought-out) system creates a problem: generative tracks are "orphaned" from their parent generative modules.  That is, there is no way, unless the module-to-track mapping happens to be 1-to-1, to determine which generative track belongs to which modules in a given data structure.  Thus, the renderer cannot figure out how to assign instrument information to the tracks.

There are a few ways around this:

  • Enforce strict patch output in modules
    • Limit the "instrument" field to a certain number of instruments (piano, guitar, etc)
    • Default to piano
    • Eliminates the need to know the track's part
  • For each module, record the number of tracks outputted
  • For each track, record the ID of the parent module

The first fix, of course, is the most intuitive.  Tracks should, ideally, be completely independent of the modules that created them.  Unfortunately, this would require recompiling almost every plugin in existence, making sure that they all output proper patch information.  The other two are rather simple but hacky fixes.

Focus on Rendering

Considering how busy I've been, it's no wonder that the past few weeks have seen few posts.  But I've come out of them with a new goal: to get the renderer working again.  At one point, mGen was able to go from nothing to a polished mp3 with a single click.  I miss those days!  Having to render each composition manually is quite a time-waster.

Building the renderer isn't much fun, since it requires constructing software-dependent macros that are anything but intuitive.  Macros, considering their fragile nature, have a tendency to be extremely buggy and difficult to code, since they operate at the level of input control.

Still, re-orienting myself towards the original mGen goal of "one-click compositions" will save a lot of time in the long run.

Ideas for Melodic Interpolation Engine

Here are a few ideas concerning melodic interpolation that I've been collecting in my mind over the past few days:

  • Certain blocks in the blockspace should have an ANCHORED flag so that they cannot be bumped vertically
    • I find that it is often the first and last notes of a phrase that really establish the tonality...perhaps they should both be "anchored" to an element of the chord?
    • Children of anchored blocks should not be anchored
  • Why not give blocks some inheritable properties? This will make the melodies less homogeneous (much like Fraccut, which tends to have, for example, short notes grouped together)
    • BLOCKINTERPCHANCE - when calculating the chance of an interpolation, use BASEINTERPCHANCE + BLOCKINTERPCHANCE
    • BLOCKVARIANCE - when calculating y deviations, use BASEVARIANCE + BLOCKVARIANCE
    • Grammar probability distribution - when calculating grammatical replacements, maybe certain blocks should be more likely to be replaced by certain words? This would have the effect of grouping words together more often. Is this a desirable property?
  • Make sure that blocks will not split if a split would create a block with a width below MINIMUMWIDTH
  • A style should include the following:
    • A grammatical dictionary
    • A grammar probability distribution
    • Intrinsic probabilities and quantitative measures that aren't part of the configuration interface
      • Like what? Need some more variables to work with.
  • To mutate a style over time, simply make small changes to the probability distribution or the other quantative measures
    • Probably not a good idea to mutate words
    • Maybe we could add new words to the dictionary
      • Add a new word
      • Slowly increase the probability of the new word
      • Slowly decrease the probability of another word (that will be phased out)
    • In this way, the dictionary undergoes slow, smooth, and continuous changes

Glide

Glide, a new plugin based on the recently-explored method of melodic interpolation, will bring a new meaning to coherence in the mGen plugin library. As detailed in the previous post, this method leverages the coherence of a Fraccut-like block/space format, while leaning on an underlying grammar engine for variability.

If current performance is at all indicative of future payoffs, Glide is here to stay. It has already pumped out several extremely impressive compositions in which the melodies display a coherence unlike that obtainable with other plugins - even Fraccut!

The real long-term challenge with Glide will be extracting enough creativity while maintaining the underlying coherence. With only simple melodic interpolation, the algorithm is a mathematical system at best - which means little creativity. Building the blocks on top of grammar will certainly help. Still, it will no doubt be a challenge to get Glide to come up with highly original material. Unlike most of the other plugins, which always border on sounding aimless and uncoherent, Glide is most in danger of sounding predictable and repetitive.

Long-term goals for Glide:

  • Develop a method of reliability generating high-quality grammar dictionaries for use with the grammar underpinnings of the plugin
  • Build distinctive style settings that differentiate melodies
    • Ideally, we shouldn't be able to tell that two melodies produced with different style settings were even created by the same plugin
    • This entails carefully monitoring any "artifacts" that crop up as a result of the mathematical nature of the algorithm, like those that did with Fraccut
  • Create an evolutionary system that slowly evolves styles to give the plugin a dynamic feel over long periods of time
    • Rather than evolve as a function of executions, evolve as a function of absolute time to ensure that the plugin does not become stale quickly
    • Imagine putting mGen away for a month and then coming back to find that the same plugin has an entirely different feel!

Melodic Interpolation: Simple but Surprising

It's amazing how the simplest methods can sometimes yield the most impressive results.  I have been experiencing this phenomenon all day.

While messing around with new methods, I actually discovered something very simple and very obvious: interpolation of melodies based on the chord progression.  It's something that I haven't used before - probably because it seems too obvious!  Here's the procedure:

  • Start by placing "blocks" in a space; place them such that they lie directly on top of the roots of the chords in the progression
  • Link the blocks such that each block is aware of the spatial indices of it's two neighbors (in back and in front)
  • Iterate through the blocks
    • Calculate the mean y/pitch offset of block i and its front neighbor
    • If the mean offset is different from both constituent offsets
      • Split block i and position its child at the previously-calculated mean
      • Update the neighbor links for block i, the child block, and the front neighbor
  • Convert blocks to notes
    • x -> time offset
    • y -> pitch offset
    • width -> duration

One could think of this as "blurring" the melody - removing sharp jumps where they exist.  It's really just interpolating the melody from the chord progression.  Now, throw in a grammatical foundation and some variation functions and we've got a very nice method.

Surprisingly, this simplistic technique sounds good!  It sounds both deliberate and coherent, with a dash of creativity thrown in.  By continuing to improve the method with a grammar, I believe that it will be possible to make the method of melodic interpolation a truly viable way of generating good melodies.

Additive Rhythmic Grammar

As part of a larger scheme for building more coherent melodies, I'm looking into a better grammatical method for representing rhythms.  The additive rhythmic grammar technique may pave the way for better rhythmic coherence in future melodic plugins.

The general idea is to create very simple rhythmic words.  Then, define probabilities for each word - for example, the probability that the word will occur in the first, second, etc. measure.  These probabilities should, for maximum coherence, be either very large (~1) or very small (~0).  Finally, when retrieving the total rhythmic format for a given measure, simply "add" each rhythmic word that has an adequate chance of occuring in the measure.  The key here is that we can treat the words as simple mathematical entities that can be summed, subtracted, and have basic binary operations performed on them.  In this way, it is possible to create a complex rhythmic pattern by using only simple building blocks.

Yes, in reality, it's a very simple idea.  Nonetheless, I've never tried such "additive grammar" before.  Anything is worth trying.

Fraccut: Third Generation

Introducing Fraccut v3, the next generation in fractal cutting.  Fraccut has always been a solid plugin.  Unfortunately, it's also been a relatively slow plugin due to the volume of information it handles.  AHK heavily bottlenecks the data transfer since the language lacks OOP capabilities.  With the c++ library in place, it's time to give Fraccut a serious speed boost.

Beyond the performance gains, the third generation of the Fraccut engine will also leverage the power of the mGN Library's RandomSource abstract interface to draw random numbers.  This should allow for much greater flexibility in configuring the random component of fractal cutting.

Some additional features that Fraccut v3 will aim to support in the future include:

  • Context-sensitive cutting parameters
    • Pay attention to phrasing
  • Optional grammar conversion (use the cutting blocks as controllers for a grammar, see Fractal Grammar)
  • Incorporation of Markov engine for added stylistic coherence
  • Loading/saving Markov styles
  • Linkage with gMSE for advanced style options