Preface:

I originally wrote the following document mostly for myself and my near colleagues, as a kind of overview of different music playback systems available for developers of PC games. There is a lot of big talk about "interactive music" these days - it seems to be one of those buzz-words that marketing people love, and journalists' eyes go all twinkly at the mere mention of the phrase.

Interactive?

Firstly, I would like to say that I am a big fan of interactive music in games. But I would also like to point out an important issue that a lot of marketing people, journalists and even game producers forget: Interactive music in games can never work exactly like it does in the movies. Why? Well, it's very simple, and here it is: In the movies, the music suggests what is going to happen, not what has already happened.

In other words, when the bad-guy appears around the corner and starts shooting at you, it's really too late to say anything about it musically. Listen to the music in movies and tv dramas, and you will notice that the music always tells of something that might happen soon - not what has just happened.

Still, we have to try. Of course, although the music may change a bit too late, that is still better than having it not change at all.

DirectMusic

DirectMusic (TM) is a system developed by MicroSoft, and is probably the most feature rich way of creating interactive music now. It was made for PC, but being a Microsoft product, I would be very surprised if the same system didn't also work on the X-Box. (At the time of writing this, X-Box audio features aren't clear yet).

DirectMusic can be interactive in many ways, and the game code can control various aspects of the music. It can jump to different parts of the song - it can change the tempo, change the key/chords of the music, and even play little extra "tunes" (called Motifs) on top of the background music.

I'm actually working on a project in DirectMusic right now, and although it's a bit complicated, I'm quite enjoying it. It's fun, and it's perfect for creating music with that "classic" game sound.

The latest version of DirectMusic (for Direct-X 8) can also do neat things like a low-pass filter on the instruments (to create those cool analog-sounding synth-sweeps, etc.)

Seeing as how DirectMusic is done with sequences and samples, it will never sound as "polished" as a full-on CD recording. You can't have a real guitar solo in there or anything like that. It's basically "chip sound", but with samples.

DirectMusic also requires very close connections between the game code and the musical content, and therefore also needs very close cooperation between the musician and the audio programmer. A lot of the changes in the music requires both that the musician creates the actual content, and then the audio programmer has to implement it into the game.

Because of the complexity, it probably also takes a bit longer for the musician to create the music, compared with straight-forward music.

If you are interested in DirectMusic, please check out this DirectMusic demonstration that I have made.

Tracker music

Another way to make "interactive music" would be to use a so called "tracker" program (like "FastTracker 2"). "Unreal" is a good example of a game that used this system (great soundtrack by Alexander Brandon). The "tracker" is a music composition tool that lets you store a set of custom samples in memory, and use a set of sound channels (much like a multitrack tape) to trigger these samples at various volumes, pitches, and other special effects. The music can then be built up in blocks of 2 or 4 bars, and these blocks can be played in a different order, depending on the game conditions. This obviously will not produce any real different music every time - afterall, it's going to be the same snippets of music every time, just played in a different order. But it does at least give you the option to, for example, let the music take a more sinister mood when the player starts to move his armed forces - or whatever.

The downsides to this system are: (1) although it sounds far better than General MIDI, it still doesn't sound quite as big and glossy as a CD track or audio streaming .(2) Writing the music in this format is time consuming; as it for the most part has to be "programmed" with numbers and letters, rather than recorded with keyboards and guitars. (3) It requires a fair amount of programming to implement this system into the game, especially if you want it "interactive".

I was actually working on a big interactive soundtrack for a game called "Pig Detective", using this system. Until, unfortunately, the whole game was cancelled, and all my work had been in vain (I hate when that happens!!!). The code for this music system was already up and running on the PlayStation, but not on the PC yet - and I guess they never got around to it before the game got cancelled. But if you want to hear what some of my "Pig Detective " music sounded like, created with "samples, patterns, numbers and codes", then go to my credits page on http://www.lynnemusic.com/credits.html and listen to the mp3 samples from "Pig Detective".

Audio Streaming

Yet another way to do "interactive" music is to use audio-streaming of multiple streams. The musician would then compose the music using all his music gear, full studio setup; he can play guitars, bring in singers, real violin players and whatever time and budget allows for. This music is then sampled into long sample-files at, typically, 22khz 16-bit stereo. The sample-files are stored on the CDROM or harddisk, and played from there to the speakers, via a RAM buffer. When the game condition changes and different music is needed, a different sample can start to stream from the CD/harddisk, and be crossfaded into the old music. (I.e. the "old" music fades out at the same time as the "new" music fades in, over a second or so). For the two music pieces not to artistically crash totally with eachother and produce unbearably disharmonic results, all the music must of course be written in the same basic key, or, alternatively, be similarly dissonant. But this doesn't represent any major problem if the composer knows about it from the start.

The up-side about this is the fact that it sounds great, and that it doesn't require a whole lot of programming. The downside is the fact that the audio files take a lot of disk-space, so you can't have this in downloadable games; you can only use it if the game comes on a CDROM. For 22khz 16-bit stereo audio streams, allow approx 5 MB per minute of music. For better sound quality, go with maybe 26 Khz or even 32 Khz. If you are an "audiophile" and only the best sound quality is good enough for you, then go 44 Khz - but this will cause a greater strain on your CPU overhead.

General MIDI

Of course, there are still other music playback systems. General MIDI is one. It's getting a bit old actually, the only up-side to this system now is the fact that the files are really small, and I belive playback is already built into Direct-X (?) although I'm not absolutely sure about this. But it sounds dated and cheesy. Most people these days would expect General MIDI music only on low-price budget game releases and perhaps kids' games.

Redbook audio / CD audio

Finally, there is redbook audio (which is just a fancy word for "normal CD audio") and this is probably the most common format these days. Up-side: It sounds great, the only limitation is the quality of the producer and his budget / quality of studio equipment. Down-side: It is totally linear - with no room for "interactivity" - and it takes a lot of disk-space (Allow approx. 10 MB per minute of music). One other "gotcha" about playing normal CD audio tracks is the fact that the entire PC "halts" for a moment or two when the CD drive seeks to a new track on the CD!

So what's best for us and our game?

Here's an important point... before a composer is hired to write music for your game, you need to figure out the music playback system. If the composer starts writing something for audio-streaming, it can't be "transformed" to tracker-format or General MIDI or whatever later on. So if the composer is in the middle of the project, and the music playback system changes, the composer is basically back to square one and he'll probably charge you for his wasted time. Or at least, he should.

So it's a good idea to figure out the music playback system first. Ask yourself these questions:

1. Is the game going to be on CD, or is it going to be a "downloadable" game? If it's going to be downloadable, forget streaming and CD audio - go for DirectMusic, General MIDI or tracker music.

2. What's more important: How interactive the music is, or how cool/professional/glossy the music sounds? If the professional/glossy sound is most important, forget DirectMusic and General MIDI - go for audio streaming or CD audio.

3. Does a lot of music have to be written over a short time? Will the composer be in a hurry? Then do him a favour and rule out "tracker" format. It takes a lot of time to write music using numbers and letters.

4. Do you want to pull out all the stops and get yourself a soundtrack that sound just as good as the TV series Babylon 5, or The X-Files? Then go for redbook CD audio, or at least audio-streaming with a high sample rate (say, 32 Khz, 16-bit, stereo). Forget all other formats. And make sure you get a great composer :-) - not to mention a budget that can pay his asking price.