From Films to Games, From Analog to Digital
Two revolutions in multi-media!

From Films to (Digital or Computer) Games–during the last 20 years there has been a revolution in films, indeed, in multimedia. This revolution is the shift from analog to digital methods of recording and manipulating sight and sound.

However, it is interesting that the development of computer games in many ways followed that of film–only what it took film 80 years to do it took games 20.

In the history of films, we saw how music and sound have been mis-perceived and mis-understood. "Silent" films were never silent, and in fact, much of what is commonly believed about sound in early film is false. The facts are that sound and music have been a major part of films since films began. The same is true for video games.

Take for instance the JAVA reincarnations of asteroids. Asteroids was one the original and most popular early console computer games. It has been recreated, almost verbatim (identically) in JAVA. However, what is missing is the original sound! And what an important element that is. The interesting fact is that a reason the sound is missing is the SAME reason sound is missing from early film!

FILM and VIDEO GAME parallels

Due to technological limitations with early film there was no way to record and synchronize sound to film. The same is true to asteroids. While the game is digital–that is the graphics are a software program running on a ???? chip, the sound is not. The sound for the original Asteroids was analog! That is to say, it was hard wired–the sounds were made by electrical circuits which were triggered by the computer code, much like how an image in the film might trigger a piano player to play a certain piece of music.

Like in film, sound in video games often gets the last consideration, not only is it afforded the smallest budget, but also the smallest space–and space (memory, that is) is what counts digitally speaking. However, this limitation caused some interesting results, as in Asteroids.

Asteroids is a classic, the classic example of sound design. The sound never ends, but it does always build–it adds to the tension. Just like in film, this sound plays a psychological role.

Background Music–the original techno music? Certain types of games, those where you move from level to level, but are essentially doing the same thing over and over in a never ending quest, screen games which fall in to Action, Fighting, and Puzzle categories (Donkey Kong, Tekken, Tetris) and also some Sports Games (Pole Position or Sega Golf), use a music that loops, is upbeat and highly rhythmic.

Asteroids took this one step further (before these games were even invented) by matching the music with the action to a certain extent, as more asteroids came on screen the music speeded up–the "music" was actually a fairly good rendition of your heart beat.

This is no accident, music, after all, is all about rhythm and what are our bodies but rhythms machines. When you take a breath, what is that if not an upbeat, when you are excited your heart beats fast and sometimes irregular, even your blood and nervous system make noise (but interestingly this noise is quite a bit different, how?). It is no coincidence that most music has tempi that lie within the normal beating pattern of the human heart–between 60 and 80 bpm.

[Game categories: Action (arcade games, subcategories: horizontal scrolling, maze chase, platform climbing, "raining shit"/breakout), Adventure and Role Playing, Fighting (third person, first person, first person shooter), Simulations, Sports, Strategy.]

This connection between Techno (also electronic, ambient) and video games is interesting. Video games are truly a postmodern expression, that is they are global without boundaries–there is no culture of ethnicity or nationality with video games, or at least such distinctions are fading–this is true of techno music as well. Where did techno begin?

===

Types of Sound and Music

Why didn’t games like Asteroids, or Space Invaders make it into the home console game set, why aren't they popular today? There are two main, and related reasons. First, Consider the sound of the Alien Ship in Asteroids and Space Invaders, what does it sound like? It is a siren, an alarm. One function of an alarm is to warn, "watch out" it says, but another function is to call attention–"come look at me" it says. When you place a video console game in a large room with 50 other VCGs, one way to attract others to the game is with sound–with an alarm. Despite Asteroids rather primitive sound, technologically, psychologically it was brilliant. It called people over to the machine to play it, it attracted attention. This is not necessary when your at home, in your living room. At home, an alarm is something that wakes you up in the morning, on a video game it became annoying.

The second reason is that console video games were designed for repeat play…play it often, keep pushing coins into the machine–essentially they are video Pachinko. But at home, once you've bought the game, there is no more money to be made. Console games have no end, because if they did you might stop playing them, home games have an objective, an end.

Notice how this affects the sound. You are much more likely to have sounds that create and add to your tension of game play in a console game–the ever increasing speed of Asteroids, the increasing density of sounds in Defender, are designed to make you screw up, to have you choke so you will pump more quarters into the machine.

The game Sinister was brilliant in this regard. It was one of the first games to use a human voice, a deep voice that said "Run, Coward, Run. Run.!" Not only did this attract the attention, called people in the Video Game Parlor to the game, but it scared the shit out of you, because when it came for you, you had better run!

Home games gravitated toward light non-threatening background music. Just to keep it interesting, when Mario jumps to the next level, the music changes, but it's basically the same happy friendly music.

However, just because home games tend to have objectives does not mean you spend less time playing them–in fact, you will most likely spend more time. This is particularly true of games with interactive story lines. What does this mean for the music? Well, you have a problem with repetition, there is only so much space, even on a CD, so music and sounds will be repeated and looped.

It is actually quite ironic, computer code has created the KEY element of the ultimate multimedia experience–what is that? Interaction. You are now in the story, in the game, not just watching it passively on the "big (or little) screen." This interaction creates many possibilities, many ways of experiencing, and we thirst for that–but for the sound and music, the irony is that the same technology "freezes" these elements. The music and sound never change. Beethoven sounds a little bit different every time you hear it live–different orchestra, different conductor, different hall, granted the differences are slight and perhaps only true experts notice all the differences, but this is why LIVE music can still draw audiences in for the same price of 1-6 CDs (in which, theoretically, you could hear the same music over and over an infinite number of times).

This interactive music (called "i-muse" in the industry) typically is of a "mood music" type where the quality of the music alters depending on the mood or action in the game. This is the kind of music typically used in Adventure/Role-Playing games.

Myst was brilliant in this aspect, because for the first time, music was used as a signifier–that is it let you know when something bad was about to happen or when there was something mysterious you might want to look at (just like the scene in Indiana Jones, the low bass…tension rises…we know we are looking at bad guys, or at least that's what we think).

The other extreme of background music in VG is Level Ambient Music. Where changing levels changes the music.

One of the musical issues involved is how you get from one state to another. Of course, the easiest way is just to jump music. But there are two other options: short bridge, like a drum riff or even just silence, or actually morphing the music (Banjo Kazooie).

Again, these more complicated techniques require more memory…

Technical Issues

Early sound effects in video games was entirely a matter of programming (even games like Asteroids , the sounds were still programmed, just not in code, they were hard wired programmed, that is programs in circuits–computer code for making sounds often mimics the physical circuit design, same algorithm). All you had to work with were four voices on an FM synthesizer chip (incidentally, do you know where the money from that chip went? well…). There were no recordings, no samples because memory was expensive and there wasn't enough of it in these early games.

Due to the limitation on the number of sounds you could have at any one time, some sound designers tried very outlandish things. Rob Hubbard would do "outrageous harmonics and stuff, a lot of weird bitonal stuff and parallels at the bottom…" [Pet Sounds]

Interestingly, the same thing happened in Video Games as in Film. When consoles hit the market, there was an explosion in wealth. Game companies developed in house teams, music was written in house, game cartridges only played on one kind of machine (just like MGM films only played in MGM theatres), and things became standardized, sounds and music became routine, and the inevitable happened, the industry crashed. Out of the ashes, came more outsourcing (independent contracting) companies writing games for multiple consoles, more attention to quality, etc. (incidentally, this coincided with a technological leap in graphic resolution and speed: can this compare to the technological advance in film, the introduction of color?)

But then there was the switch from cartridge to CD, a shift from 32 megabytes of memory to over 600! What this meant was now samples could be used, recording could be used. Sound was no longer made by synthesis (i.e., synthesis is creating the sound from scratch, using an algorithm to generate a sound–mimicking a physical process with a mathematical formula) but was played back. Therefore, you didn't need a degree in engineering or computer science to make the sound. Real composers began to be contracted to write real scores.

***there has been a shift from synthesizing sounds to playing back samples–why? because samples are more realistic (and we had better technology and so more space and speed to use samples), but now that chips are getting really fast, the question is could we go back to synthesis? What will the future be? Could going back to synthesis solve the problem of "frozen" music? Well, it might at least solve the problem of "frozen" sound, but to solve the problem of frozen music you would need to be creating new music as you play–this is possible, in fact this is a huge area of research in Computer Music, so called Algorithmic Music which crosses disciplines with Artificial Intelligence, Neural Networks, and other emerging fields to develop "smart" computers–but the problem is music is really complicated! Most attempts to have a computer "create" music produce music that is inferior to what a person could do.

AND now, we will have games released on DVD..

Who writes the Music

What had these composers been doing before? When directors and producers looked for early film composers they looked for them in the largest most popular multimedia experience that existed, Opera. The same is true for Video games, only the largest most popular multimedia experience before video games wasn't opera, it was film–the first video game sound and music guys came from Hollywood.

This was no small business! Take for example George Alistair Sanger who did the music for Wing Commander and Seventh Guest. He may not be as famous as Madonna or Garth Brooks, but he has sold more music than either of them.

And just like in Film, the VG industry opened its doors to more eclectic forms of music (only the VG industry was much much quicker to do so, it is because the technology also advances much much quicker, is this a result of Moore's law?*), not just film composers, but pop musicians, even no-name kids in basements began to be contracted to write music.

And again, what happened in Film happened in VG. Just hit films spawned hit songs and soundtrack CDs so do games. In fact, Sega, even started its own record label, "Twitch Records," had its own studio and because the studio was so good, artists having nothing to do with VG wanted to record there, so Twitch began releasing records/artists that had no direct connection to VG.

You begin to see that the boundaries blur. Where does the multimedia experience end? Is this a film or a game? Is it all just entertainment?

Some have speculated that the entire entertainment industry: film, games, TV, radio, and fringe entertainment (even Porno to an extent) will converge, that it is already doing so. They further claim that the internet will be the catalyst, the place it will converge in. And that the only barrier right now is bandwidth. When you can stream a film at 26 fps full motion picture resolution, then things will get interesting. True? Is Apple's iMovie the beginning?

Take a look at the Myst game again, this time with sound. What do you think?

For a History of Video Game Music, see:

http://gamespot.com/gamespot/features/video/vg_music/

Incidentally, what country produces the most and best video game composers? Japan! Obvious, right? Just take the JR train and bring your ketai along and you will know why.

Nathaniel Phillips
Keio University

Read this article:

http://vgmusic.com/vgpaper.shtml

NOTE: While this is a very interesting and useful article, please read it with a CRITICAL mind, i.e., DO NOT believe everything you read, ask yourself "do I agree?" And feel free to disagree!

Edited excerpt from http://web.sfc.keio.ac.jp/%7Enathan/Sound%20Environment/Class%207-Games

Original title of this article: Class 7: Theory of Sound Environment

GAME AUDIO