Sound is of great importance for how we experience moods and narratives. Especially so in the cinema where the visual and the auditive are linked together in various ways. This audio-visual interity is called synchresis, a central concept in film theory.

The function of sound in a movie is to affirm the moving pictures, thereby animating them. One can speak of the synchronization of the senses, where picture and sound go hand in hand and thereby intensify one another since, a part of our reaction to what we experience is based on our perception of sound and picture as a unity. At the same time we know that the sound is recorded separately in order to make it possible to be treated more freely when editing and mixing.

That this adding of sounds is so extremely essential for the narration can, among other things, be explained by the fact that film, to a great extent, is a question of movement, and movements are felt more natural if combined with sound. It is due to mankind’s hereditary perception – that sound is created by movements, in other words, every sound rises from an original source. This conception of the surrounding world can be connected to a pure instinct for survival, an ancient ability to learn from sounds about threats, prey etc.
When sight – man’s most important sense when receiving information – has defined the source of a sound, this information is not worked on in the brain; instead these incidents of movements and sounds are accepted as one simple event. This audiovisual, perceptual entirety is called ‘synchresis’, by the French theorist Michel Chion (1990/1994). The word is a conceptual fusion of ‘synchronization’ and ‘synthesis’. In the following I will develop Chion’s conception of ‘synchresis’ to say more about the significance of the sound in movie and cinematic narration.

Chion’s basic theory is that the movie and its viewer sign an audiovisual contract implying that the audience “agrees” to ignore perceptual inequalities between film and reality. The moving picture on the screen and the sound from the loudspeakers is perceived as one indivisible unit. This is not a matter of course. In the childhood of the sound-film such a convention-contract was not established. The audience could feel that the people on the screen were moving along with the synchronic sound as in a ballet.

As soon as sound and movement are interpreted as one event a fusion is born in man’s consciousness. The sensory impressions are overlaid so that picture and sound contribute to the nature of each other. If the picture shows a door being closed and the sound from the loudspeakers presents a heavy bang the audience imagines having witnessed how the door is in fact slammed. This kind of sound-work can sometimes be necessary since many side- scenes are rickety enough to start swaying seriously if a door were to be really slammed. Another reason could be that the director during his supplementary work realised that something extra was needed to point out that the door-closing person was angry and asked the sound editor to cut in a heavy door-bang.

Perceptual synchresis
Movements can be emphasized and made clearer by the soundtrack, which guides the attention to that which is of dramatic importance. In Jacques Tati’s Mr. Hulot’s Holiday (Les vacancy de monsieur Hulot), 1953 there is an example of how perception can be guided by sound. In two scenes of a similar kind, two card-players are sitting in the foreground while a lively match of ping-pong is taking place in the background. In one scene sounds from the latter dominate and attracts your attention. In another there are no sounds, though you can see them playing in the background and your attention is instead guided to the card-players in the foreground.

Film makers often want to elucidate and strengthen silent movements. The whining of a karate kick is in reality barely perceptible. But in filmic situations it is now an accepted convention to accentuate the movement with volume and force. Another example of accentuating is to make small movements more dramatic by sound-effects, as when Gestur draws his throwing-knife from its sheath in Hrafn Gunnlaugsson’s When the Raven Flies (Hrafninn flygur, 1984). The vigorous swish-sound tells you the knife is huge and thus extra dangerous. Perceptual synchresis also makes it possible to speed up the cutting-rate without making the action inconceivable. This has been carried to an extreme in cartoons. The high speed is made possible by accompanying every movement with a sound-effect or a musical loop. And when sight and hearing are in accord we can process more information.

Emotional synchresis
Emotional dimensions are found in all sound-narration as sounds have the power to create emotional associations. The sounds are worked upon to produce the right kind of feeling in a certain scene. The emotional aspect of synchresis is essential in any narrative film. The actors’ voices are often manipulated for better accordance with the character of his or her part. In the warm and gentle woman every harsh sound in her voice is taken away to give the voice a pleasant flow. That was for instance the case with Debra Winger’s voice in James L. Brook’s Terms of Endearment, 1983, to intensify the mild disposition of the role actress. An unpleasant person can likewise be still more so if the timbre of his voice is worked on. In Jonathan Demme’s Silence of the Lambs, 1991, Anthony Heald’s affected tone of voice of the psychiatrist Chilton helped in making the spectators experience him as a disgusting character. Also, footsteps, the rustling of clothes; car-sounds from the hero’s car among other things can be manipulated to support various dramatic needs.
The possibilities of going too far on the sound-track are considerable.

Especially in genres where a rapid flow of information keeps us constantly busy receiving the swift torrent of impressions. As a result there is not always time for us to judge whether something seems correct or not. Completely improbable sounds can pass unnoticed, as when Linda Hamilton in James Cameron’s Terminator 2 – Judgement Day, 1991, is running barefoot in a hallway and we hear the sounds of her clattering shoes! Unrealistic synchresis is a distinctive mark for the genre of horror movies where even the camera movements like zooms and tilts can be accompanied by unpleasant sound-effects.
>> Listen to the footsteps

An example of heavy synchresis is found in Hitchcock’s The 39 Steps, 1935. The sound we hear of a terrified woman screaming, at the finding of a corpse, comes from a howling steam-whistle. Since drama and intensity are great, we would not have thought her scream unrealistic had Hitchcock not shown the source of that sound by cutting in a steam-engine afterwards. To enable this form of unrealistic synchresis to work, with strong diverging sounds in a dramatic narration, the spectators must be absorbed in the film. And it is in emotionally strong scenes that odd sounds can be used to express strong feelings, in other words where your critical judgement is disconnected.

Music synchresis
The synchresis that comes from intertwining pictorial narration and background music is not hidden from the conscious perception as much as the synchresis with sound-effects and movements. This is partly due to the fact that film music is more visibly connected to the real source, the instruments.
The medieval Commedia dell’arte used a cracking ‘slapstick’ to hit another actor’s behind. A modern equivalent is seen at the circus where the falling of a clown to the ground is accentuated by cymbal and drum.

Exaggerated sound-effects and music every now and then signals jokes in cartoons. If an animated figure climbs steps, stairs or a ladder the orchestra plays an ascending scale. If he falls to the ground you hear a whistling pipe in a descending glissando and then a cymbal crash. As already mentioned, music accompanying movements is an important ingredient in the cartoon. This is sometimes nicknamed ‘mickeymousing’, which personally I find somewhat derogatory. A corresponding example from the field of the feature movie can be seen in Terence Young’s Dr. No, 1962, when the music reinforces every stamp that James Bond makes when killing a poisonous spider.

Humour synchresis
A common feature for humour synchresis is that the sound and music sound improbable to the movements followed. Sounds which are meant to indicate jokes are often heavy and surprising. They strongly diverge from the background sounds, for instance by a piercing or diverging timbre. Either it sounds too much, with a timbre that demands your attention, as in Eric Syke’s The Plank. 1979, when trombones accompany the joint efforts to close the doors of an old blue Morris. Or they sound differently with improbable sounds like the “clonking” from the swinging doors of the dining-room in Tati’s Mr. Hulot’s Holiday. The opening and closing of these doors is marked by a tone similar to a cello pizzicato. Of course unrealistic and exaggerated sounds create a comical effect, as in John Abraham’s Hot Shots, 1991, when a jet plane slams on the brakes in the air to the sound of screaming tires.

There is also a more delicate humour synchresis where you don’t notice the jokes until you have seen the movie a couple of times. How many people have observed in that scene in Mr. Hulot’s Holiday with the silent ping pong match, that at the moment one of the card players takes a swig of beer, you hear a sound reminding you of water streaming in a partially blocked sewage pipe. Anyone who wants to study delicate humour synchresis can benefit a lot from Joel Coen’s Barton Fink, 1991, or by any or all of the films by Jacques Tati.

Horror synchresis
The field in which film sound has perhaps been developed best in recent years is in the horror- and excitement film area. Sound makers are constantly looking for new sounds to create instinctive horror reactions. Sounds with a crushing and screeching character, preferably dark tones that make chairs and tables vibrate, and instinctively make the listeners want to run away. These sounds combined with instinctive dodging reactions, that is to say sudden, rapid movements filling the frame, together with an unexpectedly heavy sound, activate inherent flight instincts. Many of these scenes are so skilfully made that we give a start although we have seen the film a number of times. A well-known example of this shock effect is to be seen in Steven Spielberg’s Jaws, 1975, when a head suddenly emerges out of a sunken ship accompanied by shrill pig-like shrieks.

If a movement starts in synchresis, then the sound has defined the movement in a way that this sound and this movement belong together. When that concordance then disappears and we see foggy pictures difficult to interpret, we keep “seeing by way of our ears”. Referring to the sound, imagination takes over and makes us believe that we have seen what in fact the sound is telling us. In Niels Gaup’s Pathfinder (Ofelas), 1987, a dog is shot with a cross-bow. The audience can hear as well as see the arrow go off. Then they see in blurredness the silhouette of a dog falling backward. What people heard was the hitting sound and a dog whimpering. But inside themselves they saw the arrow hit and how the dog whimpered when falling, dead. >> Listen to the dog

Synchresis is one of the explanations to the flat pictures of the film receiving more depth when the sound is added. Sound is three-dimensional by character and has a spatial extension, while the moving pictures in a film are two-dimensional. By the fact that movement and sound are perceived as an indivisible wholeness, things and people are given a “body” by the sound. This causes us to experience the pictures as even more alive.

Literature: Chion, Michel: Audivision, New York: Columbia University Press, 1990/1994