archmagus

Even silents had sound

I’m listening to music as I type this entry. I love music. It adds a level of depth and immersion to an experience that is difficult if not impossible to achieve otherwise.  Movies figured this out very early on.  Games did too, though it took a while for technology to catch up. Along with emotive music, sound effects help capture the experience of deeper interaction with your application.  Having virtual buttons click when you press them, or even cartoonish sound effects when your flash sprite jumps up and down on the screen helps give the user of the app or player of the game closer ties to the virtual system they are interacting with.

Fortunately, HTML5 has added a whole new interface for playing audio, and while it has some surprises and limitations even on the desktop, folks are figuring out ways around it.  Unfortunately for us in iPhone land, iOS does not support these workarounds on mobile Safari, which presents us with two big problems.

That could have gone better

First, without preloading, the first time your app tries to play a sound the app animations are going to stutter as the JavaScript works on loading up the sound rather than refreshing the screen (remember, JavaScript isn’t truly multithreaded…alas).  The whole point of preloading is to remove that stutter by having any audio already loaded and ready to play the instant it is needed when the app was first loading up.  The iPhone ignores any preload requests, however (either in the HTML “audio” element or via the JavaScript Audio.load() call), so the first time you play a sound you are stuck with that hit.

Second, unlike the modern desktop browser, iOS Safari will only play a single sound at a time.  On the desktop, you can load two different audio elements (say, background music and a sound effect) with multiple Audio objects in your JavaScript, and if you call play() on each object, the browser will play both audio tracks simultaneously.  The iPhone does not support this.  Instead, when your app calls play() on the second audio object, the first audio stream is interrupted and the second stream plays.  If you call play() on the first audio stream again, the track continues from where it was interrupted by the second audio stream.

If you know the enemy and know yourself you need not fear the results of a hundred battles. – Sun Tsu

Armed with the knowledge of the issues we are facing, all is not entirely lost.  If you’re writing a game, you can still have sound effects, but they must be short.  Short audio tracks will load faster the first time they are played and thus won’t impact your game quite as much as they will if the audio is long.  Different sound effects are also unlikely to interfere with each other if any particular effect only takes a fraction of a second to play.  Music might even still be possible as well, but you won’t be able to write atmospheric music that is intended to be running in the background for the entire game session, as any sound effects will interrupt your music track.  You might, however, still be able to play some music to set the tone during an introductory or victory screen for your game.  Look for opportunities to play music when there will not be any other sounds from the game that will interfere with that music.

Good luck!