About a month before he died, I asked about his post-Reader plan. His response?
sorry, my plan is to no longer need such things.Fat lot of good that does me. Bereft and without a good RSS reader.
My friend was my audience. Now you are. I may keep at this or not. It will be very different for me in any case. Back to the listening stations...
The hardware execution left a motley collection of iOS and Android tablets -- borrowed, bought, and donated -- arrayed on my kitchen table seeking only a uniform interface that could capture so many ears and eyeballs that were unknown to me. I had a terrific asset beyond the tablets. My friend's wife teased ten great playlists from their collection and her sense of his taste.
Thomas Asbridge wrote that an 11th century visitor to Constantinople might have seen at least two heads of John the Baptist. As a child, I heard a joke about a monastery that held the heads of John as a child, as a teen, and as an adult. This project aimed for a much more minor miracle -- the ten headphones chronicling the stages of a careful listener's evolution.
I rejected both the native iOS and Android music players out of hand. I was impressed when Steve Jobs offered me a thousand songs in my pocket with the original iPod. I wanted ten new songs in your head. It's a different interface problem.
There was no way that I was going to fire up Eclipse and Xcode and build two native apps from scratch. I have some limited experience with PhoneGap. That could work. It would only have worked for me if the audio handling in each was robust. I figured that PhoneGap could work if the native browsers were up to the task themselves. If they were, why not just use them directly?
I first heard audio in a web browser in about 1995. It was a terrible MIDI file that began playing in the background of a cheesy web page for a computer parts vendor in Florida. I recall that site also had an animated GIF of a swaying palm. Both were irritating. Background MIDI files and animated GIFs only lasted a few years in the mainstream of the early web. Little animated 'under constuction' signs lasted longer. These had a special, almost sacramental, role in the web as digital headstones for stillborn web sites.
 |
| Future generations will interpret this as a sign of quarantine. What lies beneath? Perhaps the digital plague that killed the fabled GeoCities. image courtesy mikesfreegifs.com |
I think I understand how and why they came back. I think the iPhone is at the root of both.
Apple famously ditched Flash when the iPhone debuted. This was a move that now seems obvious. It was controversial at the time. Recall that Flash was unavailable even at a time when there were no native apps and the in-frame audio and video handling in Mobile Safari was nonexistent. Apple sold millions of iPhones before the first public working draft of HTML5 turned up.
HTML5 is the next generation of HTML. It includes direct support for audio and video in the browser. The working group behind the spec left almost no detail unaddressed -- except for the what video and audio formats would actually be supported. As a result, essentially no video or compressed audio file format was supported by all browsers. Lots of ink was spilled in the wake of this apparent disaster. I recall that patents were the villain.
Nearly everyone forgot that there was a 'video' format supported in all major browsers -- the animated GIF. They crawled back to the web while we were distracted by the battle over real video.
I bet that Mobile Safari included support for animated GIFs only by accident. If they had let Flash in or left GIF out, I think there would have been no niche for these files.
Perhaps animated GIF was an accidental video format. There never was an accidental audio format. There was no 'beep' tag in HTML that could be bit-banged into bad audio. The format situation remains technically unresolved after more than four years of half-hearted trying. In practice, there is no problem. The browsers on the mobile devices all play the most common formats. Flash still exists on the desktop for browsers that don't.
I remember spending some large portion of my precious BBS upload/download ratio margin on a DOS executable -- not an audio file -- that played a 'Magic Mushroom' commercial out of the PC speaker by bit banging. I no longer have any idea what a magic mushroom was but the fact of this commercial pouring out of my PC speaker certainly made an impression on me.
The major obstacles that remain for web audio have nothing to do with the original codec battle. Now we have to deal with implementations for iOS and Android that are broken almost to the point of uselessness.
One of the first problems with web audio started out looking like a solution. Implementors at Apple decided that a web site should not be able to start an audio clip playing without a user choice. That's a completely laudable goal. Javascript programmers can call a 'play' method on audio objects, but the audio won't play unless your Javascript is executing in a context that the browser understands is user directed.
button.addEventListener('click',function(e) {
audio.play();
},false);
this works for limited values of works. We have now unlocked the ability to play sounds with this audio element. We can even call play again later without needing to be in a user directed call. We can even assign the audio element an entire new clip and play it later.
The only catch is that there may be a delay of many seconds between the time the user presses play and playback begins. Mobile Safari, at least, will not buffer the song until we have unlocked playback. Browser audio nodes to receive an event when there is enough file loaded to begin playback. You can at least put up a 'buffering' notice until playback is actually ready and then issue the play command once the media is ready to go. That looks something like this:
button.addEventListener('click',function(e) {
update_status('buffering');
audio.addEventListener('canplay',function(e) {
audio.play();
},false);
},false);
It doesn't work at all. The interior call to audio.play() inherits the lexical scope of the surrounding block, but whatever portion of the environment is required to unlock playback is lost by the time the 'canplay' callback fires. This no longer counts as a user-initiated event.
One solution is to create an audio node as soon as the page loads and assign it some kind of 'silence.mp3' file. You can steal the first real user event anywhere on the page and use it to unlock play on this silent file. Once that audio node is unlocked, you recycle it and assign all of your subsequent tracks to it.
The obvious problem with this recycling approach is that it doesn't leave you with a spare audio node for pre-buffering a next track. It seems obvious that you could create two phantom nodes as easily as one and then double-buffer. Alas, some mobile browsers limit you to a single audio node.
The more subtle problem with this approach is that both major mobile browsers have bugs when re-using audio nodes. Google's Chrome for Android seems to work except that it decides that all subsequent tracks assigned to an audio node have a duration of exactly 100 seconds. With luck, a metadata event will later fire with a correct duration. If you are building a media player widget with Chrome, better to wait for the real duration to arrive.
Track duration seems handy but unnecessary. I needed it for two reasons. First, I wanted to actually use it. My DiscMan could do it in the '90s. I wanted to rock that style. Second, the events that would normally fire when a track ends are unreliable at best. I resorted to Javascript 'setTimeout' calls to rotate tracks.
I have almost no problems with the HTML5 audio API. It's actually pretty nice. I have about a hundred problems with the actual implementation in every browser. Here's another problem. Mobile Safari and Chrome for Android both prevent the user from controlling audio volume with the audio tag.
The audio API exposes a volume control. Apple and Google probably ignore it to keep you from nuking your eardrums with a rude webpage. The effective result was that my listening station users had to use the hardware buttons. I printed stickers for each device that pointed to the volume up and down buttons. This was probably the most embarrassing part of the whole endeavor.
The audio API is yesterday's news in any case. 'AudioContext' is the new hotness. The audio tag was really sort of like a noisy version of the img tag. You could whack up some neat tricks with it but it was never meant for real-time effects or serious programmatic use. AudioContext takes over there. If audio tag is like img, AudioContext is like canvas. (Note to non-Javascript readers: that means it's meant to be totally awesome).
Google built a snazzy demo called jamwithchrome. You can jam. With chrome. It's like a Casio keyboard from 1990 built right into your browser. It's even more than that. It's a Casio keyboard connected to the keyboards of all your friends. It even connects to that weird kid's Casio drum machine.
AudioContext is pretty cool. It's just the thing to build listening stations with. Alas, it is slightly less stable than a Casio keyboard. It also wasn't supported on some of the iOS devices I used.
I'm definitely using AudioContext the next time somebody asks me to build funeral listening stations. For this application, I muddled through with the audio tag.
Getting the audio marshaling right was only the first problem. The next problem was actually making the web app into an immersive experience.
iOS pulled ahead here and stayed ahead throughout. iOS has long supported turning a web page into a 'app' on the home screen. These apps can run in a full-screen mode without any of the usual web browser trappings. They can be nearly indistinguishable from native apps.
The only downside is that the web page still has to come from somewhere. Mobile Safari supports
important parts of the web standards for caching offline web apps but the support for caching audio is very weak. There was simply no way to make this work at the event without running a WiFi hotspot with access to a server. That was not going to work.
Android fared slightly better on this point and only on this point. Chrome on Android gained support for full-screen apps on tablets only a few days before the event and this support seemed a bit dodgy -- especially when the screen rotated. Chrome wouldn't cache audio any better than Safari, but Chrome would let me side-load the entire site into the sdcard directory on the Nooks and run it from the filesystem. Even here, Chrome or the Nooks let me down. There was no way to save a bookmark from the filesystem to the Android desktop and have it launch properly. This meant that if the browser exited for any reason then the user was dumped back at the Android home screen without a single-click way to get back.
This is about where I started looking into 'kiosk' type applications for both platforms. The search ended quickly in both cases. For Android, the kiosks all wrap the marginal default Android browser, not Chrome. Poor Audio support in that browser made every Android kiosk app irrelevant. I used Chrome for the memorial on the Nooks and hoped for the best.
iOS was a different story. The kiosk apps all wrap Apple's very good browser. I settled on Kiosk Pro and I was delighted with it. Kiosk Pro is the paid version of 'Kiosk Pro Lite'. The pro version adds the ability to run from side-loaded content. It worked. The only hitch was that the side-loaded content could not have sub-directories. I was able to flatten the web app for the memorial so this was not a problem.
Kiosk Pro inherited some of Safari's audio limitations. It couldn't auto-play or allow users to control volume on-screen. I had a crazy collection of borrowed iPads for the event. I loaded Kiosk Pro on each of them and arranged their home screens to have empty initial screens save for Kiosk Pro.
I recall that I promised pictures of the final installation. I'll share those on side c. Here's a photo of the intermediate installation -- after I built all ten prototypes and before I realized they were too deep for the shallow bar rail at the venue. The iPad at center is supported above the concrete floor by a few LEGO bricks and force of will. The tablets by themselves actually sat pretty nicely on the bar rail. Alas, the rail pushed the volume buttons on several when laid down carelessly. The stage curtain behind the rail was poised to sweep them all off in a moment.
 |
| iPad suspended above concrete by force of will photo courtesy your correspondent |
This installment is about a month overdue. It's been a hard, busy month. I'm banging this post out now because I have a backlog of about a dozen things to tell you all. They had to wait for this. Next comes everything else. Next comes life. With life comes lots and lots of gadgets.
You guys are fantastic. I'm grateful for your support.
No comments:
Post a Comment