A few weeks ago, I wanted to do some C++ Mozilla coding to make sure I wasn’t going soft. But I didn’t really know what to do. I left it for a bit until I found something weird about the HTML5 spec – there was a method of testing whether metadata has been loaded, but no way to expose the metadata (eg. song title, artist, album, etc) to the user such as through page info.

I think this will be useful. As media starts being embedded into the web browser, it would make sense to start exposing this to the user. I know there have been a few instances where I was listening to something on the radio, but there was little hint of what the song was called (I usually tried to remember a few lyrics and did a Google search. Mixed success).

I brought this up in the whatwg irc channel, and apparently this is being considered for the next version of the spec. Which is understandable, because the server can always display the metadata. But often, media may not be central to the website. For example, background music.

I started look at the Audio/Video backend stuff that moz uses. It got confusing real quick (it doesn’t help that the audio code itself is completely empty). Plus I was in a hurry. So I decided to implement it as an extension.

It was a lovely experience. I had a few problems, including finding out that audio/video wasn’t actually being saved to the cache (bug 469446). It was checked-in like 2 days after I found it out. Also, I hate string very much. The string guide helped, but it is still awful. And I made firefox crash a few times because I’m a nsCOMPtr n00b.

Right now, this extension is working only with ogg vorbis files. Which is stupid because <audio /> is rarely used anywhere, and if it is used, only with certain conditions (wikimedia commons uses the audio tag, but not really. Apparently, the video/audio tags start automatically downloading the media even if it isn’t under autoplay. This is a mess if you have dozens of audio tags in one page. bug 464272). It is so rarely used, that I had to create a audio demo page for testing purposes.

Using it is very simple. Right-clicking on a audio tag brings up the context menu. I decided to use the context menu over Page Info because the media tab of the Page info dialog is very much geared towards images, and that code has to be changed in the firefox source (it’s not easy/pretty to overlay).

Which brings up the audio’s metadata

While a lot of metadata is displayed, some isn’t. For example, iTunes has support for cover art as a COVERART header. While you can put that in vorbis, it should be noted that it isn’t widely supported. So I decided to put in only the standard headers for now.

This is dealing with C++ code. Which is much more dangerous than javascript code because NS_ERROR_OMGWTF doesn’t appear in your error console when I try to free an uninitialized pointer. I made basic checks so hopefully nothing bad will happen. But I didn’t do extensive checking in case we have a bad ogg file or something.

Well, to be fair to me, I always save the function’s return value. I just didn’t check whether it passed nor did anything about it. And this won’t just crash at any time. It’ll crash if you try to load the metadata (I’m very nice like that).

The name of the extension is saraswati, named after the hindu God of music and knowledge (really, a Google search helped out a lot here). Please enjoy! (Linux x86, x86-64 and Windows x86 only right now)

This post is a mashup of a few things I have been tinkering with over the last week that I think is fun to share. So if it seems I have been unfocused or whatever, this is pretty much why.

The first project I started doing for fun was working on canvas. This was different then some canvas stuff I have done in the past, The interesting people at nihilogic did a sepia filter using canvas. I wondered if it was possible to do filter so you can see an image with a red-green colour blindness. After some substandard research, I finally managed to do it. Though the quality is poor because it tends to be inaccurate. YMMV.

I wondered if you can do something like this for an entire webpage. So I moved the Javascript to an extension so I can use canvas’ drawWindow() method and take a picture of the entire website. Though I noticed that doing this on large image was computationally expensive and locking up the UI for an unreasonable amount of time.

I then tried to move all the calculations out of the main thread into a DOM worker thread. It was an interesting experience. I noticed though that while the main thread (and therefore, the UI) did not lock up, it was still sluggish and impractical to use. So I decided not to develop the extension further.

You can see the full demo here.

I then thought about what this would look like on other browsers. I didn’t expect anything requiring DOM worker threads to work on Safari/Opera. And sure enough, it didn’t. But I found out that DOM worker threads was based off of Google gears! So I looked into that and made a separate webpage that uses gears. Unfortunately, I found out that my efforts were largely wasted, as it only increased support to Firefox 2 and Mac Safari (Gears isn’t compatible with Windows Safari or Opera, and IE doesn’t have canvas support).

Either way, I made the Gears version available here.

Going away from canvas, I spent most of another day working on Google Maps API. The problem I was trying to solve was to see if I can highlight a 1 square kilometre radius from a pinpoint. This was difficult, as points on a map had a latitude, longitude co-ordinate, and I had to blindly figure out how much to reposition for a half-kilometre. Finding the distance between two points was also helpful, but hard getting a good formula for.

Of course, I am highlighting all the accomplishments and not mentioning the frustrating obstacles. There were several lessons learn on the way. Including a lot about incompatibility and how much I still don’t know how to do the kind of algorithmic research that you sometimes need. I’m starting to wonder if the BSD course taught me more than just to be a code monkey with a business touch, and made me wonder whether the theoretical/mathematical part will ever stop me doing something because “I just won’t get it”. Though, at the same time, I wasn’t willing to put the time and effort of research into pet projects. So this will probably be a problem for almost everyone, and not just me (honestly, mapping out longitude and latitude to distance is not something you learn anywhere).