I think this will be useful. As media starts being embedded into the web browser, it would make sense to start exposing this to the user. I know there have been a few instances where I was listening to something on the radio, but there was little hint of what the song was called (I usually tried to remember a few lyrics and did a Google search. Mixed success).

I brought this up in the whatwg irc channel, and apparently this is being considered for the next version of the spec. Which is understandable, because the server can always display the metadata. But often, media may not be central to the website. For example, background music.

I started look at the Audio/Video backend stuff that moz uses. It got confusing real quick (it doesn’t help that the audio code itself is completely empty). Plus I was in a hurry. So I decided to implement it as an extension.

It was a lovely experience. I had a few problems, including finding out that audio/video wasn’t actually being saved to the cache (bug 469446). It was checked-in like 2 days after I found it out. Also, I hate string very much. The string guide helped, but it is still awful. And I made firefox crash a few times because I’m a nsCOMPtr n00b.

Right now, this extension is working only with ogg vorbis files. Which is stupid because <audio /> is rarely used anywhere, and if it is used, only with certain conditions (wikimedia commons uses the audio tag, but not really. Apparently, the video/audio tags start automatically downloading the media even if it isn’t under autoplay. This is a mess if you have dozens of audio tags in one page. bug 464272). It is so rarely used, that I had to create a audio demo page for testing purposes.

Using it is very simple. Right-clicking on a audio tag brings up the context menu. I decided to use the context menu over Page Info because the media tab of the Page info dialog is very much geared towards images, and that code has to be changed in the firefox source (it’s not easy/pretty to overlay).

Which brings up the audio’s metadata

While a lot of metadata is displayed, some isn’t. For example, iTunes has support for cover art as a COVERART header. While you can put that in vorbis, it should be noted that it isn’t widely supported. So I decided to put in only the standard headers for now.

This is dealing with C++ code. Which is much more dangerous than javascript code because NS_ERROR_OMGWTF doesn’t appear in your error console when I try to free an uninitialized pointer. I made basic checks so hopefully nothing bad will happen. But I didn’t do extensive checking in case we have a bad ogg file or something.

Well, to be fair to me, I always save the function’s return value. I just didn’t check whether it passed nor did anything about it. And this won’t just crash at any time. It’ll crash if you try to load the metadata (I’m very nice like that).

The name of the extension is saraswati, named after the hindu God of music and knowledge (really, a Google search helped out a lot here). Please enjoy! (Linux x86, x86-64 and Windows x86 only right now)

Well, that made it nearly useless, because it was less effort to download each xpi file and do a diff locally.

Well, I’m glad to say that I’ve right a worlds wrong. Some stuff I was working on finally landed recently (can’t remember when. But it wasn’t working two days ago. So somewhere between Monday and today). So you may notice a few changes.

The first being the side panel that shows all the files. Any files that were modified appear italicized. Which is a bit subtle, but is the only indication I could give that wouldn’t make it stick out like a bad rash, or a honda element. Suggestions welcome.

Side Panel

So this pretty much completes what was started. Only some minor improvements were made since the last post. Including a wikipedia colour style diff :

Full screen view

I hope that this will be useful to AMO editors and help speed up the reviews. Which is in much better shape than they were two months ago.

I was also surprised by meeting Armen there, and his brothers. It makes sense now.

While loads of fun (I didn’t know what we were doing when we met up. And I didn’t know how much I would enjoy it when I found out. But it turned out to be a great way to spend a Saturday night), I didn’t get home until later in the morning, where I crashed in my bed and didn’t wake up until 16:00 today. I didn’t manage to get any pictures because I didn’t bring my camera, but I did get a video of blinkenlights on my cell. They took over City Hall, and it was an entertaining light show. They also had it rigged up to play pong and tetris, but it was not well executed.

While I can’t solve the computer rebooting problem, I have figured out a way to make connecting back to all your channels painless.

The first thing you have to do is create a network. A network would then contain a list of channels. Here’s the syntax to create a network :
/network add -nick cesar -realname “Cesar Oliveira” -autosendcmd “/^msg nickserv identify password” mozilla
It’s pretty self-explanatory.
-autosendcmd sends a message to the server once you are connected. In my case, I identified myself to nickserv with my cryptographically strong password (The /^msg means I don’t want to see the input. That way it doesn’t open up a new query window in irssi).
The last parameter is just the name of the network, which doesn’t have to be the same name as the server your connecting to (eg. irc.mozilla.org).

Then you add channels:
/channel add -auto #seneca mozilla
/channel add -auto #firefox mozilla
…
/channel add -auto #kittens mozilla
mozilla should correspond to your network. #seneca will be window 2, #firefox will be window 3…

Finally, when you get disconnected, you can connect to the irc server :
/connect -ssl -network mozilla irc.mozilla.org

Enjoy!

But as anyone can tell you, it’s not an easy task (going towards damn near impossible). Firstly, you can’t really use a lexical parser. Well, you can, but it won’t be dependable. Let’s take an example out of the Reviewer’s guide :

document["crea" + "teElement"]("s" + "c" + "r" + ["i", "p", "t"].join(""));

Which is sneaky way of creating a script element, though I question the competence of someone who will use this as their main line of attack (it’s practically spelled out for you). But taking this as a use case, and ignoring the fact that they can use document[cheese] instead, I wondering if parsing the javascript would make figuring this stuff out any better.

Happily, I have spidermonkey and a js shell to do any parsing I wish. But I found out some cool things that you can do in the shell, such as looking at the bytecode for an entire function using the dis() command.

This was interesting. Certainly, there are some optimizations you can do for :
document["crea" + "teElement"]("s" + "c" + "r" + ["i", "p", "t"].join(""));
I would be shocked if it didn’t end up spelling out :
document["createElement"]("script");

I had a few hurdles to overcome. Firstly, document is not defined in the javascript shell. Thinking it was defined in the xpcshell (owww. I was misled by some apparently unused tests and my general ignorance that xpcshell tests went into unit/ and not js/ directory) I went through the added trouble of coping dis() and related functions from js.cpp to xpcshell.cpp. Once I realized that document wasn’t defined, I made a document mock object just to see what the blasted bytecode would look like.

I was a little disappointed. This source:

var document = {
createElement : function(s) {
print("damn");
}
};

function foo() {
document["crea" + "teElement"]("s" + "c" + "r" + ["i", "p", "t"].join(""));
}

dis(foo);

Ended up being this bytecode :

00000:  name "document"
00003:  string "createElement"
00006:  callelem
00007:  string "s"
00010:  string "c"
00013:  add
00014:  string "r"
00017:  add
00018:  newinit 3
00020:  zero
00021:  string "i"
00024:  initelem
00025:  one
00026:  string "p"
00029:  initelem
00030:  int8 2
00032:  string "t"
00035:  initelem
00036:  endinit
00037:  callprop "join"
00040:  string ""
00043:  call 1
00046:  add
00047:  call 1
00050:  pop
00051:  stop

Source notes:
  0:     0 [   0] newline
  1:     6 [   6] pcbase   offset 6
  3:    37 [  31] xdelta
  4:    37 [   0] pcbase   offset 19
  6:    43 [   6] pcbase   offset 25
  8:    47 [   4] pcbase   offset 47

So, almost. The document["createElement"] part was correct, but the .join() wasn’t optimized. Although I wasn’t overly estatic, I knew that this was just one (somewhat lame) use case in the countless of possible others.

This is making me rethink whether lexical tools are the way to go. While they don’t give you any definitive proof that there is a possible security hole, they can still be useful. For example, if you want to use XMLHttpRequest, then you have to call it at least once in your program (even if you say var Widget = XMLHttpRequest). And at least that can bring up warning flags, or at least give editors a place to look.

I don’t think any tool can completely replace a human being. But hopefully, tools will help make the review process easier because you can start looking at high-risk areas first rather than starting from a arbitrary point and not coming across something until 10 minutes later.

Thank you Dia for letting my express my thoughts in boxes and stick figures. Here is a quick breakdown of some of the components

1. A URL list is simply a list of URL that are known to contain extensions. For example source repositories such as AMO and mozdev.
2. Google API for more separated addons, such as those on blogs and personal sites
3. Manual entries for addons not hosted on webpages. These are usually commercial addons such as McAfee.
4. Site-specific and generic refer to the rules that the crawler must obey. For example, a generic crawler would crawl a personal site such as example.com, while a site-specific policies would handle sites such as AMO where experimental addons require a login.
5. Crawler is a web crawler. I have been having difficulty finding the best tool for the job.
6. Parser parses .xpi files. We should also save the html files to extract contextual information where-ever possible.
7. Site-speicifc persistent storage is just a database for each site we visit. This may have to be rethought, but I want some sort of redundancy plan to keep files saved even if something horrendous happen to a central database. Especially when dealing with beta software and unfamiliar technology such as web crawlers.
8. Compared compares what is stored with a central database. Addons are updated all the time, so we want to the most up-to-date versions available.
9. View is used by the website to provide information for the user.

There are still some quirks which have to be figured out:

• Version bumping on AMO doesn’t change the actual install.rdf in the xpi file. Instead, Firefox does some update magic to fix that. I either need to work with said magic, or leave it alone (I don’t think it is entirely a big deal. But it should be noted).
• JSpider is a java spider that I have been setting my eyes on. Yeah, it’s java, but many other crawlers are too. Many other crawlers do both crawl and index, and I different functionality (I need a flexible crawler. Forget the indexer). Unfortunately, JSpider doesn’t have POST data and web form authentication. Which means I’m going to have to fix that if I want to use it.
• Google’s Search API TOS doesn’t seem to be spider friendly. I may have to try out other web search engines.

On a brighter note, I put up the sources of my project on the web. And even a nice place to play in. It’s a bit slow, but I’m probably into the “this isn’t what you should sqlite for” territory.

This is rather unfortunate for me. How would I get around this problem? The first thing I did was check for an updated python. Which there was a minor version upgrade. I found the changelog (why do they hide these things?) and noticed a few bugs resolved with the zipfile module. So I installed. Unfortunately, this didn’t solve my problem.

I managed to find a bug number in the python bug tracking software about people having similar problems. There was a patch, but hasn’t landed. I downloaded the latest stable version, but the patch wouldn’t go through. So I had to cvs checkout trunk and apply it. Once installed, I tried it and it worked! Success.

However, it broke other library I was using (PyXML). Unfortunate for me, the recent trunk build didn’t seem to fair any better.

At this point, I wasn’t in the mood for debugging. I had a few options at my disposal :

1. Ignore this particular file
2. Suck it up and debug it.
3. Find a whacky work-around

Option 1 isn’t an option. Option 2 I tried for a fair while, but nothing worked. So Option 3 was my only option!

I tried using a lower level library to see if I can fix the problem (zlib library), but that didn’t work well at all.

I finally thought I had no choice but to initiate a thread to try and unzip the xpi, and if it took longer than 10 seconds, to kill the thread somehow. While seriously looking into this, and fighting the temptation to take tequelia shots at work. I came across signals (which I thought I could use to send to the thread. I’m so naive). It turns out, you can throw a signal after a specific number of seconds and it throws the SIGALRM. This was exactly what I needed without the extra complexity. The example provided was almost exactly what I did too! Here is my solution to the problem :

		signal.signal(signal.SIGALRM, signal_handler)
		signal.alarm(10)
		try:
			zippy = zipfile.ZipFile(io, 'r')
			signal.alarm(0)
		except:
			print "\tZipFile Timeout"
			continue

Maybe python isn’t just for programming sissies after all.

There were a few goals I had in this release. The main objective however, is to help avoid making trips to the tinderbox page (because it’s large, and slow). For me at least, I am only concerned about Linux tinderboxes being red so I can checkout . But others might have different needs. So I generally tried to include everything I can. But I could have made a mess of things.

I should mention that you should have a reasonably fast connection (ie. not 56K modem). Even GoogleWiFi was able to reasonably download the json and bonsai xml files that I needed to get things working. Most developers should be fine.

I mainly tried to squeeze as much information as possible into two popup menus, making use of the tooltip to show more information then would be otherwise possible. I also show what menuitems are links by giving them an icon. But it has been a bit overdone.

Anyways, here are some images to show you what you can expect.

When loading, you’ll be amused by the animated png throbber that shows up on the statusbar

Before it can be useful, you have to set it up

The options menu shows you what tinderboxes are available to be watch. For now, you will only see Firefox tinderbox. This was mostly because I was less interested in the other trees. Timeout refers to how long the extension should wait before updating. You want to keep this value to be reasonable.

The statusbar icon will show you the worst state of any of your chosen tinderbox trees.

A left click shows tinderboxes and their status

A right click shows bonsai information. From bottom to top, it shows most recent check-ins. Hovering over menuitems gives you the time/date of check-in as well the check-in message.

Sub menus show a component::file display. The reason for this was because showing the full path took too much room, so I wanted to show what I thought would give you enough information so you can reasonably take an educated guess as to what was being changed. Hovering, of course, shows you the full path and new version.

Bwahaha, the extension lives here in this insecure site until I get it up on AMO. You can also fetch the source from repository.cesaroliveira.net. Any criticisms (hopefully constructive) can be emailed. In the meantime, enjoy this most beta software

So I have been pouring two weeks into WildOn, which is finding out how many addons exist out there in the wild. But before I start unleashing web crawlers on the web causing havoc and chaos, it will be helpful if we could compare what’s out there with what we know. What we know is everything from AMO, so we start there. The point of this extra work is to have some results, so that when we release a web crawler on AMO and tell it to find all the extensions, we’ll have something to compare it’s results to.

Actually, even this was a bit confusing. AMO provides an API to view its addons (well actually, two versions of the API, with the older being slightly more useful). But that information was eventually scrapped for several reasons. The main one being is that there is a lot of information on AMO that isn’t on the extension itself (such as, What operating systems are supported, and is the addon a theme or an extension. While the former has been supported since Firefox 2, I have rarely seen it used, the latter is completely optional). This makes any sort of conclusion inconclusive because you don’t have enough information.

Then there was the problem of having too much information in the database. To the point where ~4000 addons took up ~1.8gigs of information. To an sqlite datbase, this can get slow. When you try some queries, such as the number of extensions that support the ‘jp-JP’ locale, this can get to be even more intensive process as you build a table that comprises of tens of thousands of rows (one row for each guid/locale combination). The reason for this is because older versions where being included in the same table as the newest version of the addon. Some addons had something like 50+ different versions. The solution seemed to be to move old extensions to a different tables. SQL queries seem to go much faster.

Another issue that makes me loathe RDF is install.rdf. I strongly disagree with the use of rdf for anything It becomes difficult to parse with a regular xml parser (there are a few python rdf libraries out there. But rdflib, the most promising, seems to like not working and not having good examples. Only sheppy can save them now, but he’s working on mdc). Especially with rdf:resource, which I am completely ignoring right now. So it seems that AMO editors like to get creative with install.rdf, which has caused problems for me (eg. I can not rely on targetPlatform. Some extensions actually have their targetPlatoform in the Description tag. I know this because one of the extensions had Firefox’s GUID ). Also, some other quirks like having the id as an attribute of Description instead of a new tag. All things that are probably perfectly valid, but make my life significantly more difficult.

YAP was that many early extensions did not use chrome.manifest. And some newer ones don’t. So to look up locale information, they were either in install.rdf or contents.rdf. This makes me (and by extension, kittens and baby Jesus) sad. I don’t have a fix for this yet.

But enough about problems, what about SUCCESS!?

Ok. So I managed to get a local copy of every extension that is on AMO. Since parsing an analyzing and writing to persistent storage takes a long time, I decided to save myself some trouble and just do the first 2500 extensions (out of the ~7K folders that I have).

Of the 2500 ‘extensions numbers’, 1630 where successfully analyzed. This is mainly because extension numbers don’t increment perfectly (eg. there is no addon #1. The first one starts at #4. Only about 100 addons failed to parse, giving me a success rate of 94%. Some extensions had quirks in them (eg. bad RDF) that were either invalid or I couldn’t figure them out.

Out of the 1630 extensions, this is what xulrunner-like applications they supported :

And Here are the approximate numbers :

This looks ok so far. One expects a few non-Firefox extensions. The Thunderbird numbers seem a little low. Reminder that this is only ~33% of the total addons.

Locales seem to be a bigger mess, as there are many early extensions that don’t use chrome.manifest, so I decided to skip it, but now realize I have to fix it. Out of 1630 addons, only 464 addons had chrome.manifest files that I was able to read. But here is the breakdown anyways :

Number of locales : 173 (en, en-US, en-GB are all considered different locales). There are some invalid locales. For example, Xultris has an invalid locale called xultrisLocale. This can be fixed with a regex expression, but anyways.

Some locales will have 0 supported extensions. This is because We are only counting the most up-to-date extension, and not counting previous versions which may have supported that locale. While doing a graph for each locale would be unwise, a much wiser choice would be to break it down into language.

So which languages are best supported?

And here is the obligatory graph for those numerically challenged by high school mathematics teachers.

So what does this lead to? First I need to fix locales. We need to get the vast majority of them. Next, I want to profile all the extensions and not just the first 2500. And then, I want to start looking at web crawlers and learning how to crawl a simple website before unleashing a monster on AMO.

First day at interning at Mozilla. I finally found out what I get to do this summer. I got the OK to blog about it, because you know how secret them Mozilla folks are about their secret in-house project (ie. What is this guy up to? ).

The actual wiki page was apparently out in the open, but no-one heard about it. It’s called WildOnAddons. While a new name is, IMO, mandatory, it’s actually a pretty neat idea. There are many great extensions such as Ted’s Extension Developer’s Extension that aren’t hosted on AMO. Some other extensions are hosted on AMO, but frequently have updates much sooner on their website before it goes public.

Sometimes, extensions come in bundled with packages such as Norton and McAfeee. Google Notebook is one of many Google Labs extension hosted on their own server.

In short, they’re hosted everywhere. But that presents a problem, how many are out there and can find and index them?

This is actually a lot harder then going on google and typing filetype:xpi, because according to those results, AMO only has 78 extensions. In fact, there are several repositories of addons each catering to a different crowd (yes, we are counting all addons). While I don’t think that AMO can satisfy everyone all the time. It might help us figure out how many extensions are out there and how many are hosted on our servers. Actually figuring this out will take a lot of work, and not as straight-forward as it sounds (ie. All of AMO’s sandboxed addons require authentication, so a web crawler would have to know about it if we were crawling through the web), but it will be worth it in the end.

I’ll keep blogging about it under wildon tag RSS feed if your interested on how progress goes.