Today I decided to work from home, since all I'm really doing is research. I've been looking at a few FireFox extensions that involve web page thumbnails in one way or another, trying to determine how they generate the thumbnail.
After jumping back and forth between each method that involves any "thumbnail" reference I think I've found the source, the following SQLite statement
"FROM moz_historyvisits " +
"JOIN moz_places ON moz_historyvisits.place_id = moz_places.id " +
"LEFT JOIN wr_places ON wr_places.url = moz_places.url " +
"WHERE moz_historyvisits.id = ?1 " +
"AND visit_type NOT IN (4,7);";
The origin of all thumbnail references come from this SQLite statement and, in particular, there is,
webreviewDBConn.executeSimpleSQL("CREATE TABLE IF NOT EXISTS wr.wr_places ( url TEXT PRIMARY KEY, frequency REAL, dayvisits INTEGER, domain TEXT, subdomain TEXT, screenshot TEXT, daysession INTEGER );");
Screengrab uses some Java methods to generate it's screenshots, the key ones being
var image = java.ava.awt.Robot().createScreenCapture(new java.awt.Rectangle(box.x, box.y, box.width, box.height));
Packages.javax.imageio.ImageIO.write(image, "png", b64os);
return "data:image/png;base64," + baos.toString();
where box has a references to the screen dimensions. Screengrab uses its own custom class, Base64$OutputStreem, which I may have to decompile and read. The program basically gets the screen capture through the Robot's createScreenCapture method, and saves it to their custom Base64 OutputStreem. What's returned is the Base64 string representation of the image, which can be applied to any html img object through img.src = the returned string. I like this way of getting the screenshot - nothing is actually being saved, only the raw data of the screenshot is stored in the image src attribute. But would it be okay to decompile this custom made Base64$OutputStream file, understand it, then use it for my own extension?
One extra piece of information I found while reading the source code is that there appears to be a method to save files built in to Mozilla. nsIFilePicker creates an open/save dialog box, so this would be very useful for implementing a future save/load function to the graph. Also, nsIFile XPCOM should help in creating temporary files if needed.
Showcase produces a nice thumbnail view of all the pages in tabs you currently have open. The source was pretty daunting to rummage through - one file had nearly 7,000 lines - but when I found how they actually make the thumbnails I was surprised, since it was so easy. It uses a drawWindow method for canvas's which renders the entire web content, given the dimensions, into a canvas object. I don't know why the previous two extensions didn't use this method, as opposed to their elaborate work around. I suppose there is the possibility that, for the Screengrab extension, saving it to a canvas does not help with actually saving the image. Using a canvas seems to be the most reasonable route to create the thumbnails. However, JSViz uses a CSS style property backgroundImage to set the image background, and it takes a URL as the parameter. This means I won't be able to use the JSViz library to create the graphs if I use canvas' to draw the thumbnails on.
On closer inspection it appears that WebReview uses a canvas to draw it's thumbnails as well, except it uses the drawImage method instead of the drawWindow method. This could be useful if I have problems with the drawWIndow method. However, I would still need to know what information is contained in the screenshot column of the database table. Based on what I learned from the Screengrab extension, I think that since it is a string it is most likely the base 64 encoding of the image that gets store in that column. How the information gets there with no table inserts is a mystery to me, though.
The last thing I'd like to make a note about is this Python script using Mozilla to create a thumbnail of any URL. It's deceptively simple and only a hundred lines or less, but if it works correctly (haven't bothered testing) it would be a good option to look into if I have to cut my application away from being a FireFox extension and default to be a stand alone browser application.
I'll spend the rest of the day now cleaning up my test extension, which is now almost doesn't require clicking to log every website.