Thursday, July 23, 2009

Progress, Problems, and Another Release (soon)

    The problem with thumbnails not always extracting still hasn't been completely solved. I wanted to use a timer, but a timer doesn't work the same as, say, wait or sleep in C programming. I tried making it recurse on itself until the thumbnail that at least 1000 characters, but that exceeded the recurssion depth too easily when websites load extremely slow. Also, I can't simply put it in a loop because Firefox will think my script is frozen while it loops until the web page sort of loads, and then Firefox freezes. The Mozilla Developer Center says here that it isn't a good idea to extract page thumbnails during the onload event, and I would have to use the MozAfterPaintEvent. But, that is only in Firefox 3.5. Right now I just have a default image if the thumbnail could not be extracted properly.
    Yesterday Jon Pipitone was giving me lots of suggestions for improvement. One idea that I really like is to add a filtering function, to remove all pages of a certain website. I don't think this would be too hard to implement, using a simple regular expression comparison while building the graph can easily filter out nodes of the chosen type. 
  The big issue to add is some sort of relevant ordering to the nodes. This is hard to do because I have to work around the limitations of JSViz. However, what I think I can do - for now - make root nodes appear in a sensible manner. Currently they spawn randomly across the screen, which isn't helpful. I can alter it so that root nodes appear in chronological order, either lined up vertically or horizontally. This would only help with at least a few root nodes, but it is better than nothing. For now, I'll try this and see if people think it helps.
    Two recent features I've added: Quick Edit Title button and Live Updates. The Quick Edit Title button is a button on the browser toolbar to instantly open up the window for editing the title, it is basically a shortcut. Instead of getting to a site, thinking of a little note to leave, opening the graph, finding that node, right clicked and entering the new note, you can just click this button and do it fast. I've implemented Live Updates so that you no longer have to reload the graph any time you log a new site, it will update itself in real time. 
    I haven't gotten much feedback from posting my extension a few weeks ago, but then again there were only 10 downloads. Soon I'll be putting it up on my CSLab page, with all the latest features. I think before I do that I want to have some more reliable way to organize nodes (or at least root nodes) along with testing everything even more. And I have to write up another set of release notes - that's never fun though.

    And finally, I found out that JSViz has a limitation for the number of edges come from/to a single node: 18. 18 edges are fine, but adding a 19th edge causes everything to catastrophically fail. The only reason I discovered this was because Ainsley tested her Trac plug-in on a large repository and it failed spectacularly, so I tested this by going to Wikipedia and opening every link on one page in new tabs until it crashed. My theory is that the minimum distance set for the magnetic forces can not be maintained once the 19th node is created, since there is not enough room for all of the nodes to evenly spread out around it. It could also be that the magnetic repulsive forces are so compacted and balanced that the 19th node throws the center node off center, causing it to rocket accross the screen. It has to do with the forces because a root node can have more than 19 edges attached to it. Here are so example files:

    Latest Version of BreadCrumbs

    (Note: I'm providing it early with no release notes just to illustrate this edge/node limit per node, some features still are not completely ready [ Stop Animation still has bugs, along with some deleting issues]. If you have an older version of BreadCrumbs please go to your Firefox Profile directory and delete "breadcrumbs.sqlite" as that old database does not have a thumbnail column.)

    Session file that illustrates the breaking [Copy into a text document and save as .session]

    Session file that shows a root with ober 19 nodes/edges [Copy into a text document and save as .session]

1 comment:

  1. I finally downloaded your extension, and tested it on the use case I talked to you about this morning: searching for intersting webdesign.

    It's really interesting for me: I can see immediately what website are interesting, and which aren't to find the webdesign.

    Two things showed up as I was using it:
    - if I arrive on the same page with two different paths, breadcrumbs doesn't associate the two nodes. In some caseit would be interesting (for example, if I copy paste a url in a new tab, I would like this page not to appear in tow different nodes).

    - I think it would be very interesting to have the rating (a star system, or any other) for a node or a path. From a UI point of view, it would be nice just to right click, and be able to add a 'grade' and/or a comment directly in a sort of submenu (for example below 'add a bookmark', as it is the same type of tool).

    - Another features which would be useful, and it's join a bit the 'delete all pageof this website' idea, would be to create clusters of pages belonging to a same website. It would be an easy way to organize a bit the view.

    - The last feature I thought about was a hide option, a bit similar like the filter option. Instead of deleting some path or node, it would be nice to hide them, and unhide them if needed.

    Is it possible to annotate node, and not paths?