Since 1996, the Internet Archive has been making copies of web sites on the Internet to preserve them.  They can be searched using their Wayback Machine, and although there are gaps (thankfully not caching some of the earliest web sites I made in the late 90′s..) it’s an invaluable resource.  About a month ago I decided to create my own solution to link rot, and I first wrote a Greasemonkey script that would detect when I’m on a 404 page, and redirect the browser to the most-recent archived copy of that web page located at the Internet Archive; I call it 404ward.  My idea was, and still is, that users should have seamless experience researching on the web no matter if that link works today or it did 10 years ago.  I want to spend less clicking and searching.  Since writing the original code and asking some friends on Twitter (my handle is davelester), I’ve ported it over to a Firefox extension that’s easier for individuals to install.

It turns out that 404ward isn’t the first, or only solution to link rot out there.  A colleague passed around links to Firefox extensions that do similar things, so I’ll briefly review them.  There are already four extensions that are compatible with Firefox 3 and work with the Internet Archive:

• Passive Cache and CacheIt! both allow you to right-click the link on a web page and choose whether or not you’d like to see a version cached by Google or the Internet Archive.  It’s nice, but doesn’t help you one you arrive at a 404 page, or if you directly typed in a URL that wasn’t working.

• A third extension with the lengthy name “404: File is Not Found?  Now it will be!” is poorly implemented.  I can say this, because they’ve approached redirection the same way I originally did with 404ward, which is to search a web page for the words “404 error” in the HTML, and then automatically redirect a user to the Internet Archive.  For example, this doesn’t work when you’re visiting a page that’s about 404 errors — the extension thinks that you’ve hit a dead link and it automatically redirects you.  Additionally, sites that don’t exist at all won’t return HTML for the extension to read so they won’t work as well.

• The most mature out of all the current extensions that address link rot is called Resurrect Pages.  When you arrive at a 404 page, Resurrect Pages offers you a set of links that visit the web caches of CoralCDN, Google, Yahoo, Internet Archive, Live Search, GigaBlast, and WebCite.  It does a great job making these resources more available, but there’s still a bunch of clicking involved.

Here’s what the 404 or page not found errors look like with Resurrect Pages:

Where does 404ward fit into the mix?  I had hoped to release an alpha version of my blog, but it’s not compatible with Firefox 3.  But that’s OK – I can table my javascript experiment for what’s currently out there.  Resurrect Pages makes terrific strides towards link rot, and I’m throwing my support behind it as a must-have tool for researchers this fall, along with my other fav firefox extension, Zotero.

[link to wikipedia article where I first found the McCown reference]

When first approaching the tool, my instinct was that Polar Rose – or an abstraction of it, could be useful to digital archivists.  In the age of Google Books and with increasing pressure from patrons to see digital copies of archival content, archival objects that have already been organized and described in a finding aid at the collection-level are thrown into an item-level digital archive on the web.  How do I search that content?  How are those objects described for me, as the researcher, to discover them?  When finding aid keywords become item-level tags, how accurate are my searches going to be about the specific image that I’m viewing?  That can vary.  Polar Rose and its ability to recognize individuals in photos begins to describe the “ofness” of a photo, in contrast to larger categorical key words that describe the “aboutness” of a photo and may be in a finding aid.  Simply put, it provides a different form of description — one that is unique at the item-level that may be useful to researchers.  The fact that Polar Rose performs actions automatically could begin to free up time for archivists and augment the information used to search those digital objects.

Polar Rose isn’t only useful to digital archivists though.  It’s easy to see its practical application in small collections of photos — even on a consumer-level.  Local historians would likely have luck with this type of tool as well, if there were enough reoccurring faces for Polar Rose to recognize.  Family photos?  I’d love to have software than can automatically keep track of the faces of my extended family.  Best of all, Polar Rose is built on top of some open source tools, although it’s unclear how much of it could be useful for creating a more generalized application.

The integration of visual search into archives may still be a ways off, and I’m not suggesting that Polar Rose is by any means perfect right now, but I’d be interested in what my colleagues think about visual search and its potential.  Similar to how text-mining can help us get “inside” written works, visual search may be able to do the same for images, while allowing us to search based upon images themselves rather than text.

The photograph above is marked up by Polar Rose, a Firefox toolbar that does facial recognition on photos loaded in your browser. Polar Rose automatically detected three faces in the photograph based upon patterns and compared the images to a database of faces stored on their servers. Had someone already tagged a particular face or person, Polar Rose would have told me who appeared in the photo. Although their database is currently seeded with mostly celebrities and politicians, there’s something going on with Polar Rose that historians and digital archivists should take notice of.

At the heart of my excitement is the possibility of making photograph-to-photograph connections that would otherwise have remained undiscovered. Thinking particularly about collections of family photos, where some have persons labeled and others do not, the ability to automate the identification of individuals in photos goes far in understanding the relationship between these digital objects. The concept of tagging the faces of people in photos, rather than writing a short description accompanying a photo, has become popular in Facebook. There, an individual’s profile aggregates all photos they’ve been tagged in, and presents that corpus for public browsing.. Facebook only expands upon the uses of that information in a limited way, by allowing you to see photos including yourself and another specific person.

Polar Rose is not the only company investing and innovating in visual search. Facial recognition and visual search are slowly being integrated into pre-existing services. MyHeritage’s celebrity face recognition allows you to upload a photo of yourself and see what celebrities you resemble. Like.com offers the ability to see items that are visually similar to the ones you’re shopping for online. And the big gorilla of them all, Google, has quietly implemented facial recognition into their image search. Try this: append &imgtype=face to the url of a image search query, you’ll see with those key words only those images that Google identifies as facelike. The first search I tried was for “digital history“, and I wasn’t impressed. However, when adding &imgtype=face to the query, I happily saw the faces of Roy, Dan, and Bill. Here’s without appending the text, and after. A world of a difference.

I’m breaking this post into two parts. Part 2 will be posted soon and will discuss how facial recognition could improve metadata for digital images and help researchers wade through the seas of digitized images.

[original photo of Northern Pacific Railroad workers from TrainNet]

Anonymous Blogging
For those who followed the story of PhDinHistory, you may be interested in reading Penelope Trunk’s post arguing bloggers should not be anonymous – similar to Dan Cohen’s position. Without injecting myself into a discussion that’s long over, I’ve reaped the benefits of openly blogging, including landing some freelance work and meeting like-minded scholars who have generously helped my research.

Firefox + Second Life = The 3D Web
Linden Lab will incorporate Mozilla Firefox into a future Second Life client, hoping to eventually blur the differences between the 2D web and 3D metaverse. I’m excited about the possibilities of having Firefox in SL – does this mean I’ll be able to use Zotero in-world and grab COinS data from virtual objects? We’ll have to wait and see if the Second Life/Firefox browser will support extensions, but imagine walking through a virtual museum in Second Life and grabbing citations or even copies of objects in that visualized space… it would drastically change the experience.