When first approaching the tool, my instinct was that Polar Rose – or an abstraction of it, could be useful to digital archivists.  In the age of Google Books and with increasing pressure from patrons to see digital copies of archival content, archival objects that have already been organized and described in a finding aid at the collection-level are thrown into an item-level digital archive on the web.  How do I search that content?  How are those objects described for me, as the researcher, to discover them?  When finding aid keywords become item-level tags, how accurate are my searches going to be about the specific image that I’m viewing?  That can vary.  Polar Rose and its ability to recognize individuals in photos begins to describe the “ofness” of a photo, in contrast to larger categorical key words that describe the “aboutness” of a photo and may be in a finding aid.  Simply put, it provides a different form of description — one that is unique at the item-level that may be useful to researchers.  The fact that Polar Rose performs actions automatically could begin to free up time for archivists and augment the information used to search those digital objects.

Polar Rose isn’t only useful to digital archivists though.  It’s easy to see its practical application in small collections of photos — even on a consumer-level.  Local historians would likely have luck with this type of tool as well, if there were enough reoccurring faces for Polar Rose to recognize.  Family photos?  I’d love to have software than can automatically keep track of the faces of my extended family.  Best of all, Polar Rose is built on top of some open source tools, although it’s unclear how much of it could be useful for creating a more generalized application.

The integration of visual search into archives may still be a ways off, and I’m not suggesting that Polar Rose is by any means perfect right now, but I’d be interested in what my colleagues think about visual search and its potential.  Similar to how text-mining can help us get “inside” written works, visual search may be able to do the same for images, while allowing us to search based upon images themselves rather than text.

The photograph above is marked up by Polar Rose, a Firefox toolbar that does facial recognition on photos loaded in your browser. Polar Rose automatically detected three faces in the photograph based upon patterns and compared the images to a database of faces stored on their servers. Had someone already tagged a particular face or person, Polar Rose would have told me who appeared in the photo. Although their database is currently seeded with mostly celebrities and politicians, there’s something going on with Polar Rose that historians and digital archivists should take notice of.

At the heart of my excitement is the possibility of making photograph-to-photograph connections that would otherwise have remained undiscovered. Thinking particularly about collections of family photos, where some have persons labeled and others do not, the ability to automate the identification of individuals in photos goes far in understanding the relationship between these digital objects. The concept of tagging the faces of people in photos, rather than writing a short description accompanying a photo, has become popular in Facebook. There, an individual’s profile aggregates all photos they’ve been tagged in, and presents that corpus for public browsing.. Facebook only expands upon the uses of that information in a limited way, by allowing you to see photos including yourself and another specific person.

Polar Rose is not the only company investing and innovating in visual search. Facial recognition and visual search are slowly being integrated into pre-existing services. MyHeritage’s celebrity face recognition allows you to upload a photo of yourself and see what celebrities you resemble. Like.com offers the ability to see items that are visually similar to the ones you’re shopping for online. And the big gorilla of them all, Google, has quietly implemented facial recognition into their image search. Try this: append &imgtype=face to the url of a image search query, you’ll see with those key words only those images that Google identifies as facelike. The first search I tried was for “digital history“, and I wasn’t impressed. However, when adding &imgtype=face to the query, I happily saw the faces of Roy, Dan, and Bill. Here’s without appending the text, and after. A world of a difference.

I’m breaking this post into two parts. Part 2 will be posted soon and will discuss how facial recognition could improve metadata for digital images and help researchers wade through the seas of digitized images.

[original photo of Northern Pacific Railroad workers from TrainNet]