Facial Recognition in Digital Photo Collections (Part 1 of 2)

The photograph above is marked up by Polar Rose, a Firefox toolbar that does facial recognition on photos loaded in your browser. Polar Rose automatically detected three faces in the photograph based upon patterns and compared the images to a database of faces stored on their servers. Had someone already tagged a particular face or person, Polar Rose would have told me who appeared in the photo. Although their database is currently seeded with mostly celebrities and politicians, there’s something going on with Polar Rose that historians and digital archivists should take notice of.

At the heart of my excitement is the possibility of making photograph-to-photograph connections that would otherwise have remained undiscovered. Thinking particularly about collections of family photos, where some have persons labeled and others do not, the ability to automate the identification of individuals in photos goes far in understanding the relationship between these digital objects. The concept of tagging the faces of people in photos, rather than writing a short description accompanying a photo, has become popular in Facebook. There, an individual’s profile aggregates all photos they’ve been tagged in, and presents that corpus for public browsing.. Facebook only expands upon the uses of that information in a limited way, by allowing you to see photos including yourself and another specific person.

Polar Rose is not the only company investing and innovating in visual search. Facial recognition and visual search are slowly being integrated into pre-existing services. MyHeritage’s celebrity face recognition allows you to upload a photo of yourself and see what celebrities you resemble. Like.com offers the ability to see items that are visually similar to the ones you’re shopping for online. And the big gorilla of them all, Google, has quietly implemented facial recognition into their image search. Try this: append &imgtype=face to the url of a image search query, you’ll see with those key words only those images that Google identifies as facelike. The first search I tried was for “digital history“, and I wasn’t impressed. However, when adding &imgtype=face to the query, I happily saw the faces of Roy, Dan, and Bill. Here’s without appending the text, and after. A world of a difference.

I’m breaking this post into two parts. Part 2 will be posted soon and will discuss how facial recognition could improve metadata for digital images and help researchers wade through the seas of digitized images.

[original photo of Northern Pacific Railroad workers from TrainNet]