Project: Automatic Matching and Retrieval of Scars, Marks and Tattoos


PI: Anil K. Jain, Department of CSE, Michigan State University

Co-PI: Rong Jin and Pang-Ning Tan, Department of CSE, Michigan State University



In this project,  we focus on the problem of identifying individuals using tattoo images. People have used tattoos for over 5,000 years to differentiate themselves from others. A recent study published in the Journal of the American Academy of Dermatology in 2006 showed the rising popularity of tattoos amongst the younger section of the population, i.e., about 36% of Americans in the age group 18 to 29 have at least one tattoo. Tattoos engraved on the human body have been successfully used to assist in human identification in forensics applications. For intance, tattoos were used to identify victims of the 9/11 terrorist attacks and the Asian tsunami in 2004. Law enforcement agencies routinely photograph and catalog tattoo patterns for the purpose of identifying victims and convicts (who often use aliases). Current practice is based on manual search to match a query tattoo image with tattoo images in a database, which has limited performance, and is very time-consuming. This motivated us to develop a content-based image retrieval (CBIR) system for automated tattoo image matching. Given a query tattoo image, our system will identify tattoo images from a large database that share similar visual content as the query. It is consisted of two major components: archiving component
that indexes tattoo images, and a retrieval component that matches query images with tattoo images in the database. In addition, we plan to develop (a) multi-label learning algorithms for automatically mapping tattoo images to the ANSI/NIST classes for Scars, Marks, and Tattoos, and (b) clustering algorithms for automatically updating the existing ANSI/NIST classes.


  1. Fengjie Li
  2. Wei Tong


  1. A. Jain,  R. Jin, and J.-E. Lee, Tattoo Image Matching and Retrieval, IEEE Computer 45(5): 93-96, 2012

  2. J. E. Lee, R. Jin, A. K. Jain, and W. Tong, Image Retrieval in Forensics: Tattoo Image Database Application. IEEE Multimedia 19(1): 40-49, 2012

  3. Anil K. Jain, Jung-Eun Lee, Rong Jin, Nicholas Gregg: Content-based image retrieval: An application to tattoo images. International Conference on Image Processing (ICIP), 2009

  4. A. K. Jain, J-E. Lee, and R. Jin, Graffiti-ID: Matching and Retrieval of Graffiti Images, ACM MM, MiFor'09, 2009

  5. Fengjie Li, Wei Tong, Rong Jin, Anil K. Jain and Jung-Eun Lee, An Efficient Key Point Quantization Algorithm for Large Scale Image Retrieval, ACM MM, LS-MMRM, 2009

  6. A. K. Jain, J.-E. Lee, R. Jin, N. Gregg, Content Based Image Retrieval: An Application to Tattoo Images, ICIP, 2009.

  7. J-E. Lee, A. K. Jain, and R. Jin, Scars, Marks and Tattoos (SMT): Soft Biometric for Suspect and Victim Identification, BCC, 2008 (Best Paper Award)