Toward Bridging the Annotation-Retrieval Gap in
Image Search by a Generative Modeling Approach
Ritendra Datta, Weina Ge, Jia Li, and James Z. Wang
The Pennsylvania State University, University Park, PA 16802
While automatic image annotation remains an actively pursued
research topic, enhancement of image search through its use has not
been extensively explored. We propose an annotation-driven image
retrieval approach and argue that under a number of different
scenarios, this is very effective for semantically meaningful image
search. In particular, our system is demonstrated to effectively
handle cases of partially tagged and completely untagged image
databases, multiple keyword queries, and example based queries with
or without tags, all in near-realtime.
Because our approach utilizes extra knowledge from a training dataset, it
outperforms state-of-the-art visual similarity based retrieval techniques.
For this purpose, a novel structure-composition model
constructed from Beta distributions is developed to capture the
spatial relationship among segmented regions of images. This model
combined with the Gaussian mixture model produces scalable
categorization of generic images. The categorization results are
found to surpass previously reported results in speed and accuracy.
Our novel annotation framework utilizes the categorization results
to select tags based on term frequency, term saliency, and a
WordNet-based measure of congruity, to boost salient tags while
penalizing potentially unrelated ones. A bag of words distance
measure based on WordNet is used to compute semantic similarity. The
effectiveness of our approach is shown through extensive
Full Paper in Color
Ritendra Datta, Weina Ge, Jia Li and James Z. Wang, ``Toward Bridging
the Annotation-Retrieval Gap in Image Search by a Generative Modeling
Approach,'' Proceedings of the ACM Multimedia Conference, pp. 977-986, ACM,
Santa Barbara, CA, October 2006.
Copyright 2006 ACM.
Personal use of this
material is permitted. However, permission to reprint/republish this
material for advertising or promotional purposes or for creating new
collective works for resale or redistribution to servers or lists, or
to reuse any copyrighted component of this work in other works, must
be obtained from the ACM.
July 25, 2006