The Story Picturing Engine: Finding Elite Images to Illustrate a Story
Using Mutual Reinforcement
Dhiraj Joshi, James Z. Wang, Jia Li
The Pennsylvania State University, University Park, PA 16802
Abstract:
In this paper, we present an approach towards automated story
picturing based on mutual reinforcement principle. Story picturing
refers to the process of illustrating a story with suitable
pictures. In our approach, semantic keywords are extracted from the
story text and an annotated image database is searched to form an
initial picture pool. Thereafter, a novel image ranking scheme
automatically determines the importance of each image. Both lexical
annotations and visual content of an image play a role in determining its rank. Annotations are processed using the Wordnet to
derive a lexical signature for each image. An integrated region based
similarity is also calculated between each pair of images. An overall
similarity measure is formed using lexical and visual features. In the
end, a mutual reinforcement based rank is calculated for each image
using the image similarity matrix. We also present a human behavior
model based on a discrete state Markov process which captures the
intuition for our technique. Experimental results have demonstrated
the e ectiveness of our scheme. Categories and Subject Descriptors:
H.3.3 [Information Search and Retrieval]: retrieval models; search
process; selection process.
Full Paper in Color
(PDF, 3MB)
Try the Story Picturing Engine Demo
Other On-line Info
Citation:
Dhiraj Joshi, James Z. Wang and Jia Li, ``The Story Picturing Engine:
Finding Elite Images to Illustrate a Story Using Mutual
Reinforcement,'' Proc. 6th International Workshop on Multimedia
Information Retrieval, in conjunction with ACM Multimedia,
pp. 119-126, New York, NY, ACM, October 2004.
Copyright 2004 ACM.
Personal use of this
material is permitted. However, permission to reprint/republish this
material for advertising or promotional purposes or for creating new
collective works for resale or redistribution to servers or lists, or
to reuse any copyrighted component of this work in other works, must
be obtained from the ACM.
Last Modified:
August 13, 2004
© 2004