The Story Picturing Engine - A System for Automatic Text Illustration
Dhiraj Joshi, James Z. Wang, Jia Li
The Pennsylvania State University, University Park, PA 16802
In this paper, we present an unsupervised approach towards automated story picturing. Story picturing refers to the process of illustrating a story with suitable pictures. In our approach, semantic keywords are extracted from the story text and an annotated image database is searched to form an initial picture pool. Thereafter, a novel image ranking scheme automatically determines the importance of each image. Both lexical annotations and visual content of an image play a role in determining its rank. Annotations are processed using the Wordnet to derive a lexical signature for each image. An integrated region based similarity is also calculated between each pair of images. An overall similarity measure is formed using lexical and visual features. In the end, a mutual reinforcement based rank is calculated for each image using the image similarity matrix. We have implemented the methods in our Story Picturing Engine (SPE) system. Experiments on two large-scale image databases are reported. A user study has been performed and statistical analysis of the results has been presented.
Full Paper in Color
Try the Story Picturing Engine Demo
Dhiraj Joshi, James Z. Wang and Jia Li, ``The Story Picturing Engine -
A System for Automatic Text Illustration,'' ACM Transactions on
Multimedia Computing, Communications and Applications,
vol. 2, no. 1, pp. 68-89, 2006.
Copyright 2005 ACM.
Personal use of this
material is permitted. However, permission to reprint/republish this
material for advertising or promotional purposes or for creating new
collective works for resale or redistribution to servers or lists, or
to reuse any copyrighted component of this work in other works, must
be obtained from the ACM.
Nov 1, 2005