Tagging over Time: Real-world Image Annotation by Lightweight Meta-learning
Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang
The Pennsylvania State University, University Park, PA 16802
Automatic image annotation has been a hot-pursuit among
multimedia researchers of late. Modest performance guarantees
and limited adaptability often restrict its applicability
to real-world settings. We propose tagging over time
(T/T) to push the technology toward real-world applicability.
Of particular interest are online systems that receive
user-provided images and feedback over time, with
user focus possibly changing and evolving. The T/T framework
consists of a principled probabilistic approach to metalearning,
which acts as a go-between for a black-box annotation
system and the users. Inspired by inductive transfer,
the approach attempts to harness available information, including
the black-box models performance, the image representations,
and the WordNet ontology. Being computationally
lightweight, this meta-learner efficiently re-trains over
time, to improve and/or adapt to changes. The black-box
annotation model is not required to be re-trained, allowing
computationally intensive algorithms to be used. We experiment
with standard image datasets and real-world data
streams, using two existing annotation systems as blackboxes.
Both batch and online annotation settings are experimented
with. It is observed that the addition of this metalearning
layer produces much improved results that outperform
best-known results. For the online setting, the T/T approach
produces progressively better annotation with time,
significantly outperforming the black-box as well as the static
form of the meta-learner, on real-world data.
Full Paper in Color
Ritendra Datta, Dhiraj Joshi, Jia Li and James Z. Wang, ``Tagging over
Time: Real-world Image Annotation by Lightweight Meta-learning,''
Proceedings of the ACM Multimedia Conference, pp. 393-402, ACM, Augsburg,
Germany, September 2007.
Copyright 2007 ACM.
Personal use of this
material is permitted. However, permission to reprint/republish this
material for advertising or promotional purposes or for creating new
collective works for resale or redistribution to servers or lists, or
to reuse any copyrighted component of this work in other works, must
be obtained from the ACM.
October 1, 2007