|
Image annotation
|
-
Jia Li, James Z. Wang, ``Automatic linguistic indexing of pictures by a
statistical modeling approach,'' IEEE Transactions on Pattern Analysis and
Machine Intelligence, 25(9):1075-1088, 2003.
Abstract:
Automatic linguistic indexing of pictures is an important but highly
challenging problem for researchers in computer vision and content-based
image retrieval. In this paper, we introduce a statistical modeling approach
to this problem. Categorized images are used to train a dictionary of
hundreds of statistical models each representing a concept. Images of any
given concept are regarded as instances of a stochastic process that
characterizes the concept. To measure the extent of association between an
image and the textual description of a concept, the likelihood of the
occurrence of the image based on the characterizing stochastic process is
computed. A high likelihood indicates a strong association. In our
experimental implementation, we focus on a particular group of stochastic
processes, that is, the two-dimensional multiresolution hidden Markov models
(2D MHMMs). We implemented and tested our ALIP (Automatic Linguistic Indexing
of Pictures) system on a photographic image database of 600 different
concepts, each with about 40 training images. The system is evaluated
quantitatively using more than 4,600 images outside the training database and
compared with a random annotation scheme. Experiments have demonstrated the
good accuracy of the system and its high potential in linguistic indexing of
photographic images.
|
|
Similarity search
|
-
James Z. Wang, Jia Li, Gio Wiederhold, ``SIMPLIcity: Semantics-sensitive
integrated matching for picture libraries,'' IEEE Transactions on Pattern
Analysis and Machine Intelligence , 23(9):947-963, 2001.
Abstract:
The need for efficient content-based image retrieval has increased
tremendously in many application areas such as biomedicine, military,
commerce, education, and Web image classification and searching. We present
here SIMPLIcity (Semantics-sensitive Integrated Matching for Picture
LIbraries), an image retrieval system, which uses semantics classification
methods, a wavelet-based approach for feature extraction, and integrated
region matching based upon image segmentation. As in other region-based
retrieval systems, an image is represented by a set of regions, roughly
corresponding to objects, which are characterized by color, texture, shape,
and location. The system classifies images into semantic categories, such as
textured-nontextured, graph-photograph. Potentially, the categorization
enhances retrieval by permitting semantically-adaptive searching methods and
narrowing down the searching range in a database. A measure for the overall
similarity between images is developed using a region-matching scheme that
integrates properties of all the regions in the images. Compared with
retrieval based on individual regions, the overall similarity approach (1)
reduces the adverse effect of inaccurate segmentation, (2) helps to clarify
the semantics of a particular region, and (3) enables a {\it simple} querying
interface for region-based image retrieval systems. The application of
SIMPLIcity to several databases, including a database of about 200,000
general-purpose images, has demonstrated that our system performs
significantly better and faster than existing ones. The system is fairly
robust to image alterations.
|
|
Image clustering
|
-
Jia Li, ``Two-scale image retrieval with significant meta-information
feedback,'' Proc. ACM Multimedia, Singapore, 4pp. -, November 2005.
Abstrat :
A two-scale image retrieval system is developed to provide efficient search
in large-scale databases as well as flexibility for users to incorporate
subjective preferences during retrieval. A new clustering method is
developed for images each characterized by a varying number of weighted
feature vectors. Furthermore, significant meta-information is mined within
every cluster. A scanning mode of retrieval is created using cluster
centers, which serve as a low scale version of a database in contrast to
original images. In particular, users are presented with representative
images of highly ranked clusters along with prominent meta-information. This
retrieval approach enables users to quickly examine a large and diverse
portion of a database surrounding a query and to learn about hidden
connections between visual patterns and non-imagery types of data. The
clusters formed also facilitate fast search in the case of individual
image-based retrieval by filtering out images whose cluster centers are far
from the query. The two-scale retrieval system has been implemented on a
fine art painting database. Advantages of the system have been
demonstrated by quantitative evaluation of the retrieval performance.
|
|
Semantic learning
|
-
Jia Li, ``A mutual semantic endorsement approach to image retrieval and
context provision,''
Proc. International Workshop on Multimedia Information
Retrieval
, ACM, Singapore, 10pp. -, November 2005.
Abstrat :
Learning semantics from annotated images to enhance content-based retrieval
is an important research direction. In this paper, annotation data are
assumed available for only a subset of images inside the database. An on the
fly learning method is developed to capture the semantics of query images.
Specifically, the semantics of annotated images in a visual proximity of a
query are compared with each other to determine the amount of mutual
endorsement. An image is considered endorsed by another if they possess
similar semantics. Annotations with high mutual endorsement are used to
narrow down a candidate pool of images.
The new retrieval method is inherently dynamic and treats seamlessly
different forms of annotation data. Experiments show that semantic
endorsement can increase precision by as much as 70% in average for a wide
range of parameter settings. We also develop a context provision mechanism
to reveal the relationship between a query and semantic clusters extracted
from the database. Context helps users explore the content of a
database and provides a platform for them to tailor searches by stressing
different perspectives in the interpretation of a query.
|
|
Computational art
|
-
Jia Li, James Z. Wang, ``Studying digital imagery of ancient paintings by
mixtures of stochastic models,'' IEEE Transactions on Image Processing,
12(3):340-353, 2004.
Abstract:
This paper addresses learning based characterization of fine art painting
styles. The research has the potential to provide a powerful tool to art
historians for studying connections among artists or periods in the history
of art. Depending on specific applications, paintings can be categorized in
different ways. In this paper, we focus on comparing the painting styles of
artists. To profile the style of an artist, a mixture of stochastic models is
estimated using training images. The 2-D multiresolution hidden Markov model
(MHMM) is used in the experiment. These models form an artist's distinct
digital signature. For certain types of paintings, only strokes provide
reliable information to distinguish artists. Chinese ink paintings are a
prime example of the above phenomenon; they do not have colors or even
tones. The 2-D MHMM analyzes relatively large regions in an image, which in
turn makes it more likely to capture properties of the painting strokes. The
mixtures of 2-D MHMMs established for artists can be further used to classify
paintings and compare paintings or artists. We implemented and tested the
system using highresolution digital photographs of some of China s most
renowned artists. Experiments have demonstrated good potential of our
approach in automatic analysis of paintings. Our work can be applied to other
domains.
|
