CS 545I - Advanced Image Databases, W 95/96
1a - Web sites re Query by Image Content
- QBIC System http://wwwqbic.almaden.ibm.com
Try out the IBM QBIC system. Comments for improvement are welcome.
- Berkeley Digital Library Project,
Try out the Cypress image retrieval system. Not a very friendly front end.
A medical image retrieval system being developed by the Section of Medical
Informatics, Dept of Medicine, U of Pittburgh.
- MIT Photobook
Also visit sites of Profs. Pentland and Lippman.
- Jacob (Just A COntent-Based) Query system for Video DB , U of Palermo
Describes system and lets you try some queries. Not a very friendly front
- Virage Corp.
Lets you try queries. Virage is a small image retrieval company in the
San Diego area. At the present time, the only other commercial offering
besides IBM QBIC.
- Excalibur http://www.excalib.com
- WebSEEk http://disney.ctr.columbia.edu/WebSEEk
- VisualSEEk http://disney.ctr.columbia.edu/VisualSEEk
- WebSeer (University of Chicago) http://infolab.cs.uchicago.edu.webseer
- Image Surfer (Interpix Software Corporation) http://www.interpix.com
- Publishers Depot http://www.publishersdepot.com
- Boston University (description of research, papers, images) http://www.cs.bu.edu/groups/ivc/Home.html
1b - Related to images (not content-based, but interesting)
- Yahoo Search engine
Search for multimedia, pictures,archives; online image archives OR picture
- Getty Art History Information Program,
How to build image databases; image capture; resources for indexing art.
1c - University work or courses
- MIT Media Lab has many good projects, especially from Profs. Picard,
Pentland, and Lippman
- Prof. Sclaroff, Boston university
Description of student projects
- Sean Landis Term Paper
A very informative student term paper describing a system for "Content-based
image retrieval systems for interior design."
1d - Database stuff
- IBM DB2 database
Mouse "Data Managemen" icon and then DB2 under that. Example
of database offering on the Web that integrates SQL and WWW, both at the
user interface and internally.
- Getty Museum
Natural language interface to image database (not query by image content)
1d - Digital Library Inititative
The Digital Library Iniative consists of six university projects funded
jointly by NSF, DARPA, NASA. The funding is about one million dollars a
year for each institution. Some of these projects, particularly UC Santa
Barbara, CMU, UC Berkeley, and U of Illinois, have an image retrieval aspect.
As of March 1996, none of the Web sites is particular strong on details
of image analysis.
- University of California, Berkeley : "The Environmental Electronic
Library: A Prototype of a Scalable, Intelligent, Distributed Electronic
- University of California, Santa Barbara: "The Alexandria Project:
Towards a Distributed Digital Library with Comprehensive Services for Images
and Spatially Referenced Information" http://alexandria.sdc.ucsb.edu
- Carnegie Mellon University: "Informedia: Integrated Speech, Image
and Language Understanding for Creation and Exploration of Digital Video
- University of Illinois at Urbana-Champaign : "Building the Interspace:
Digital Library Infrastructure for a University Engineering Community"
- University of Michigan : "The University of Michigan Digital Libraries
Research Proposal" http://www.sils.umich.edu/UMDL/HomePage.html
- Stanford University : "The Stanford Integrated Digital Library
1f- Just for fun
- Browsing for art:
http://www.cmp.ucr.edu photography exibits
- http://www.metmuseum.org Metropolitan Museum of Ar
- http://www.rosprint.ru/art/museum/pushkin Pushkin Museum
- http://sunsite.unc.edu/wm WebMuseum, Paris
- http://www.echonyc.com/~whitney Whitney Museum of Art
2- Books The book we recommend has several sections on content based
retrieval for image and video DB - the only book to address this topic
Furht, Smoliar, and Zhang: Video and Image Processing in Management
Systems; Florida Atlantic University, Boca Raton, FL. Kluwer Academic
Publishers, 1995 (ABOUT $150!)
3-Collections and Journals
- Storage and Retrieval for Images and Video Databases II; IS&T/SPIE
Symposium on Electronic Imaging Science and Technology,Feb 95, SPIE
- Storage and Retrieval for Images and Video Databases III; IS&T/SPIE
Symposium on Electronic Imaging Science and Technology,Feb 95, SPIE
- Multimedia Systems Journal
- Proc. ACM Multimedia '93, Anaheim, CA, Aug.1993.
- Proc. ACM Multimedia '94, San Francisco CA, Oct.1994
4 - Papers primarily on still imagery:
- P. Aigrain et al.: Content-based Representation and Retrieval of Visual
Media: A State-of-the art Review,Multimedia Tools and Applications,
Kluwer Academic Publishers, Vol. 3, No. 3, November 1996.
- Pentland et al: Photobook: Tools for Content-Based Manipulation of
Image Databases; Perceptual Computing Section, The Media laboratory, MIT,
Cambridge, MA 02139, SPIE, Storage and Retrieval Image and Video
Databases II, No. 2185, Feb 1994
Describes Photobook, a set of interactive tools for browsing and searching
images and image sequences. Direct search on image content is made possible
by the use of semantics-preserving image compression. The paper
describes three Photobook tools, one that allows search based on grey-level
appearance, one that uses 2-D shape, and a third that allows search based
on textural properties. W. Niblack et al.: Efficient and Effective
Querying by Image Content; Special Issue on Integrating Artificial Intelligence
and Database Technology, Journal of Intelligent Information Systems,
No. 3, 1994.
- W. Niblack et al. Efficient and Effective Querying by Image Content,
special issue on Integrating Artificial Intelligence and Database Technology,
Journal of Intelligent Information Systems, No. 3, 1994.
- Flickner et al: Query by Image and Video Content: The QBIC System;
IBM Almaden Research Center, San Jose, CA 95120-6099, Computer,
Vol. 28, No. 9, pp.23-31, Sept 1995
Presents an overview of QBIC. Nice color pictures.
- Petkovic et al: Recent Applications of IBM's Query by Image Content
(QBIC); IBM Almaden Research Center, San Jose, CA 95120-6099, SAC '96,
Presents a brief overview of QBIC technology and products, and a summary
of some applications.
- Faloutsos et qal: Efficient and Effective Querying by Image Content;
Journal of Intelligent Information Systems, 3, 231-262(1994)
An excellent technical paper describing the IBM QBIC system, giving details
of the image features and distance functions, indexing details, and experiments
on the effectiveness of QBIC. Two problems are specifically addressed,
non-Euclidean distance measures and the high dimensionality of feature
- no author listed: IBM Database 2 Version 2, Relational Extenders;
A brief description of several extenders forIBM's DB2, including text,
image, video, audio, and fingerprint extenders.
- Okon, C.: IBM's Image Recognition Tech for Databases at Work: QBIC
or not QBIC?; Advanced Imaging Magazine, pp. 63-65, May, 1995.
A short, breezy, layman's view of QBIC.
- Holt, B. & Hardwick, L.: Retrieving Art Images by Image Content:
the UC Davis QBIC Project; Aslib Proceedings, Vol 46, n. 10, October,
1994, pp. 243-248.
Describes the construction of a pilot database for retrieving art images
based on what they look like, rather than relying on text indexing. Uses
IBM QBIC as the search engine. Preliminary findings of the project are
- Hafner et al: Efficient Color Histogram Indexing for Quadratic From
Distance Functions; IBM Almaden Research Center, San Jose, Ca 95120-6099,
IEEE Trans. PAMI, Vol. 17, N.7, pp. 729-736, July 1995.
In image retrieval based on color, the weighted distance between color
histograms of two images, represented as a quadratic form, may be defined
as a match measure. However, this distance measure is computationally expensive,
and it operates on high-dimensional features. This paper proposes the use
of low-dimensional, simple-to-computer distance measures between the color
distributions, and shows that these are lower bounds on the histogram distance
- Aigrain, P.: Image and Sound Digital Libraries Need More Than Storage
and Networked Access; Institut de Recherche en Informatique de Toulouse,
Universite Paul Sabatier, 118 route de Narbonne, 31062 Toulouse Cedex,
France, 7 pp.
Good general paper that argues for the need for content-based retrieval,
content-skimming and abstracted representations, and content-based user
interfaces for interactive viewing and listening.
- Pentland et al: View-Based and Modular Eigenspaces for Face Recognition;
Perceptual Computing Section, The Media laboratory, MIT, Cambridge, MA
02139 IEEE CVPR, 1994, 11pp.
This paper describes experiments with eigenfaces for recognition,
verification, and interactive search in a large-scale face database. In
mathematical terms, they find the principal components of the distribution
of faces, or the eigenvectors of the covariance matrix of the set of face
images. These eigenvectors can be thought of as a set of features which
together characterize the variation between face images. The eigenvector
can be displayed as a sort of ghostly face which is called an eigenface.
Questions of overall accuracy and sensitivity to orientation, scale, and
illumination are addressed.
- Tamura et al: Image Database Systems: a Survey; Electrotechnical Laboratory,
1 1-4 Umezono, Sakura-mura, Niihar-gu, Ibaraki 305, Japan Pattern Recognition,
vol. 17, n. 1, 1984, pp 29-43.
Although an old paper, this survey still provides an excellent summary
of image database systems.
- Jacobs et al: Fast Multiresolution Image Querying; Department of Computer
Science and Engineering University of Washington, Seattle, WA 98195-2350
ACM SIGGRAPH 1995, 11 pp., Computer Graphics Proceedings,
Annual Conference Series, 1995.
Presents a method for searching in an image database using a query image
that is similar to the intended target. The query image may be a hand-drawn
sketch or a scan of the image to be retrieved. The searching algorithm
makes use of multiresolution wavelet decompositions of the query and database
images. The coefficients of these decompositions are distilled into small
"signatures" for each image, and an "image query metric"
operates on these signatures. Experiment with hundreds of queries in databases
of 1000 and 20,000 images show dramatic improvement, in both speed and
success rate, over using conventional norms.
- Liu et al: Periodicity, Directionality and Randomness: Wold Features
for Perceptual Pattern Recognition; Perceptual Computing Section, The Media
laboratory, MIT, Cambridge, MA 02139 Proceedings of the 12th ICPR
6pp., Jerusalem, Israel, October, 1994.
In applications such as image retrieval, it is important that features
used by the system in pattern comparison provide good measures of "perceptual
similarity." A new set of features and an image model are presented
based on the three mutually orthogonal components produced by the 2-D Wold
decomposition of random fields. The effectiveness of the new Wold features
for retrieving perceptually similar natural textures is demonstrated.
- Cawkell, A.: Picture-queries and Picture Databases; Citech, Ltd., Iver,
Bucks, UK, Journal of Information Science, 19 (1993) pp. 409-423,
A non-mathematical exposition of existing image database systems, indexing
by query pictures, examples of query-image systems, and database organization.
- Gong et al: An Image Database System with Content Capturing and Fast
Image Indexing Abilities; School of Electrical and Electronic Engineering,
Nanyang Technological University, Nanyang Avenue, Singapore 2263. IEEE
Int. Conf on Multimedia, Computing, and Systems, Boston, May 1994,
Describes a system very similar to QBIC, using both keywords and image
properties in the query.
- Barber et al: Efficient and Effective Querying by Image Content; IBM
Almaden Research Center, San Jose, Ca 95120-6099, Journal of Intelligent
Information Systems, 3, pp 231-262 Kluwer, 1994.
A set of features and similarity measures allowing query by image content
are described, along with the QBIC system. The effectiveness of the system
with normalized precision and recall experiments is demonstrated. The problems
of non-Euclidean distance measures and high dimensionality of feature vectors
are specifically addressed
- Jain, R. et al: Workshop Report: NSF-ARPA Workshop on Visual Information
Management Systems, 20 pp.; Electrical and Computer Engineering, University
of California at San Diego, La Jolla, CA 92093-0407.
Summarizes results of a workshop held on June 19, 1995 in the areas of
visual information systems, image databases, video databases, and infrastructure.
Issues and research topics for each of these areas are indicated. For bibliography,
- Hermes et al Image retrieval for information Dept of Comp. Sci., U
of Bremen, Germany Storage and Retrieval for Images and Video Databases
III; IS&T/SPIE Symposium on Electronic Imaging Science and Technology,Feb
95, Vol. 2420, pp.281-290.
Project IRIS (Image Retrieval for Information Systems) combines methods
and technique in computer vision and AI to automatically generate content
descriptions of images in a textual form. IRIS then retrieves the images
by means of text retrieval using the IBM SearchManager/6000.
5 - Papers on Medical Imaging:
- Wong, et al Interactive query and visualization of medical image data
on the World Wide Web, Proc. Int. Soc. for Optical Eng., SPIE medical
Imaging,Vol. 2707-40, Newport Beach, CA Feb. 10-15, 1996.
This paper presents the system design and tools for interactive query and
visulaization of medical images on the Web. Examples from breast and brain
imaging applications are used to illustrate the operations and capabilities
of such tools.
- Wong, et al A Hospital Integrated Framework for Multimodality Image
Base Mmanagement, IEEE Trans. Systems, Man, and Cybernetics, July,
1996 (in press)
Describes a hospital integrated framework of multimodality image database
management for digital radiology of the future. The paper describes the
three hierarchical components (1) a hospital-integrated picture archiving
and communication system, (2) a medical image database system, and (3)
a set of image-based medical applications.
- Special Invited Issue on Medical Image Databases,J. Computerized
Medical Imaging and Graphics, 1996
This special issue is completely devoted to medical image databases. It
contains papers on content-based retrieval, database systems, image data
modeling, and temporal image databases.
6 - Papers primarily on video retrieval:
- Kanade et al: Immersion into Visual Media: New Applications of Image
Understanding, IEEE Expert Intelligent Systems, Vol. 11, No. 1,
Feb 96, pp. 73-80.
Provides an overview of the CMU digital library visual indexing, retrieval,
and presentation. Of interest is the analysis of audio to aid in the partitioning
and labeling of frames.
- Boreczsky, J. and Rowe, L.: Comparison of video shot boundary detection
techniques, SPIE, San Jose, CA, Feb 1996.
This paper covers five of the most popular boundary detection algorithms
for video-frame boundary detection over a large video database with manually
established ground truth. The results show that there is no single algorithm
satisfying all types of content, i.e., news, films, commercials, and that
variations of histogram-based algorihms are the best. This is the only
paper on this topic that evaluates algorithms in such a systematic manner.
- Zhang et al: Automatic parsing of News Video; Institute of Systems
Science, National University of Singapore, IEEE Conference on Multimedia
Computing and Systems, Boston, May, 1994.
Video content parsing is possible when one has an a priori model
of the structure of a video based on domain knowledge. This paper presents
work on using domain knowledge to parse the content of news video programs.
Approaches have been developed for locating and identifying frame structure
models based on temporal and spatial structure of news video data, along
with algorithms to apply these models in parsing news video. Experimental
results are discussed in detail.
- Ahanger et al Video query formulation Dept of Elect, Comp, and Systems
Eng., Boston Univ. and Siemans, Princeton, NJ Storage and Retrieval
for Images and Video Databases III; IS&T/SPIE Symposium on Electronic
Imaging Science and Technology,Feb 95, Vol. 2420, pp.281-290.
Reviews current video retrieval, describes video attributes and the issues
related to video data. Identifies queryable attributes unique to video
data, namely audio, temporal structure, motion, and events. The approach
is based on visual query methods to describe predicates interactively while
providing feedback that is as similar as possible to the video data. An
initial prototype of the visual query system for video data is presented.
- Rowe et al Indexes for user access to large video databases Computer
Science EECS, U of Calif, Berkeley, CA Storage and Retrieval for Images
and Video Databases II; IS&T/SPIE Symposium on Electronic Imaging Science
and Technology, Feb 95, Vol. 2185, pp.150-161.
This paper describes the design and implementation of a metadata database
and query interface for locating a video of interest in a large video database.
The design was guided by the results of interviews with users. A mixed-mode
query interface was built that allow a user to select a set of videos and/or
frame sequences and incrmentally modify the answer set.