CS 545I - Advanced Image Databases, W 95/96

References

1a - Web sites re Query by Image Content

1. QBIC System http://wwwqbic.almaden.ibm.com
   Try out the IBM QBIC system. Comments for improvement are welcome.
2. Berkeley Digital Library Project,
   http://elib.cs.berkeley.edu/cypress/
   Try out the Cypress image retrieval system. Not a very friendly front end.
3. ImageEngine
   http://www.cml.upmc.edu/ImageEngine.html
   A medical image retrieval system being developed by the Section of Medical Informatics, Dept of Medicine, U of Pittburgh.
4. MIT Photobook
   http://www.media.mit.edu/groups/casr/picard.html
   Also visit sites of Profs. Pentland and Lippman.
5. Jacob (Just A COntent-Based) Query system for Video DB , U of Palermo
   http://wwwcsai.diepa.unipa.it:80/research/projects/jacob
   Describes system and lets you try some queries. Not a very friendly front end.
6. Virage Corp.
   http://www.virage.com
   Lets you try queries. Virage is a small image retrieval company in the San Diego area. At the present time, the only other commercial offering besides IBM QBIC.
7. Excalibur http://www.excalib.com
8. WebSEEk http://disney.ctr.columbia.edu/WebSEEk
9. VisualSEEk http://disney.ctr.columbia.edu/VisualSEEk
10. WebSeer (University of Chicago) http://infolab.cs.uchicago.edu.webseer
11. Image Surfer (Interpix Software Corporation) http://www.interpix.com
12. Publishers Depot http://www.publishersdepot.com
13. Boston University (description of research, papers, images) http://www.cs.bu.edu/groups/ivc/Home.html

1b - Related to images (not content-based, but interesting)

1. Yahoo Search engine
   http://www.yahoo.com
   Search for multimedia, pictures,archives; online image archives OR picture archives.
2. Getty Art History Information Program,
   http:www.ahip.getty.edu/ahip
   How to build image databases; image capture; resources for indexing art.

1c - University work or courses

1. MIT Media Lab has many good projects, especially from Profs. Picard, Pentland, and Lippman
2. Prof. Sclaroff, Boston university
   http://cs-pub.bu.edu/faculty/sclaroff/courses/cs835-95/projects.html
   Description of student projects
3. Sean Landis Term Paper
   http://www.tc.cornell.edu/~landis/
   A very informative student term paper describing a system for "Content-based image retrieval systems for interior design."

1d - Database stuff

1. IBM DB2 database
   http://www.software.ibm.com
   Mouse "Data Managemen" icon and then DB2 under that. Example of database offering on the Web that integrates SQL and WWW, both at the user interface and internally.
2. Getty Museum
   http://simpr1.compapp.dcu.ie/images/query.html
   Natural language interface to image database (not query by image content)

1d - Digital Library Inititative
The Digital Library Iniative consists of six university projects funded jointly by NSF, DARPA, NASA. The funding is about one million dollars a year for each institution. Some of these projects, particularly UC Santa Barbara, CMU, UC Berkeley, and U of Illinois, have an image retrieval aspect. As of March 1996, none of the Web sites is particular strong on details of image analysis.

1. University of California, Berkeley : "The Environmental Electronic Library: A Prototype of a Scalable, Intelligent, Distributed Electronic Library" http://elib.cs.berkeley.edu/
2. University of California, Santa Barbara: "The Alexandria Project: Towards a Distributed Digital Library with Comprehensive Services for Images and Spatially Referenced Information" http://alexandria.sdc.ucsb.edu
3. Carnegie Mellon University: "Informedia: Integrated Speech, Image and Language Understanding for Creation and Exploration of Digital Video Libraries" http://informedia.cs.cmu.edu
4. University of Illinois at Urbana-Champaign : "Building the Interspace: Digital Library Infrastructure for a University Engineering Community" http://www.grainger.uiuc.edu/dli
5. University of Michigan : "The University of Michigan Digital Libraries Research Proposal" http://www.sils.umich.edu/UMDL/HomePage.html
6. Stanford University : "The Stanford Integrated Digital Library Project" http://www-diglib.stanford.edu

1f- Just for fun

1. Browsing for art:
   http://www.cmp.ucr.edu photography exibits
   
2. http://www.metmuseum.org Metropolitan Museum of Ar
   
3. http://www.rosprint.ru/art/museum/pushkin Pushkin Museum
   
4. http://sunsite.unc.edu/wm WebMuseum, Paris
   
5. http://www.echonyc.com/~whitney Whitney Museum of Art
   

2- Books The book we recommend has several sections on content based retrieval for image and video DB - the only book to address this topic
Furht, Smoliar, and Zhang: Video and Image Processing in Management Systems; Florida Atlantic University, Boca Raton, FL. Kluwer Academic Publishers, 1995 (ABOUT $150!)

3-Collections and Journals

1. Storage and Retrieval for Images and Video Databases II; IS&T/SPIE Symposium on Electronic Imaging Science and Technology,Feb 95, SPIE Vol. 2185.
2. Storage and Retrieval for Images and Video Databases III; IS&T/SPIE Symposium on Electronic Imaging Science and Technology,Feb 95, SPIE Vol. 2420.
3. Multimedia Systems Journal
4. Proc. ACM Multimedia '93, Anaheim, CA, Aug.1993.
5. Proc. ACM Multimedia '94, San Francisco CA, Oct.1994

.

4 - Papers primarily on still imagery:

1. P. Aigrain et al.: Content-based Representation and Retrieval of Visual Media: A State-of-the art Review,Multimedia Tools and Applications, Kluwer Academic Publishers, Vol. 3, No. 3, November 1996.
2. Pentland et al: Photobook: Tools for Content-Based Manipulation of Image Databases; Perceptual Computing Section, The Media laboratory, MIT, Cambridge, MA 02139, SPIE, Storage and Retrieval Image and Video Databases II, No. 2185, Feb 1994
   Describes Photobook, a set of interactive tools for browsing and searching images and image sequences. Direct search on image content is made possible by the use of semantics-preserving image compression. The paper describes three Photobook tools, one that allows search based on grey-level appearance, one that uses 2-D shape, and a third that allows search based on textural properties. W. Niblack et al.: Efficient and Effective Querying by Image Content; Special Issue on Integrating Artificial Intelligence and Database Technology, Journal of Intelligent Information Systems, No. 3, 1994.
3. W. Niblack et al. Efficient and Effective Querying by Image Content, special issue on Integrating Artificial Intelligence and Database Technology, Journal of Intelligent Information Systems, No. 3, 1994.
4. Flickner et al: Query by Image and Video Content: The QBIC System; IBM Almaden Research Center, San Jose, CA 95120-6099, Computer, Vol. 28, No. 9, pp.23-31, Sept 1995
   Presents an overview of QBIC. Nice color pictures.
5. Petkovic et al: Recent Applications of IBM's Query by Image Content (QBIC); IBM Almaden Research Center, San Jose, CA 95120-6099, SAC '96, 5 pp.
   Presents a brief overview of QBIC technology and products, and a summary of some applications.
6. Faloutsos et qal: Efficient and Effective Querying by Image Content; Journal of Intelligent Information Systems, 3, 231-262(1994)
   An excellent technical paper describing the IBM QBIC system, giving details of the image features and distance functions, indexing details, and experiments on the effectiveness of QBIC. Two problems are specifically addressed, non-Euclidean distance measures and the high dimensionality of feature vectors.
7. no author listed: IBM Database 2 Version 2, Relational Extenders; 4pp., 1995?
   A brief description of several extenders forIBM's DB2, including text, image, video, audio, and fingerprint extenders.
8. Okon, C.: IBM's Image Recognition Tech for Databases at Work: QBIC or not QBIC?; Advanced Imaging Magazine, pp. 63-65, May, 1995.
   A short, breezy, layman's view of QBIC.
9. Holt, B. & Hardwick, L.: Retrieving Art Images by Image Content: the UC Davis QBIC Project; Aslib Proceedings, Vol 46, n. 10, October, 1994, pp. 243-248.
   Describes the construction of a pilot database for retrieving art images based on what they look like, rather than relying on text indexing. Uses IBM QBIC as the search engine. Preliminary findings of the project are presented.
10. Hafner et al: Efficient Color Histogram Indexing for Quadratic From Distance Functions; IBM Almaden Research Center, San Jose, Ca 95120-6099, IEEE Trans. PAMI, Vol. 17, N.7, pp. 729-736, July 1995.
    In image retrieval based on color, the weighted distance between color histograms of two images, represented as a quadratic form, may be defined as a match measure. However, this distance measure is computationally expensive, and it operates on high-dimensional features. This paper proposes the use of low-dimensional, simple-to-computer distance measures between the color distributions, and shows that these are lower bounds on the histogram distance measure .
11. Aigrain, P.: Image and Sound Digital Libraries Need More Than Storage and Networked Access; Institut de Recherche en Informatique de Toulouse, Universite Paul Sabatier, 118 route de Narbonne, 31062 Toulouse Cedex, France, 7 pp.
    Good general paper that argues for the need for content-based retrieval, content-skimming and abstracted representations, and content-based user interfaces for interactive viewing and listening.
12. Pentland et al: View-Based and Modular Eigenspaces for Face Recognition; Perceptual Computing Section, The Media laboratory, MIT, Cambridge, MA 02139 IEEE CVPR, 1994, 11pp.
    This paper describes experiments with eigenfaces for recognition, verification, and interactive search in a large-scale face database. In mathematical terms, they find the principal components of the distribution of faces, or the eigenvectors of the covariance matrix of the set of face images. These eigenvectors can be thought of as a set of features which together characterize the variation between face images. The eigenvector can be displayed as a sort of ghostly face which is called an eigenface. Questions of overall accuracy and sensitivity to orientation, scale, and illumination are addressed.
13. Tamura et al: Image Database Systems: a Survey; Electrotechnical Laboratory, 1 1-4 Umezono, Sakura-mura, Niihar-gu, Ibaraki 305, Japan Pattern Recognition, vol. 17, n. 1, 1984, pp 29-43.
    Although an old paper, this survey still provides an excellent summary of image database systems.
14. Jacobs et al: Fast Multiresolution Image Querying; Department of Computer Science and Engineering University of Washington, Seattle, WA 98195-2350 ACM SIGGRAPH 1995, 11 pp., Computer Graphics Proceedings, Annual Conference Series, 1995.
    Presents a method for searching in an image database using a query image that is similar to the intended target. The query image may be a hand-drawn sketch or a scan of the image to be retrieved. The searching algorithm makes use of multiresolution wavelet decompositions of the query and database images. The coefficients of these decompositions are distilled into small "signatures" for each image, and an "image query metric" operates on these signatures. Experiment with hundreds of queries in databases of 1000 and 20,000 images show dramatic improvement, in both speed and success rate, over using conventional norms.
15. Liu et al: Periodicity, Directionality and Randomness: Wold Features for Perceptual Pattern Recognition; Perceptual Computing Section, The Media laboratory, MIT, Cambridge, MA 02139 Proceedings of the 12th ICPR 6pp., Jerusalem, Israel, October, 1994.
    In applications such as image retrieval, it is important that features used by the system in pattern comparison provide good measures of "perceptual similarity." A new set of features and an image model are presented based on the three mutually orthogonal components produced by the 2-D Wold decomposition of random fields. The effectiveness of the new Wold features for retrieving perceptually similar natural textures is demonstrated.
16. Cawkell, A.: Picture-queries and Picture Databases; Citech, Ltd., Iver, Bucks, UK, Journal of Information Science, 19 (1993) pp. 409-423, pub. Elsevier.
    A non-mathematical exposition of existing image database systems, indexing by query pictures, examples of query-image systems, and database organization.
17. Gong et al: An Image Database System with Content Capturing and Fast Image Indexing Abilities; School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 2263. IEEE Int. Conf on Multimedia, Computing, and Systems, Boston, May 1994, 10 pp.
    Describes a system very similar to QBIC, using both keywords and image properties in the query.
18. Barber et al: Efficient and Effective Querying by Image Content; IBM Almaden Research Center, San Jose, Ca 95120-6099, Journal of Intelligent Information Systems, 3, pp 231-262 Kluwer, 1994.
    A set of features and similarity measures allowing query by image content are described, along with the QBIC system. The effectiveness of the system with normalized precision and recall experiments is demonstrated. The problems of non-Euclidean distance measures and high dimensionality of feature vectors are specifically addressed
19. Jain, R. et al: Workshop Report: NSF-ARPA Workshop on Visual Information Management Systems, 20 pp.; Electrical and Computer Engineering, University of California at San Diego, La Jolla, CA 92093-0407.
    Summarizes results of a workshop held on June 19, 1995 in the areas of visual information systems, image databases, video databases, and infrastructure. Issues and research topics for each of these areas are indicated. For bibliography, see http://www.virage.com/vir-res/
20. Hermes et al Image retrieval for information Dept of Comp. Sci., U of Bremen, Germany Storage and Retrieval for Images and Video Databases III; IS&T/SPIE Symposium on Electronic Imaging Science and Technology,Feb 95, Vol. 2420, pp.281-290.
    Project IRIS (Image Retrieval for Information Systems) combines methods and technique in computer vision and AI to automatically generate content descriptions of images in a textual form. IRIS then retrieves the images by means of text retrieval using the IBM SearchManager/6000.

5 - Papers on Medical Imaging:

1. Wong, et al Interactive query and visualization of medical image data on the World Wide Web, Proc. Int. Soc. for Optical Eng., SPIE medical Imaging,Vol. 2707-40, Newport Beach, CA Feb. 10-15, 1996.
   This paper presents the system design and tools for interactive query and visulaization of medical images on the Web. Examples from breast and brain imaging applications are used to illustrate the operations and capabilities of such tools.
2. Wong, et al A Hospital Integrated Framework for Multimodality Image Base Mmanagement, IEEE Trans. Systems, Man, and Cybernetics, July, 1996 (in press)
   Describes a hospital integrated framework of multimodality image database management for digital radiology of the future. The paper describes the three hierarchical components (1) a hospital-integrated picture archiving and communication system, (2) a medical image database system, and (3) a set of image-based medical applications.
3. Special Invited Issue on Medical Image Databases,J. Computerized Medical Imaging and Graphics, 1996
   This special issue is completely devoted to medical image databases. It contains papers on content-based retrieval, database systems, image data modeling, and temporal image databases.

6 - Papers primarily on video retrieval:

1. Kanade et al: Immersion into Visual Media: New Applications of Image Understanding, IEEE Expert Intelligent Systems, Vol. 11, No. 1, Feb 96, pp. 73-80.
   Provides an overview of the CMU digital library visual indexing, retrieval, and presentation. Of interest is the analysis of audio to aid in the partitioning and labeling of frames.
2. Boreczsky, J. and Rowe, L.: Comparison of video shot boundary detection techniques, SPIE, San Jose, CA, Feb 1996.
   This paper covers five of the most popular boundary detection algorithms for video-frame boundary detection over a large video database with manually established ground truth. The results show that there is no single algorithm satisfying all types of content, i.e., news, films, commercials, and that variations of histogram-based algorihms are the best. This is the only paper on this topic that evaluates algorithms in such a systematic manner.
3. Zhang et al: Automatic parsing of News Video; Institute of Systems Science, National University of Singapore, IEEE Conference on Multimedia Computing and Systems, Boston, May, 1994.
   Video content parsing is possible when one has an a priori model of the structure of a video based on domain knowledge. This paper presents work on using domain knowledge to parse the content of news video programs. Approaches have been developed for locating and identifying frame structure models based on temporal and spatial structure of news video data, along with algorithms to apply these models in parsing news video. Experimental results are discussed in detail.
4. Ahanger et al Video query formulation Dept of Elect, Comp, and Systems Eng., Boston Univ. and Siemans, Princeton, NJ Storage and Retrieval for Images and Video Databases III; IS&T/SPIE Symposium on Electronic Imaging Science and Technology,Feb 95, Vol. 2420, pp.281-290.
   Reviews current video retrieval, describes video attributes and the issues related to video data. Identifies queryable attributes unique to video data, namely audio, temporal structure, motion, and events. The approach is based on visual query methods to describe predicates interactively while providing feedback that is as similar as possible to the video data. An initial prototype of the visual query system for video data is presented.
5. Rowe et al Indexes for user access to large video databases Computer Science EECS, U of Calif, Berkeley, CA Storage and Retrieval for Images and Video Databases II; IS&T/SPIE Symposium on Electronic Imaging Science and Technology, Feb 95, Vol. 2185, pp.150-161.
   This paper describes the design and implementation of a metadata database and query interface for locating a video of interest in a large video database. The design was guided by the results of interviews with users. A mixed-mode query interface was built that allow a user to select a set of videos and/or frame sequences and incrmentally modify the answer set.