CS 545I - Advanced Image and Video Databases, Winter 2001

Friday, 19 Jan 2001. Second lecture CS545I

Clustering Text and Image Features for Automated Image Data Organization

Dr. Kobus Barnard, Computer Science, UC Berkeley (work done with David Forsythe.)

In this talk we introduce an approach for organizing image data which makes use of both features extracted from images and any textual information which accompanies the images. Powerful tools for organizing image data are needed as the amount of such data online is often too large for effective use as is. As the number of images that is available increases, the likelihood that an image of interest is available also increases, but the difficulty of finding it also increases. Therefore, to take full advantage of online image data, tools to help the user find images of use to them are required.

In this work, we take the idea of searching on the conjunction of image features and text one step further. First, the use of simple conjunctions works best with skilled users accessing a database with good keywords. Second, even with these strong assumptions, finding the desired images is still difficult. Methods to organize and present the data to users still are required. We make the obvious claims that solving this problem will be done best by using all available information. Furthermore, the details of the indexing should be hidden from the user. The user should be able to find images that are visually similar to ones of interest without being asked to quantify the similarity (unless by choice). Therefore, we are interested in producing clusters of images that can be used to navigate towards images of interest.