A Model of Multimedia Information Retrieval.

Journal of the ACM, 48 (5): 909-970, 2001.

Abstract: A multimedia document may be seen as a complex information object, with components of different kinds, such as text, images, video and sound, all in digital form. An ever growing amount of people need to retrieve the multimedia documents that would prove useful to their information needs. Research on multimedia information retrieval (MIR) has witnessed a booming interest during the last five years or so. A prominent feature of this research trend is its simultaneous but independent materialization within several fields of computer science. The resulting richness of paradigms, methods and systems that has occurred as a result may, on the long run, result in a fragmentation of efforts prone to slow down progress. The primary goal of this study is to promote an integration of methods and techniques for MIR by contributing a conceptual model which places in a unified and coherent perspective the many efforts and results that are being produced under the label of MIR.Besides this foundational goal, the model provides the guidelines for the design of systems that are able to provide a generalized multimedia retrieval service, where the existing forms of retrieval not only coexist in harmony, but can be combined in any desired manner. Following these guidelines, and using an appropriate tool that implements the model, a designer can quickly build a prototype of the system to be developed, and use such a prototype to test its adequacy to the user's functional requirements. The paper presents a prototype of a significant portion of such a tool. Finally, by its viewing MIR as a special form of uncertain implication, this model may be used as a basis for formal investigations on specific aspects of MIR.