Abstract: In this paper we describe a unified vision of content-based image retrieval (CBIR), which brings together image matching, similarity and recognition. The motivation stems from the observation that there is still a large gap between CBIR end-user expectations and current CBIR possibilities. Shedding light on the problems that traditional CBIR systems experience, we show how matching and recognition can be used to remedy the lack of semantical information in a CBIR system. In particular, we advocate the use of scope-limited metrics for gaining image understanding. We refine the notion of image similarity and derive a powerful, flexible paradigm for CBIR.