Efficient and robust image ranking for object retrieval.

Abstract

This thesis focuses on efficient and effective object retrieval from an unlabelled collection of images. The goal of object retrieval is to, given a query image depicting an object, return the dataset images containing that same object, quickly and accurately. Due to its simplicity and efficiency, it is common to use a “Bag-of-Words” (BoW) model in which each image is represented as a weighted vector of quantised features, known as visual words. Although the BoW retrieval system is efficient, the extraction and quantisation of local image features introduces errors into the retrieval results. We build our retrieval system on the BoW model, proposing three kinds of method to improve the retrieval accuracy: i) refinement of BoW image representation; ii) refinement of image similarity; iii) retrieval result re-ranking. Firstly, a visual thesaurus structure is proposed to discover the spatial relatedness of visual words. Based on these, a spatial expansion method is able to enrich the original query with those spatially related visual words (enriched by a general thesaurus) and spatially related foreground words (enriched by an object-based thesaurus). Therefore, the BoW image representation is improved. The second contribution improves the standard image similarity used in the BoW retrieval system such that the similarity between query/dataset images is better described. We do this by a cross-word image matching scheme, such that matching features mapped to different visual words are able to contribute to the similarity score. Thirdly, we also aim at efficient result re-ranking methods to improve the initial retrieval results. We present two re-ranking methods in this thesis. A context based re-ranking method is based on the analysis of correlated subsets of image dataset, called “contexts”. Images that share contexts are weakly correlated to each other, and should therefore mutually influence each other’s ranking. The initial ranking scores are refined by this contextual information. We also present a ranking verification method that is able to extract a set of reliable query relevant images from the retrieved results and thus can be applied in a number of object retrieval applications. Note that neither method needs to recover low level feature information or prior knowledge from the dataset. Instead, they utilize ranking information during run time. We also revisit the definition of the object retrieval problem and propose a group-query method, in which the query is a collection of images depicting the same object instead of a single query image used in the traditional “query-by-example” methods.