We address the problem of establishing point correspondences in images for computing camera pose through Perspective-n-Point (PnP) algorithm. We compute the 3D map i.e. 3D coordinates and visual characteristics of some of the points in the environment through an offline training stage using a set of training images. Given a new test image we apply PnP using the 2D coordinates of 3D points in the image detected by using the 3D map. During the training stage we cluster the SIFT descriptors extracted from training images to obtain 2D-tracks of some of the 3D points in the environment. Each 2D-track consists of a set of 2D image coordinates of a single 3D point in different training images. SfM (Structure from Motion) is performed on these 2D-tracks to obtain the coordinates of the corresponding 3D points. During the test stage, the SIFT descriptors associated the 2D-track of a 3D point is used to recognize the 3D point in a given image. The overall process is similar to visual word framework used in different fields of computer vision. During training, visual word formation is performed through clustering and during testing 3D points are identified through visual word recognition. We experiment with different clustering schemes (k-means and mean-shift) and propose a novel scheme for visual word formation for training stage. We use different matching rules including some of the popular supervised pattern classification methods to perform visual word recognition during test stage. We evaluate these various matching strategies in both stages. In order to achieve robustness against pose variation between train and test images, we explore different ways of incorporating SIFT descriptors extracted from synthetic views generated from the training images. We also propose an exact acceleration strategy for mean-shift computation.
