This paper describes experiments with techniques for tracking hands and recognizing gestures. Complementary techniques are presented for detecting and tracking hands and tools. These techniques are integrated within a system which uses multiple image processing techniques to estimate the position and orientation of a hand. Images of the tracked hand are normalized in orientation and position and then projected into a principal components space. Hand configurations are represented using a probabilistic classification. Gestures are recognized in this space as sequences of hand configurations using finite state machines. 1 Direct Manipulation of Objects as an Interaction Modality Human gesture serves three functional roles [4]: semiotic, ergotic, and epistemic. The semiotic function of gesture is to communicate meaningful information. The structure of a semiotic gesture is conventional and commonly results from shared cultural experience. The good-bye gesture, the American sign language... (http://www-prima.imag.fr/Prima/Homepages/jlc/papers/PUI97.Crowley.pdf)
