Real-time multi-view human action recognition using a wireless camera sensor network

This thesis describes the implementation of real-time, multiple-view camera network used for human action recognition system. The fusion technique presented here shows how information obtained from multiple views using a network of cameras can be effectively combined to yield a reliable and fast human action recognition system. A score-based fusion technique is used for combining information from multiple cameras that can handle arbitrary orientation of the subject with respect to the cameras. This fusion technique does not rely on a symmetric deployment of the cameras and does not require that camera network deployment configuration be preserved between training and testing phases. Being a real-time implementation on a distributed embedded camera, the amount of computation needs to be minimum to ensure a higher frame processing rate. To classify human actions, we use motion information characterized by the spatio-temporal shape of a human silhouette over time. By relying on feature vectors that are relatively easy to compute, our technique lends itself to an efficient distributed implementation while maintaining a high frame capture rate. A view specific classifier is trained for all the actions, which differentiates itself from rest of the actions. We evaluate the performance of our system under different camera densities and view availabilities. We note that a recognition accuracy of greater than 90% is achieved in this system. By systematically analyzing the impact of removal of views from the system, we establish the significance of multiple views for action recognition. Finally, we demonstrate the performance of our system in an online setting where the camera network is used to identify human actions as they are being performed. A high classification rate is also observed in the real-time implementation of the system.