Selective Covariance-Based Localization, Classification and Tracking in Video Streams from Multiple Cameras

This paper proposes a novel selective covariance-based method for human localization, classification and tracking in video streams from multiple cameras. Such methods are crucial for security and surveillance systems, smart environments and robots. The method is called selective covariance-based because before classifying the object into this or that class (in this case the classes are the different people being tracked) we use covariance descriptors and sort out (select) definite regions, which are typical for the class of objects we deal with (people). In our case, the region being sorted out is the human head and shoulders. We develop and describe new feature functions for covariance region descriptors and compare the efficiency of their application to that of basic feature functions. Moreover, we propose and evaluate a mask, filtering out the most of the background information from the target area. The use of the proposed feature functions and mask significantly improved the human classification performance (from 75% when using basic feature functions to 94.6% accuracy with the proposed method) while keeping computational complexity moderat.

References

[1] Lowe D.G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, vol. 60, no. 2, pp. 91-110. DOI: 10.1023/B:VISI.0000029664.99615.94

[2] Fazli S., Pour H.M., Bouzari H. Particle filter based object tracking with sift and color feature. International Conference on Machine Vision, 2009, pp. 89-93. DOI: 10.1109/ICMV.2009.47

[3] Belongie S., Malik J., Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, vol. 24, no. 4, pp. 509-522. DOI: 10.1109/34.993558

[4] Ioffe S., Forsyth D.A. Probabilistic methods for finding people. International Journal of Computer Vision, 2001, vol. 43, no. 1, pp. 45-68. DOI: 10.1023/A:1011179004708

[5] Elzein H., Lakshmanan S., Watta P. A motion and shape-based pedestrian detection algorithm. IEEE Intelligent Vehicles Symposium. Proc., 2003, pp. 500-504. DOI: 10.1109/IVS.2003.1212962

[6] Liu H., Wang L., Sun F. Mean-shift tracking using fuzzy coding histogram. International Journal of Fuzzy Systems, 2014, vol. 16, no. 4, pp. 457-467.

[7] Comaniciu D., Ramesh V., Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, vol. 25, no. 5, pp. 564-577. DOI: 10.1109/TPAMI.2003.1195991

[8] Perez P., Hue C., Vermaak J., Gangnet M. Color-based probabilistic tracking. European Conference on Computer Vision, 2002, pp. 661-675.

[9] Tuzel O., Porikli F., Meer P. Region covariance: A fast descriptor for detection and classification. European Conference on Computer Vision, 2006, pp. 589-600.

[10] Porikli F., Tuzel O., Meer P. Covariance tracking using model update based on Lie algebra. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, vol. 1, pp. 728-735. DOI: 10.1109/CVPR.2006.94

[11] Wu Y., Cheng J., Wang J., Lu H. Real-time visual tracking via incremental covariance tensor learning. Proceedings of IEEE International Conference on Computer Vision, 2009, pp. 1631-1638. DOI: 10.1109/ICCV.2009.5459369

[12] Wu Y., Wang J., Lu H. Robust Bayesian tracking on Riemannian manifolds via frag-ments-based representation. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2009, pp. 765-768. DOI: 10.1109/ICASSP.2009.4959696

[13] Zivkovic Z. Improved adaptive Gaussian mixture model for background subtraction. Proceedings of International Conference on Pattern Recognition, 2004, vol. 2, pp. 28-31. DOI: 10.1109/ICPR.2004.1333992

[14] Viola P., Jones M.J. Robust real-time face detection. International Journal of Computer Vision, 2004, vol. 57, no. 2, pp. 137-154. DOI: 10.1023/B:VISI.0000013087.49260.fb

[15] Watada J., Musaand Z.B. Tracking human motions for security system. SICE Annual Conference, 2008, pp. 3344-3349. DOI: 10.1109/SICE.2008.4655242

[16] Alfimtsev A.N. Fuzzy aggregation of multimodal information in an intelligent interface. Programmnyeprodukty i sistemy [Software & Systems], 2011, no. 3, pp. 10 (in Russ.).

[17] Bellotto N., Hu H. Multisensor-based human detection and tracking for mobile service robots. IEEE Transactions on Systems, Man, and Cybernetics, 2009, vol. 39, no. 1, pp. 167-181. DOI: 10.1109/TSMCB.2008.2004050