Co-Tracking, Improving online tracking using Co-Training:
This work was performed at the University of California: Santa Cruz under Professor Hai Tao and in co-operation with Feng Tang. This work was published at ICCV 2007 and the paper can be found here.
The main contribution of this work is providing a fusion framework for allowing multiple representations (feature descriptors) of an object, each with its own classifier, be combined in an online tracking framework. The "glue" that allows the different representations to learn online without succumbing to the "self-learning" problem inherent in most online trackers is the Co-Training framework which allows each classifier to provide labeled data to every other classifier.
By no means have we solved the tracking problem. This framework allows for more robust tracking over more difficult sequences. The basic idea is that if at any given time at least one of the classifiers is performing well then tracking will continue, and the good classifier can be providing accurate labels for the other classifiers, so that those classifiers can be improved and perform better in future frames.
The mechanism that allows the system to know which classifier to trust at any given time is based on a weighting scheme which evaluates a classifiers performance at each frame. Currently our weighting system is rather primitive and we are working on improving it.
Some other work is adding on more data representations into the framework in addition to the two we currently have (color histograms and HoG). Our two current ideas of representations to add are motion information and texture descriptors.
Below are some videos demonstrating the system. The videos have four parts. The upper-left corner shows the confidence map of the color classifier. The top-right corner shows the HoG confidence map. The bottom-left corner shows the weighted combined confidence map. The bottom-right corner shows the results of the tracker. The white box represents the tracked object, and the colored boxes represent the difficult background regions selected in Co-Training.
Courtyard
Crosswalk, a sequence used in Shai Avidan's Ensemble Tracking paper
Another sequence used in Shai Avidan's Ensemble Tracking paper
A PETS video
Another PETS video