2. Introduce discrimination into GPDM (D-GPDM), the classification is take care of by based on MIL. The tem In MIL, we use the learned latent point as the feature,
   and we learn a set of instance prototypes. The MIL learning is based on Diverse Density.

3. Using GPDM to learn the latent space trajectory, then classification is done by matching trajectory in the latent space via method in [1]

4. Introduce discrination into GPDM, the classification is taken care of by Gaussian Process classification. But it is hard to really innovate on the
Gaussian Process Classification part. The GPC inference involves approximation approaches like (loopy) belief propagation, expectation propagation,
variational approximations, or Monte Carlo Sampling.

[1] M. Black and A. Jepson. A probabilistic framework for matching temporal trajectories:Condensation-based recognition of
gestures and expressions. In ECCV, 1998.

The papers that I have been reading

1. Conditional random field

1.1 Lafferty, J., Zhu, X. and Liu, Y. 2004. Kernel conditional random fields: Representation and clique selection. In Proc. Twenty-First International Conference on Machine Learning (ICML). 

1.2 Sanjiv Kumar and Martial Hebert, Discriminative random field, Volume 68, Number 2 / June, 2006, pp. 179-201.

1.3 C. Sutton, A. McCallum, An introduction to Conditional random fields for relational learning, Introduction to Statistical Relational Learning, Ed. Lise Getoor and Ben Taskar, the MIT press.

1.4 C. Sminchisescu, A. Kanaujia, Z. Li, D. Metaxas, Conditional models for contextual human motion recognition, in: IEEE International Conference on Computer Vision, Vol. 2, 2005, pp. 1808--1815.

1.5 B. Taskar, C. Guestrin, and D. Koller. Max-margin markov networks. In Sebastian Thrun, Lawrence Saul, and Bernhard Sch olkopf, editors, Advances in Neural Information Processing Systems 16, 2003.

1.6 Ariadna Quattoni, Michael Collins and Trevor Darrel. Conditional Random Fields for Object Recognition. In Advances in Neural Information Processing Systems 17 (NIPS 2004), 2004.

2 Gaussian Process

2.1 Gaussian Process Regression, Chapter 2, Machine Learning for Pattern Recognition, M. Bishop, MIT Press.

2.2 Gaussian Process Classification, Chapter 3, Machine Learning for Pattern Recognition, M. Bishop, MIT Press.

2.3 N. Lawrence, Probabilistic Non-linear Principle Component Analysis with Gaussian Process Latent Variable Models, Journal of Machine Learning Research 6, pp. 1783-1816, 2005.

2.4 Wang, J. M., Fleet, D. J., Hertzmann, A. Gaussian Process Dynamical Models for Human Motion. In IEEE Trans. PAMI. February, 2008. pp. 283-298.

2.5 Yasemin Altun, Thomas Hofmann and Alexander J. Smola. Gaussian process classification for segmenting and annotating sequences. In Proceedings of the Twenty-First International Conference on Machine Learning (ICML 2004), 2004.

2.6 Y. Altun, T. Hofmann, and M. Johnson. Discriminative learning for label sequences via boosting. In Advances in Neural Information Processing Systems (NIPS*15), 2003.

2.7  Urtasun and T. Darrell. Discriminative Gaussian process latent variable model for classification. In 24th International Conference on Machine Learning, 2007.

3, Combine multiple data source

3.1 M. Girolami, M. Zhong, Data Integration for classification problems employing gaussian process priors, NIPS 2006.

1.       No need to construct the latent space

2.       No need to learn the latent to ambient and/or ambient to latent mapping

3.       The learning of the correlation coefficients only need very few parameters. GPLVM, GPDM, Spectrial LVM need to tune lots of parameters

4.       The training for obtaining the correlation coefficients are much faster than GPLVM and GPDM.

5.       In laten variable models, we need to assume the parametric form of the probability distribution of the ambient variables given the  latent variables. But in our method we do not need that assumption and thus is more general

6.       As long as there are structural relationships among the state variables, the PLS is able to discover it. 

Some disadvantage of GPLVM and GPDM

1. The kernel gram matrix bias towards the testing points that are similar to the training points. That is only if the training samples are similar to the testing points, the learned coefficients matrix is useful.

In order to verify the idea, I need to

 1) Do predictions using the coefficients learned from kernel pLS, without integrating into the tracking framework. Try to analyze the predictions errors obtained from different training data points.

 2) Try to use training motion cycles that are 'similar' to the testing motions. That only works if we have the testing motions available.

Some unresolbed questions:

1)Do I need to make the data zero-mean or not? I tried on two data sets, and I found that non-centralized kernel PLS is better, though still worse than PLS.

2) how to determine Kernel parameter? now the sigmma (RBF kernel parameter in the kernel PLS) is determined by cross validatation.  

2. What is the advantage of our method? Using examples to illustrate 'smart sampling'

3. Partition the sample space in upper and lower body to see the performance.

4. Try a more sophisticated dynamical model, at least a second order markov model. "Your dynamical model is very simple, just a 1st order markov model. Why don't you use something more complicated, or at least a second order markov model?"

5. Using kernel to capture nonlinearties in the motion space. "Why are you using a linear relationship between left and right side of the body? "

 6. Analyze the error cases. "I also concern about the error cases shown in the vidoes. They should have been analyzed and discussed. "

Improve the tracking performance by improving the algorithm from the following aspects:

1. More accurate and robust background subtraction method.

2.More powerful dynamic model. Reference:learning and classification of complex dynamics by North PAMI 2000.

3. The parameters of the Kalman filter, including A, H or the noise covariance matrix, can be learned in someway. Reference:Qiang Wang, Learning object intrinsic structure for robust visual tracking. CVPR 2003.

4.Good image measurements such as optical flow. Reference:T.B. Moeslund and E. Granum. Multiple cues used in model-based human motion capture. In International Conference on Face and Gesture Recognition, 2000.  S. X. Ju, M. J. Black, and Y. Yacoob. Cardboard people: A parameterized model of articulated image motion. In Intl. Conf. on
Automatic Face and Gesture Recognition, pages 38–44, 1996.

5. Incoporate real measurement into the Kalman filter update. But how to? Reference:S. Wachter and H.-H. Nagel. Tracking persons in monocular image sequences. Computer Vision and Image Understanding, 74(3):174–192, June 1999.

6. Incoporate temporal information into the learned correlation model. ReferenceI. Kakadiaris and D. Metaxas. Model-based estimation of 3D human motion. IEEE PAMI, 22(12):1453–1459, December 2000.

7. Try Kernel PLS to capture the non-linear dynamics between left- and right-side motions.

The current 3D tracking work can be improved from the following aspects:

1.The interpretation of the PLS model including the loadings, the LVs and the Bipmap.

2.How significant the learned correlation is? This can be done by visualizing the predicted left-side joint angles and the grond truth right side angles.

3.Analyze the performance of the tracker to the (i) amount of training data.(ii) the variance captured by the PLS. In addition, answer " what criteria measures the goodness of the learned correlation model? Is it the variance captured by the PLS regression? IF not, then what?"

4. Put examplar images in the 3D latent-space visualization of the correlation model.

5. The presentation of the tracking algorithm in the ICCV paper Sec.4 could be improved. See Qiang Wang CVPR 2003

1. Does it exist an inverse mapping from the embedding space to the original high-dimensional state space?

2. What is the difference between these three methods?

Two papers applied dimensionality reduction to visual tracking

Qiang Wang et al. Learning object intrinsic structure for robust visual tracking. CVPR 2003

Almed Elgammal et al. Inferring 3D pose from silhouettes using activity manifold learning.

The anatomy of a color profile

You can examine the contents of profiles with ICC Profile Inspector, which you can download from the ICC Resource Center by clicking on CONTINUE and following the instructions. When you run it, click Browse... to load the profile. The header information and tag table are displayed. Double-click on a tag to see its contents. A typical tag, gXYZ (green primary color), is illustrated on the right.

There are three classes of profile, indicated in the Device Class field of the header: Input ('scnr'), Display ('mntr'), and Output ('prtr'). Each has a set of required tags and a set of optional tags. Display profiles are used to define color spaces. Many monitor profiles contain the ://developer.apple.com/techpubs/mac/ACI2.5/WhatsNewColorSync25.6.html">vcgt tag for setting lookup tables when a loader program is run, i.e., for calibrating the monitor.

The meaning of the tags is specified in the formidable (126 page) ICC File Format for Color Profiles (Version 4.0.0), which is rich in content and readable if you skip the bureaucratic parts (strong coffee recommended). In most cases the meaning will be obvious from the ICC Profile Inspector display. The basic tag signatures (4-character abbreviations) are

• desc is the description of the profile, used in PW Pro selection boxes.

• rXYZ, gXYZ and bXYZ specify the R, G, and B primaries that determine the gamut of the color space or device.

• wtpt is the white point. The two standard white points are 6500K (D65): X=0.95045, Y=1.0, Z = 1.08905, and 5000K (D50): X=0.96429, Y=1.0, Z=0.82510. Y is always 1.0 and Z varies the most. Used for absolute colorimetric gamut mapping, which is of little interest to photographers.

• rTRC, gTRC and bTRC are the R, G and B Tone Reproduction Curves that define device or color space gamma in Input and Monitor profiles. An example (gTRC) is shown on the right. Gamma is often indicated on the upper right.

•  
  gamma = -ln(y₅)/0.69315
If it isn't, it can be calculated from
  .
where x' = x/x_max ,  y' = y/y_max , and y₅ = y' at x'=0.5 (the middle of the x-axis) = 0.22 (in the curve on the right). Typical values: y₅ = 0.287 for gamma = 1.8, 0.25 for gamma = 2.0; 0.218 for gamma = 2.2. [ This equation can be easily derived from (y/y_max) = (x/x_max)^gamma ].
 
• AToBn or BToAn are gamut mapping tables used in printer profiles. A refers to the device; B refers to the profile connection space (PCS); n = 0 for perceptual, 1 for colorimetric or 2 for saturation rendering intent. BToAn tags are used for printing; AToBn are used for proofing (previewing the print). These tables are large. Profiles that contain them can be several hundred kilobytes-- sometimes over a megabyte. TRC tags are omitted in printer profiles. All the printer profiles I've examined have gamt (out-of-gamut) tags, but little information is available about them. The best way to examine the actual performance of printer profiles is with Gamutvision.

Manufacturer-private tags make it difficult to figure out what a profile is supposed to do. An example is vcgt (Video card gamma tag, registered to Apple), widely used in monitor profiles (Adobe, Monaco, Praxisoft, etc.) to set video card LUT's. It is not to be found in the ICC specification. Another example: Mtbx, in the monitor profiles created by Adobe Gamma and MonacoEZcolor. Try searching Google and you'll find pages on mountain bikes. Here is what the spec says (p. 3): "Private data tags allow CMM developers to add proprietary value to their profiles. By registering just the tag signature and tag type signature, developers are assured of maintaining their proprietary advantages while maintaining compatibility with this specification. However, the overall philosophy of this format is to maintain an open, cross-platform standard, therefore the use of private tags should be kept to an absolute minimum." And that is how things would be in the best of all possible worlds.

Of course a profile's actual performance is more than the sum of its parts. To see how a profile functions with different rendering intents under a variety of conditions, you'll need Gamutvision.