1.       No need to construct the latent space

2.       No need to learn the latent to ambient and/or ambient to latent mapping

3.       The learning of the correlation coefficients only need very few parameters. GPLVM, GPDM, Spectrial LVM need to tune lots of parameters

4.       The training for obtaining the correlation coefficients are much faster than GPLVM and GPDM.

5.       In laten variable models, we need to assume the parametric form of the probability distribution of the ambient variables given the  latent variables. But in our method we do not need that assumption and thus is more general

6.       As long as there are structural relationships among the state variables, the PLS is able to discover it. 

Some disadvantage of GPLVM and GPDM

"GPLVM’s lack of latent space prior makes it somewhat agnostic in visual inference applications, where it is useful to penalize drifts from the manifold of typical configurations. "----- Kanaujia, ICCV 2007.

"Urtasun et al [22] use GPLVM to track walking based on image tracks of the human joints obtained using the WSL tracker of Jepson’s et al. For more expressive kinematic representations, and in order to compensate for GPLVM’s lack of latent space prior, the authors [22] use an augmented, constrained (latent, ambient) state for tracking. This is feasible but, once again, renders the state estimation problem high-dimensional."  ----- Kanaujia, ICCV 2007.

references;

[22] R. Urtasun, D. Fleet, A. Hertzmann, and P. Fua. Priors for people tracking in small training sets. In ICCV, 2005.

I just tried using kernel pLS to extract the nonlinearies in the 3D human motion data, and then used the extracted coefficients matrix for prediction in the RBPF tracking framework. However I found that the tracking error is very high. The reason, I am thinking, could be:

1. The kernel gram matrix bias towards the testing points that are similar to the training points. That is only if the training samples are similar to the testing points, the learned coefficients matrix is useful.

In order to verify the idea, I need to

 1) Do predictions using the coefficients learned from kernel pLS, without integrating into the tracking framework. Try to analyze the predictions errors obtained from different training data points.

 2) Try to use training motion cycles that are 'similar' to the testing motions. That only works if we have the testing motions available.

Some unresolbed questions:

1)Do I need to make the data zero-mean or not? I tried on two data sets, and I found that non-centralized kernel PLS is better, though still worse than PLS.

2) how to determine Kernel parameter? now the sigmma (RBF kernel parameter in the kernel PLS) is determined by cross validatation.  

1. Compare with PCA in terms of latent space mapping

2. What is the advantage of our method? Using examples to illustrate 'smart sampling'

3. Partition the sample space in upper and lower body to see the performance.

4. Try a more sophisticated dynamical model, at least a second order markov model. "Your dynamical model is very simple, just a 1st order markov model. Why don't you use something more complicated, or at least a second order markov model?"

5. Using kernel to capture nonlinearties in the motion space. "Why are you using a linear relationship between left and right side of the body? "

 6. Analyze the error cases. "I also concern about the error cases shown in the vidoes. They should have been analyzed and discussed. "

Today, Professor Gang Qian called me and discussed with me several questions regarding my work on 3D articulated tracking. He has several good points. I think the discussion with him also stimulate me to think deeply about the advantage of PLS-RBPF and why it can be called RBPF inference.He pointed the following points:1. The Kalman prediction using correlation B effectively restrict the samples only to be draw from the temproal dynamics that conforms well to human motion, eliminating samples that does not conform to human motion structure. 2. Dr. Qian thinks that until weighting each sample by the image likilihood, we are not in the RBPF tracking framework, because he thinks that until the "weighting" step, no true measurement is incorporated. I have convinced him that we are in the RBPF tracking framework because of the Kalman Update step which incorporate the true measurement through the auxiliry measurement Ot-1 that is the leaf mean state at the previous time step.3. Dr. Qian thinks that our movement modeling method of learning the correlation using PLS regression is more general in the sense that as long as there are correlations in the joint angle data, we can then learn this from the training data and apply it to tracking. Thus our method is able to track other kinds of motions like jogging, running or even dancing motion sequence.  

Only use view 2 and view 3 of the ASU data. The full set of ASU data is located in Through manual fitting, we have calculated a new camera calibration matrix for view 2 and view 3. Using the new calculated camera calibration matrix the first frame (000050.png) can be fitted. But we can not guarantee that the projection of the 3D angles fit the other images using the new camera calibration matrix.The fitted first frame joint angles, the limb length and limb parameters have been stored in The matlab code for manual fitting the parameters of the first frame (including the joint angle vector of the first frame, the limb lengths and limb parameters and the camera calibration matrix) is located in We will test our algorithm using the code and data located in C:\RA\BodyTracking\RBPF-PLS on ASU data.