The steps in using matlab compiler to call matlab code from C++ enviornment.

1. in Matlab prompt, type

>>mbuild -setup to configure the C++ compiler

2 in Matlab prompt, type

>>mcc -W cpplib:libIMclassify -T link:lib addshape.m colorf.m fea4blocks.m

This will create a shared library from an arbitrary set of M-files (or mex-file). The MATLAB Compiler generates a wrapper file, a libIMclassify.h, a libIMclassify.dll file, a libIMclassify.lib file, a libIMclassify.ctf file. What will used in the C++ application include the .dll, .lib, .h and .ctf file and other matlab files (e.g. .mat or .dll or that the matlab .m files depend on).

3 Put the the .dll, .lib, .h and .ctf file in the folder of the C++ application

4 Write a C++ wrapper that calls the matlab compiler generated shared library

In my image categorization project, I wrote the following C++ wrapper named main.cpp

#ifdef __APPLE_CC__
#include
#include
#endif

#include "libIMclassify.h"
using namespace std;

#ifdef _MSC_VER
extern "C"
#endif

void* ClassifyIM(const char** inIMfn, const int &nIM, void *x)
{
    int *err = (int *)x;
    if (err == NULL) return 0;

    // Call application and library initialization. Perform this
    // initialization before calling any API functions or
    // Compiler-generated libraries.
    if (!mclInitializeApplication(NULL,0))
    {
        std::cerr << "could not initialize the application properly" << std::endl;
        *err = -1;
        return x;
    }
    if( !libIMclassifyInitialize() )
    {
        std::cerr << "could not initialize the library properly"<< std::endl;                  
        *err = -1;
    }
    else
    {     
        try
        {
   // Create the input data
   // Input parameters:
   // B:   test image file name cell array and
   // nIM:  number of test images to be classifile   
   int arr[1] = {nIM};
   mwArray N(1,1,mxINT8_CLASS,mxREAL);
   N.SetData(arr, 1);   
   mwArray B(1, nIM, mxCELL_CLASS);//the image name in cell array format
   for(int i = 1; i<= nIM; i++)
   {
    mwArray fn(inIMfn[i-1]);
    B.Get(1,i).Set(fn);
   }   
   //Create output parameters
   //lbl: the predicted label, an array of size nIM
   mwArray lbl(nIM,1,mxUINT8_CLASS,mxREAL);

   //Call the library function
   ImageCategorization(1, lbl, B, N);

   //convert the predicted label from mwArray in matlab to unsigned int array in C++
   unsigned int *lblp = new unsigned int[nIM];
   lbl.GetData(lblp,nIM);

   //Display the returned predicted label for each image
   for(int i = 0; i < nIM; i++)
    std::cout << "Label of "  << inIMfn[i] << " is: " << lblp[i]<  }
  catch (const mwException& e)
        {
          std::cerr << e.what() << std::endl;
          *err = -2;
        }
        catch (...)
        {
          std::cerr << "Unexpected error thrown" << std::endl;
          *err = -3;
        }   
  // Call the library termination routine
  libIMclassifyTerminate();
    }
/* On MAC, you need to call mclSetExitCode with the appropriate exit status
 * Also, note that you should call mclTerminate application in the end of
 * your application. mclTerminateApplication terminates the entire
 * application and exits with the exit code set using mclSetExitCode. Note
 * that this behavior is only on MAC platform.
 */
#ifdef __APPLE_CC__
    mclSetExitCode(*err);
#endif
    mclTerminateApplication();
    return 0;
}

int main()
{
 int err = 0;
 int nIM = 3;
 const char* imfn[] = {"test001.jpg", "test2.jpg","test0003.jpg"};
 
#ifdef __APPLE_CC__
    pthread_t id;
    pthread_create(&id, NULL, run_main, &err);

    CFRunLoopSourceContext sourceContext;
    sourceContext.version         = 0;
    sourceContext.info            = NULL;
    sourceContext.retain          = NULL;
    sourceContext.release         = NULL;
    sourceContext.copyDescription = NULL;
    sourceContext.equal           = NULL;
    sourceContext.hash            = NULL;
    sourceContext.schedule        = NULL;
    sourceContext.cancel          = NULL;
    sourceContext.perform         = NULL;

    CFRunLoopSourceRef sourceRef = CFRunLoopSourceCreate(NULL, 0, &sourceContext);
    CFRunLoopAddSource(CFRunLoopGetCurrent(), sourceRef, kCFRunLoopCommonModes);
    CFRunLoopRun();
#else 
    ClassifyIM(imfn, nIM, &err);
#endif
    return err;
}

5. Create a win32 console project using Visual Studio C++ 2003, add main.cpp and libIMclassify.h to the project.

6 Set C++ project settings in Visual Studio .NET 2003.

 6.1

Click menu Project->Project settings->Linker->General, add C:\Program Files\MATLAB\R2006b\extern\lib\win32\microsoft;F:\RA\TactileOurData\NET\ImageCategorization\ImageCategorization to the "Additional Library Directories. The second directory is the one that holds your C++ project.

6.2 Click Project->Project settings->Linker->Input, add "libdflapack.lib libeng.lib libfixedpoint.lib libmat.lib libmex.lib libmwlapack.lib libmwservices.lib libmx.lib libut.lib mclcom.lib mclcommain.lib mclmcr.lib mclmcrrt.lib mclxlmain.lib libIMclassify.lib" to the "Additional dependencies" entry. The last libIMclassify.lib is the lib file generated by the matlab compiler and is located in the current project directory (F:\RA\TactileOurData\NET\ImageCategorization\ImageCategorization )

7. Build and run.

Standalone matlab application may also be built by using mcc and mbuild command. For example, in matlab prompt or OS prompt, type

mbuild main.cpp libIMclassify.lib

will create an executable named main.exe

From Jul 10 to Jul 20, Tom and I are busy with capturing some new data. Now we have finished caputuring the video data, and in the following days we are going to do data analysis including:

1 Splitting the video into individual frames and select insecting frames by the four views to serve as the tracking data.

2 Do background substraction to segment the profile of the person out. These will serve as observation likelihood in our tracking algorithm.

3 Manual initialization on the first frame including approximate the limb length and limb parameters, the angle values for the first frame and possibly the camera calibaraition information.

Some real problems/difficulties exist when trying our algorithm on the new data:

1 The limb length and limb scaling parameters are not available.

2 The camera calibration matrix is unavailable and needs to be estimated by ourselves.

3 Numerical training angles are unavailable( some tracking parameters need to be estimated from the training numerical angles).

4Munal initializing the frame could also pose problem if not accurate.

Following are some preliminary thoughts on how to innovate on the MA classification using MI, and how to improve the classification accuracy

1. Currently the dimension of the feature space is 13 including color, texture, shape and edge histogram. Consider to include other features that better distinguish MA from other instances like the color (HSV) differnce between the current block and the up, bottom, leaf and right blocks.

2.Using Kernel PCA to reduce the dimensionality of feature vector, leading to more efficient maximum DD point search.

3. A more efficient and effective multidimension density search stratigy that localizes the maximum DD points.

4. From where to start search?

5.Other MI learning methods or framework.

My tasks from July 15 to Aug 15 is as follows:

1. Do experiments on new dataset(capture new dataset, manual initialization, background substraction).

2. Think about how to write the ICASSP 3D tracking paper.

3. Think about what else can we do to enhance the NV simulation journal paper. And start doing it.

            a) Calculating the fractal dimension is one work;

            b)2 pages evaluation by UTMB.

4. Think about what innovations can we do to improve the current MI MA classification. This will possibly lead to another ICASSP paper.

I came across some real difficulties when trying PLS_RBPF on new data that I think I can not be easily solved:

1 How to determine the limb length and the 10*4 limb parameters? In the Brown data set, these parameters are determined from the motion captured marker data, but in my own data, I do not have the motion capture system, and can not calculate these parameters precisely. The only way is to approxiate by munual initialization, but that will make the tracking inaccurate.

OR can I try a new human body model that does not involve the limb parameters?

2 We do not have the calibration information for the wireless cameras.

3 Some tracking parameters need to be calculated from the motion captured training angles, we do not have the numerical training angles.

4 Manual initialization for the first frame is also a problem.

1 By next Thu, submit a 3 page report to Dr.Li regarding the MA classification using MI learning. Containing the implementation details, the results and some thinkings.

2 By next Friday, submit a 1-2 page report to Dr.Li regarding the 3D tracking, what have been done so far, what are the results in video format and what are your thinkings.Implement the Dynamic Time Warping error comparison method and include the results on the report.

3 Develop a utility using MATLAB to manually do the initialization for the 3D PLS_RBPF_25D tracking.

4For the arrangements of my work in the three directions in the next five months:

        a)3D tracking, I will do most of the work by myself. I have roughly OCt and Nov two full months working on testing the algorithm on the new sequences.

        b) Medical image. Zheshen will be involved in this project but I need to take a lead and have my own new ideas and implementations,and do most of the experiments and communicate in time the results to Dr.Li

        c)Tactile display. Zheshen will take the lead in this project. For me I need to do more reading. Think in the scenario that we will use a tactile printer rather than a touchable display.and Thinking how and what we can do based on the UW's work.What can we proposed to do with a static tactile image printer?and HOw can we achieve that?

5 Some other staff. Dr.Li asked me to help zheshen to pass the TA exam as much as I can so that she will get the financial aid. 

When generating the NV paths using GenerateNEOAngle, I want to simulate the resistance force of the background cell. For example, where there are vessels on the background image, then the underlying pixels have lower probability of being chosen, indicating that the resistance force of this pixel is high.

A optimal way to simulated the background force is to extract the background image beneath of the simulated NV and then assign some probabilities to each pixel based on the presence/absence of normal vessel.(normal vessel will hinder the growth of NV??)

Because the background image can only be obtained after the size of simulated NV is determined, so the aboved method can not be realized. So we turned to randomly assign some probabilies to the lattice sites of simulated NV.

On the meeting today, Dr.Li suggests me to validate the PLS method by visualizing the result.

It can be done in the following steps:

1 Use training data to get the PLS coefficients

2 Predict leaf variables using the PLS coefficients and the test data right hand variables

3 Plot the error between predicted leaf and true leaf

4 Visualize the human walking squence using the true right hand angles and predicted left hand angles.

Tom will do this starting June 7. I have to verify the result after he finishes.

I have done some experiments using the PLS to estimate the relationship between right
hand side angles and left hand side angles.

From all the experiments, I observed that the PLS2_25D_RegAngles (regress on the angles
but do Kalman Prediction as before) performs best.

The next step is to

1)check other regression methods like MLR,PCR. What percent of variance can they capture?

if higher then PLS, then use them for tracking, then compare the result.

2)Inter-comparison:

    PLS_RBPF_Tracking V.S. Annealled PF

    PLS_RBPF-Tracking V.S.RBPF tracking using curve fitting

3)Test PLS_RBPF_Tracking on new sequences such as running sequence which must have training and test data available

4)what happened if the nuber of regression samples is decreased or increased in the PLS regression?

5)HOw to compare with Annealled PF? using what kind of criterion as the error measure?

06/05/2006

Woking on revising the GenerateNEOAngle algorithm, such that

a)  it can generate NV branches in multiple levels and multiple branches at each level.

Purpose to increase the complexity of the NV appearance

b)add a resistance force to simulate the resistance force that comes from the background retina cell.