Latex Matrix Example

 \[  
 \begin{bmatrix}  
  X^1_1 & X^1_2 & \cdots & X^1_d \\  
  X^2_1 & X^2_2 & \cdots & X^2_d \\  
   \vdots & & & \\  
   X^N_1 & X^N_2 & \cdots & X^N_d    
 \end{bmatrix}  
 \]  

You can reference to the latex wiki page.

Latex: a minimum working set

 \documentclass[12pt]{article}  
 \usepackage[utf8]{inputenc}  
 \usepackage{amsmath}  
 \title{Template}  
 \date{}  
 \begin{document}  
  \maketitle   
 \end{document}  

Install GoAgent in 5 Minutes (Chrome Version)

You are in China, and you feel like you need to access beyond Chinese websites. I know how you feel, since I have been through this before. The good news is that you can now use goagent to bypass the Great Fire Wall. This tutorial shows how it can be done in 5 minutes.

Step 1. Create an application at Google App Engine. Remember the app id.

If you have already created one for the purpose of goagent before you don't need to create it again. You can use the same app to host goagent.

Step 2. Download GoAgent files. I have put a copy at my Google Drive so that you can access it here. In case you are not able to access Google Drive, I have put a copy at Weiyun which could be downloaded here.

Step 3. Unzip the files, run Server\Uploader.bat. Write down your Google App id as well as your email address and your password. After that you should be able to see Deployment Successful.

Step 4. Edit Local\Proxy.ini. Write down your appid and leave everything else as it is.

Step 5. Open Chrome and install the extension Proxy SwitchSharp here. The setting page should open automatically. If not manually open the setting page.

Step 6. In the export/import tab, you should see "Recover from local files" under "SwitchSharp options". Upload the file Local\SwitchOptions.bak. Click Yes when confirmation window popped up.

Step 7. In the "Switching Rules" tab, click "Update list immediately". However this is likely to fail as it is probably blocked by GFW, so here you should bypass it by turning on SwithSharp first. Click SwitchSharp icon on Chrome and choose "goagent".

Configuration is done. Go ahead and enjoy freedom.

Latex References for Equations and Hyperlinks

Equations:

 \usepackage{amsmath}  
 \begin{eqnarray*}  
 \min\frac{1}{2}w^Tw+C\sum\limits_{i=1}^l\epsilon_i \\  
 \mbox{subject to }y_i(w^T\phi(x_i)+b)\geq 1-\epsilon_i,\\  
 \epsilon_i\geq 0,i=1,...,l,  
 \end{eqnarray*}  

Hyperlinks:

 \usepackage{hyperref}  
 \url{https://www.statsoft.com/textbook/support-vector-machines}  

A Misconception about Soft-threasholding

Watch out for the "0.5".

Three Easy Steps to Install Scikit-Learn

1. Install WinPython: http://sourceforge.net/projects/winpython/?source=dlp

This will install python distribution package together with numpy, scipy, etc.

2. Register Python using "WinPython Control Panel.exe" under WinPython folder.

3. Download and run scikit-learn installation executable. Make sure their versions are matched.

--Simple Test

>>> from sklearn import linear_model
>>> clf=linear_model.LinearRegression()
>>> clf.fit([[0,0],[1,1],[2,2]],[0,1,2])
LinearRegression(copy_X=True, fit_intercept=True, normalize=False)
>>> clf.coef_
array([ 0.5,  0.5])
>>>

Ridge Regression Model with Generalized Cross Validation

An introduction to GCV can be found here:

http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/ebooks/html/csa/node123.html

In ridge regression model with generalized cross validation, we can determine the ridge parameter by selecting \$\lambda\$ that minimizes the following:


where matrix \$H(\lambda)\$ is given as
\$X(X^{T}X+\lambda I)X^{T}\$.

The code:

 void CRidgeRegressionGCV::ComputeRegression()  
 {  
      double minGCV = FLT_MAX;  
      for (int t = 0; t < m_alphas.size(); t++)  
      {  
           double alpha = m_alphas[t];  
           int m = m_X.size();  
           int n = m_X[0].size();  
           MatrixXd mat(m, n + 1);  
           VectorXd rhs(m);  
           for (int i = 0; i < m; i++)  
           {  
                for (int j = 0; j < n + 1; j++)  
                {  
                     if (j < n)  
                     {  
                          mat(i, j) = m_X[i][j];  
                          rhs(i) = m_Y[i];  
                     }  
                     else  
                     {  
                          mat(i, j) = 1;  
                     }  
                }  
           }  
           MatrixXd tmp = mat.transpose()*mat + alpha*MatrixXd::Identity(n + 1, n + 1);  
           JacobiSVD<MatrixXd> svd(tmp, ComputeThinU | ComputeThinV);  
           MatrixXd res = svd.solve(mat.transpose()*rhs);  
           vector<double> coef(n);  
           double intercept;  
           coef.resize(n);  
           for (int i = 0; i < n; i++)  
           {  
                coef[i] = res(i, 0);  
           }  
           intercept = res(n, 0);  
           double GCV = 0;  
           for (int i = 0; i < m; i++)  
           {  
                double sum = 0;  
                for (int j = 0; j < n; j++)  
                {  
                     sum += m_X[i][j] * coef[j];  
                }  
                sum += intercept;  
                double diff = m_Y[i] - sum;  
                double diffsquare = diff*diff;  
                GCV += diffsquare;  
           }  
           GCV /= m;  
           MatrixXd tmp2=svd.solve(mat.transpose());  
           MatrixXd H = mat*tmp2;  
           GCV /= (1-H.trace()/m)*(1-H.trace()/m);  
           if (GCV < minGCV)  
           {  
                minGCV = GCV;  
                m_coef = coef;  
                m_intercept = intercept;  
                m_alpha = alpha;  
           }  
      }  
 }