Assignment 3:This assignment is done by MATLAB. Compress all code fifiles into a .zip fifile and submit through Blackboard.
This assignment is done by MATLAB. Compress all code fifiles into a .zip fifile and submit through Blackboard.
1.
The handwritten digit dataset mnist5k.mat is modifified from the original MNIST gray-scale image dataset, where samples of digit 9 belong to class 1 and otherwise class −1. It contains a training set Xtr with labels Ytr and a testing set Xte with labels Yte. There are 5000 sample images in both training and testing sets and each sample is stored a vector of 784 gray-scale pixel values between 0 and 255. We do binary classifification using logistic regression:
(a) Visualize the fifirst 9 training images in Xtr using MATLAB built-in function imshow. Dispaly the 9 images as a 3 × 3 tabular in the same fifigure using subplot. Note that you need to reshape each sample into a 28 × 28 matrix before visualization.
(b) Write a function named as logit.m to implement gradient decent algorithm for the logistic regression problem (1) using a constant step size η. The input arguments of logit include Xtr, Ytr, Xte, Yte, the constant step size η, and the regularization parameter λ. The output of logit should be the training accuracy, test accuracy, and the objective value at each iteration (stored as a vector). You should adopt a proper stopping criterion for gradient decent implementation. Call your function logit.m by choosing proper values of η and λ. Note that your main fifile is supposed to be separated from logit.m.
2.
In this problem, we use PCA to reduce the dimension of raw face images. Load the data face.mat, and we will have the variable X which is the data matrix of size 400 × 10304, where each row vector represents a gray-scale image originally of 112 × 92 pixels.
First of all, center the data points (i.e., row vectors) in X by subtracting their mean mu from each row. Denote the preprocessed data matrix by variable X_0. Apply PCA to X_0 and reduce data’s dimension to k = 350. You can use the following command (taking the i-th image as an example) to recover the image:
Recon = X_0(i,:)∗V_k∗V_k‘ + mu,
where V_k= V(:,1:k) contains the fifirst k principal components. Recall the set of principal components can be computed via built-in function svd for SVD. Remember to reshape the vector Recon into a 112 × 92 matrix to show the image. Pick any image from X, and show the effffect of PCA by comparing the original image and the recovered images side by side using subplot. Give a title to each subfifigure.