Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
COMP10001 Project 2 Game Description
Objectives
This assignment consists of three parts, which cover latent variables models and neural networks (Modules 4 and 5). The total marks of this assessment are 100 and will contribute 16% to your final s core. In your Jupyter notebook, you can use any import from the libraries numpy, scipy, matplotlib, and scikit-learn to solve the tasks.
1 Document Clustering
In this part, you solve a document clustering problem using unsupervised learning algorithms (i.e., soft and hard Expectation Maximization) for document clustering.
Question 1 [EM for Document Clustering, 40 Marks]
I Derive Expectation and Maximization steps of the hard-EM algorithm for Document Clustering, show your work in your submitted PDF report. In particular, include all model parameters that should be learnt and the exact expression (using the same math convention that we saw in the Module 4) that should be used to update these parameters during the learning process (ie., E step, M step and assignments).
II Load necessary packages.
III Load Task2A.txt file (if needed, perform text preprocessing similar to what we did in Activity 4.2).
IV Implement the hard-EM (you derived above) and soft-EM (derived in Chapter 5 of Module 4). Please provide enough comments in your submitted code. Hint: If it helps, feel free to base your code on the provided code for EM algorithm for GMM in Activity 4.1).
V Set the number of clusters K=4, and run both the soft-EM and hard-EM algorithms on the provided data.
VI Perform a PCA on the clusterings that you get based on the hard-EM and soft-EM in the same way we did in Activity 4.2. Then, visualize the obtained clusters with different colors where x and y axes are the first two principal components (similar to Activity 4.2). Attach the plots to your PDF report and report how and why the hard and soft-EM are different,based on your plots in the report.
2 Perceptron vs. Neural Network [30 Marks]
In this part, you will be working on a binary classification task on a given synthetic dataset. You will use machine learning tools including a Perceptron and a 3-layer Neural Network to solve the task. Here, we are looking for your meaningful observations and discussions towards the differences between Perceptron and Neural Networks.
Question 2 [Neural Network’s Decision Boundary, 30 Marks]
I Load Task2B_train.csv and Task2B_test.csv datasets, plot the training and testing data separately in two plots. Mark the data with different labels in different colors. Attach the plot to your PDF report submission.
II Train two Perceptron models on the loaded training data by setting the learning rates η to 0.1 and 1.0 respectively. Calculate the test errors of two models and find the best η and its corresponding model, then plot the decision boundary and the test data in one plot; attach the plot to your PDF report. Hint: We expect the decision boundary of your perceptron to be a linear function that separates the testing data into two parts. You may also choose to change the labels from [0, 1] to [−1, +1] for your convenience.
III For each combination of K (i.e, number of units in the hidden layer) in 5, 10,15, …, 40, (i.e. from 5 to 40 with a step size of 5), and η (i.e., learning rate) in 0.01, 0.001 run the 3-layer Neural Network and record testing error for each of them. Plot the effect of different K values on the accuracy of the testing data and attach it to your PDF report. Based on this plot, find the best combination of K and η and obtain your best model,then plot the decision boundary and the test data in one plot; attach the plot to your PDF report.
IV In your PDF report, explain the reason(s) responsible for such difference between Perceptron and a 3-layer Neural Network by comparing the plots you generated in Steps II and III. Hint: Look at the plots and think about the model assumptions.