Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
Homework Assignment
Problem 1. (40 points) Consider two classification concepts given in Figure 1, where x ∈ X = [?6, 6] ×
[?4, 4], y ∈ Y = {?1,+1} and p(y|x) ∈ {0, 1} is defined in the drawing.
Figure 1: Two concepts where examples that fall within any of the three 3 × 3 (panel A) or 1 × 1 (panel
B) squares are labeled positive and the remaining examples (outside each of the squares but within X ) are
labeled negative. The position of the point x = (x1, x2) in the upper left-hand corner for each square is
shown in the picture. Consider horizontal axis to be x1 and vertical axis as x2.
Your experiments in this question will rely on generating a data set of size n ∈ {250, 1000, 10000} drawn from
a uniform distribution in X and labeled according to the rules from Figure 1; e.g., P (Y = 1|x) = 0.97 if x
that was randomly drawn is inside any of the three squares in either of the two panels, and P (Y = 1|x) = 0.01
otherwise (notice that this introduces a certain amount of asymmetric noise in the data). The goal of the
following two problems will be to train and evaluate classifiers created from the data generated in this way.
You can use any library you want in this assignment and do programming in Python, MATLAB, or R. Your
code should be easy to run for each question and sub-question below so that we can replicate your results
to the maximum extent possible.
Consider single-output feed-forward neural networks with one or two hidden layers such that the number of
hidden neurons in each layer is h1 ∈ {1, 4, 12} and h2 ∈ {0, 3}, respectively, with h2 = 0 meaning that there
is no second hidden layer. Consider one of the standard objective functions as your optimization criterion
and use early stopping and regularization as needed. Consider a hyperbolic tangent activation function in
each neuron and the output but you are free to experiment with others if you’d like to. For each of the
architectures, defined by a parameter combination (h1, h2), evaluate the performance of each model using
classification accuracy, balanced accuracy, and area under the ROC curve as your two performance criteria.
To evaluate the performance of your models use cross-validation. However, to evaluate the performance of
performance evaluation, generate another very large data set on a fine grid in X . Then use the predictions
1
2 Homework Assignment # 4
from your trained model on all these points to determine the “true” performance. You can threshold your
predictions in the middle of your prediction range (i.e., at 0.5 if you are predicting between 0 and 1) to
determine binary predictions of your models and to then compare those with true class labels you generated
on the fine grid.
Provide meaningful comments about all aspects of this exercise (performance results for different network
architectures, accuracy in and of cross-validation, true accuracy when you disregard class-label noise, run
time, etc.). The comments should not just re-state the results but rather capture trends and give reasoning
as to why certain behavior was observed.