Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
Consider the two variables in the dataset Assign3.csv. We are interested in predicting the second variable Y given the first variable X.
First draw a random number U uniformly on the interval spanned by the minimum and maximum values of the inputs (x1; :::; xn) and then use it to construct the following function whose purpose is to give the prediction of Y given X = x:
f(x) = a1I(U 6 x) + a2I(U > x);
where a1 and a2. are just unknown constants to be learned. It goes without saying that I(some statement) is the indicator function that equals 1 when the statement is true and 0 otherwise.
a.Usetwo different methods to compute the estimate f^(x) = a^1I(U 6x)+a^2I(U > x). Is f^ a strong learner?
b.Use one of the previous two methods to write an Rfunction that takes as input x and the data (x1; :::; xn; y1; :::; yn) and gives as output f^(x).
Make sure the function is capable of dealing with the case where
c.Usingthree different runs of the previous function, create three dif- ferent plots where, on each, f^ is shown together with the scatter plot of the
4.Write an R function that applies boosting to the previous step function learner.
That R function should take as inputs: the data, B the number of boosting iterations, L the learning rate and an optional argument indicating thesize of the test subsample in case a validation set approach is
As output the function should give: f^boost the boosted learner evaluated
at the training data and the training mean squared error evaluated for each iteration b = 1; :::; B of the boosting algorithm. Also, in case the size of the test subsample is greater than zero, the function should output: f^boost evaluated
at the test sample and the test MSE evaluated for each iteration b = 1; :::; B.
a.Use that function to plot f^boost on top of the data scatter plot for
L = 0.01 and for B = 10000. Show the same with different values of B.
b.Plot the training MSE vs. the number of iterations
c.Was there overfitting when B =10000?
Note: Even though the algorithm is described in detail in both the slides and textbook, for the sake of making the implementation easier, its special case per- taining to the questions in the assignment is presented here.
A sample of covariates (i.e. inputs) x1; :::; xn and responses (i.e. out- puts) y1; :::; yn.
a.Givenx1; :::; xn as covariates and r1; :::; rn as responses, fit a learner f^b by first sampling U and then estimating f^b(x) = a^1I(U 6 x) + a^2I(U > x).
b.Set f^boost(x) f^boost(x) + Lf^b(x).
c.Setri ri ¡ Lf^b(xi).
4.Output: f^boost(x).