PGEE11136 Applications of Sensor and Imaging Systems Assignment
Applications of Sensor and Imaging Systems Assignment
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
PGEE11136 Applications of Sensor and Imaging Systems Assignment
1. You are given a training set of data points {x(n)}10n=1 where x ∈ R2. Each point has
an associated class label y ∈ {0, 1}. These are depicted in Figure 1.
(a) Use a k-NN classifier with k = 1 to classify a test point at
[
2 −2]⊤ (1)
(b) Use a k-NN classifier with k = 3 to classify a test point at
[
0 2
]⊤
(1)
(c) Use a k-NN classifier with k = 5 to classify a test point at
[−2 0]⊤ (1)
(d) Use a k-NN classifier with k = 7 to classify a test point at
[
4 −4]⊤ (1)
2. You are given a training set of data points {x(n)}3n=1 where x ∈ R5. Each point has
an associated class label y ∈ {1,−1}.
x(1) =
[−1 2 2 −2 0]⊤ y(1) = −1
x(2) =
[
1 1 0 −1 −1]⊤ y(2) = 1
x(3) =
[−2 −4 2 1 0]⊤ y(3) = −1
(a) Use a k-NN classifier with k = 1 to classify a test point at
[
0 1 0 −1 2]⊤ (2)
(b) Give an example of something that could be represented with a 5D data point. (1)
4 3 2 1 0 1 2 3 4
x1
4
3
2
1
0
1
2
3
4
x 2
class 0
class 1
Figure 1: A 2D dataset with binary class labels.
Page 1 of 4 Continued
PGEE11136 Applications of Sensor and Imaging Systems Assignment 3
3. Desmond wants a model that can accurately predict whether a person has a moustache
or not from their picture. He assembles a training set and a test set each consisting of
images of people with and without moustaches. Desmond trains five different machine
learning models using the training set and records the training and test accuracy of
each model. He notices that the model with the highest train accuracy and the model
with the highest test accuracy are different. He isn’t sure what to do next, so comes
to you for advice. What would you tell him? (4)
4. Consider a training set of data points and labels {(x(n), y(n))}4n=1 where x ∈ R and
y ∈ {0, 1}:
x(1) = −1 y(1) = 0
x(2) = −2 y(2) = 0
x(3) = 2 y(3) = 1
x(4) = 1 y(4) = 1
Now consider a linear classifier which consists of a linear model f(x) = wx+ b and a
threshold function that outputs a class prediction yˆ:
yˆ =
{
1 if f(x) ≥ 0.5
0 if f(x) < 0.5
To learn the parameters of the linear model (w,b) we can minimise the mean squared
error loss (MSE) across our training data:
LMSE =
1
4
4∑
n=1
(f(x(n))− y(n))2
(a) Plot the training data. Annotate this plot with the classifier’s decision boundary
for initial parameters wt=0 = 1 and bt=0 = 1. (2)
(b) Determine which side of the decision boundary is allocated to which class, thence
classify a test point at x(t) = −0.6. (1)
(c) Derive an expression for ∂LMSE
∂w
and ∂LMSE
∂b
and explain how these may be used
in gradient descent to update the classifier’s parameters. (2)
(d) Perform a single iteration of gradient descent using your expressions from (c) to
compute wt=1 and bt=1. Use a learning rate α = 0.1. (2)
(e) Determine where the decision boundary is after this update, and reclassify the
test point at x(t) = −0.6. (1)
Page 2 of 4 Continued
PGEE11136 Applications of Sensor and Imaging Systems Assignment 3
5. Sketch each of the functions below, and determine whether they are convex.
(a) f1(x) = |x| (1)
(b) f2(x) = e
x (1)
(c) f3(x) = x
3 (1)
(d) f4(x) =
{
x3 ifx ≥ 0
0 ifx < 0
(1)
(e) f5(x) =
sinx
x
(1)
6. Consider the Booth function, which is convex. It takes in a vector θ =
[
θ1
θ2
]
and
returns a scalar:
B(θ) = (θ1 + 2θ2 − 7)2 + (2θ1 + θ2 − 5)2
(a) Derive an expression for ∇θB(θ) (2)
(b) Use your answer to (a) to solve minimise
θ
B(θ) analytically (2)
7. If we have a training set of data point-label pairs {(x(n), y(n))}Nn=1 (x ∈ RD, y ∈ {0, 1})
and a linear model f(x) =
[
w⊤ b
] [x
1
]
that outputs the log-odds of some point x
belonging to class 1 then the log loss Llog is
Llog = − 1
N
∑
n
[
y(n) log σ(f(x(n))) + (1− y(n)) log(1− σ(f(x(n))))
]
where σ is the sigmoid function. Let θ =
[
w
b
]
.
(a) Show that dσ(a)
da
= σ(a)(1− σ(a)) for some variable a ∈ R (2)
(b) Show that ∂(w
⊤x)
∂w
= x (1)
(c) Use your answers to (a) and (b) to show that
∇θLlog = − 1
N
∑
n
(
y(n) − f(x(n)))[x(n)
1
]
(4)
Page 3 of 4 Continued
PGEE11136 Applications of Sensor and Imaging Systems Assignment 3
8. Consider a validation set of data points and labels {(x(n), y(n))}4n=1 where x ∈ R2 and
y ∈ {c, d}:
x(1) =
[−1 −1]⊤ y(1) = c
x(2) =
[
+1 −1]⊤ y(2) = c
x(3) =
[−1 +1]⊤ y(3) = d
x(4) =
[
+1 +1
]⊤
y(4) = d
and a classifier consisting of a linear model f(x) = −x1 + x2 + 1 and a threshold
function
yˆ =
{
d if f(x) ≥ τ
c if f(x) < τ
(a) Produce a plot of the validation data in the style of Figure 1 (1)
(b) Draw the decision boundary of the classifier when τ = 0 and calculate its vali-
dation accuracy (2)
(c) Draw the decision boundary of the classifier when τ = 1 and calculate its vali-
dation accuracy (1)
(d) Is there a value of τ that will give us perfect accuracy on the validation set for
this linear model? (1)