Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
Errata: All corrections are in red.
Note: For the purposes of this assignment, if X is a random variable we let pX denote the probability
density function (pdf) of X, FX to denote it’s cumulative distribution function, and P to denote
probabilities. These can all be related as follows:
P (X ≤ x) = FX(x) =∫ x∞pX(z)dz
P (a ≤ X ≤ b) = FX(b)? FX(a) =∫ bapX(z)dz
Often, we will simply write pX as p, where it’s clear what random variable the distribution refers to.
You should show your derivations, but you may use a computer algebra system (CAS) to assist
with integration or differentiation. We are not assessing your ability to integrate/differentiate here.1.
Question 1 Continuous Bayesian Inference 5+5+2+4+4+6+6+5=37 credits
Let X be a random variable representing the outcome of a biased coin with possible outcomes X =
{0, 1}, x ∈ X . The bias of the coin is itself controlled by a random variable Θ, with outcomes2 θ ∈ θ,
where
θ = {θ ∈ R : 0 ≤ θ ≤ 1}
The two random variables are related by the following conditional probability distribution function of
X given Θ.
p(X = 1 | Θ = θ) = θ
p(X = 0 | Θ = θ) = 1? θ
We can use p(X = 1 | θ) as a shorthand for p(X = 1 | Θ = θ).
We wish to learn what θ is, based on experiments by flipping the coin.
We flip the coin a number of times.3 After each coin flip, we update the probability distribution for θ
to reflect our new belief of the distribution on θ, based on evidence.
Suppose we flip the coin n times, and obtain the sequence of coin flips 4 x1:n.
a) Compute the new PDF for θ after having observed n consecutive ones (that is, x1:n is a sequence
where ?i.xi = 1), for an arbitrary prior pdf p(θ). Simplify your answer as much as possible.
b) Compute the new PDF for θ after having observed n consecutive zeros, (that is, x1:n is a sequence
where ?i.xi = 0) for an arbitrary prior pdf p(θ). Simplify your answer as much as possible.
c) Compute p(θ|x1:n = 1n) for the uniform prior p(θ) = 1.
d) Compute the expected value μn of θ after observing n consecutive ones, with a uniform prior
p(θ) = 1. Provide intuition explaining the behaviour of μn as n→∞.
1For example, asserting that
∫ 10x2(x3 + 2x)dx = 2/3 with no working out is adequate, as you could just plug the
integral into Wolfram Alpha using the command Integrate[x^2(x^3 + 2x),{x,0,1}]
2For example, a value of θ = 1 represents a coin with 1 on both sides. A value of θ = 0 represnts a coin with 0 on
both sides, and θ = 1/2 represents a fair, unbaised coin.
3The coin flips are independent and identically distributed (i.i.d).
4We write x1:n as shorthand for the sequence x1x2 . . . xn.
1
e) Compute the variance σ2n of the distribution of θ after observing n consecutive ones, with a uniform
prior p(θ) = 1. Provide intuition explaining the behaviour of σ2n as n→∞.
f) Compute the maximum a posteriori estimation θMAPn of the distribution on θ after observing
n consecutive ones, with a uniform prior p(θ) = 1. Provide intuition explaining how θMAPn varies
with n.
g) Given we have observed n consecutive coin flips of ones in a row, what do you think would be a
better choice for the best guess of the true value of θ? μn or θMAP ? Justify your answer. (Assume
p(θ) = 1.)
h) Plot the probability distributions p(θ|x1:n = 1) over the interval 0 ≤ θ ≤ 1 for n ∈ {0, 1, 2, 3, 4} to
compare them. Assume p(θ) = 1.
Question 2 Bayesian Inference on Imperfect Information (4+5+8+4+4=25 credits)
We have a Bayesian agent running on a computer, trying to learn information about what the pa-
rameter θ could be in the coin flip problem, based on observations through a noisy camera. The noisy
camera takes a photo of each coin flip and reports back if the result was a 0 or a 1. Unfortunately, the
camera is not perfect, and sometimes reports the wrong value.5 The probability that the camera makes
mistakes is controlled by two parameters α and β, that control the likelihood of correctly reporting a
zero, and a one, respectively. Letting X denote the true outcome of the coin, and X denoting what
the camera reported back, we can draw the relationship between X and X? as shown.
We would now like to investigate what posterior distributions are obtained, as a function of the
parameters α and β.