Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
DATA DRIVEN COMPUTING Answer THREE questions. All questions carry equal weight. Figures in square brackets indicate the percentage of avail- able marks allocated to each part of a question. COM 2004 1
1. This question concerns probability theory. a) The discrete random variable X represents the outcome of a biased coin toss. X has the probability distribution given in the table below, x H T P(X = x) θ 1−θ where H represents a head and T represents a tail. (i) Write an expression in terms of θ for the probability of observing the sequence H, T, H, H. [5%] ANSWER: θ× (1−θ)×θ×θ = θ3(1−θ) (ii) A sequence of coin tosses is observed that happens to contain NH heads and NT tails. Write an expression in terms of θ for the probability of observing this specific sequence. [5%] ANSWER: θNH (1−θ)NT (iii) Show that having observed a sequence of coin tosses containing NH heads and NT tails, the maximum likelihood estimate of the parameter θ is given by NH NH +NT [20%] ANSWER: Write down the expression for p(x) in terms of the parameter θ, P(x;θ) = θNH (1−θ)NT Find maximum by differentiating w.r.t. the parameter θ and setting to 0, dP(x;θ) dθ = NHθ NH−1(1−θ)NT −θNH NT (1−θ) NT−1 = 0 Solve for θ, NHθ NH−1(1−θ)NT −θNH NT (1−θ) NT−1 = 0 NH(1−θ)−θNT = 0 NH = NHθ+NT θ θ = NH NH +NT
b) The discrete random variables X1 and X2 represent the outcome of a pair of independent but biased coin tosses. Their joint distribution P(X1,X2) is given by the probabilities in the table below, X1 = H X1 = T X2 = H λ 3λ X2 = T 2λ ρ (i) Write down the probability P(X1 = H,X2 = H). [5%] ANSWER: λ (ii) Calculate the probability P(X1 = H) in terms of λ. [5%] ANSWER: λ+2λ = 3λ (iii) Calculate the probability P(X2 = H) in terms of λ. [5%] ANSWER: λ+3λ = 4λ (iv) Given that the coin tosses are independent and that λ is greater than 0, use your previous answers to calculate the value of λ. [15%] ANSWER: Independence implies that P(X1 = H,X2 = H) = P(X1 = H)×P(X2 = H) So λ = 3λ×4λ λ = 1 12 COM 2004 3 UNIVERSITY OF SHEFFIELD COM 2004 (v) Calculate the value of ρ. [5%] ANSWER: Also entries in the table must sum to one so λ+3λ+2λ+ρ = 1 Therefore ρ = 1 2
c) Consider the distribution sketched int the figure below. 0 1 2λ λ b p(x) x p(x) = 2λ if 0 <= x < b λ if b <= x <= 1 0 otherwise (i) Write an expression for λ in terms of the parameter b. [15%] ANSWER: 2λb+(1−b)λ = 1 λ = 1 1+b (ii) Two independent samples, x1 and x2, are observed. x1 has the value 0.25 and x2 has the value 0.75. Sketch p(x1,x2;b) as a function of b as b varies between 0 and 1. Using your sketch, calculate the maximum likelihood estimate of the parameter b given the observed samples. [20%] ANSWER: • For b in the range 0 to 0.25, p(x1,x2) = λ×λ = 1 (1+b)2 , i.e., 1 to 16/25 • For b in the range 0.25 to 0.75, p(x1,x2) = λ× 2λ = 2 (1+b)2 , i.e., 32/25 to 32/49 • For b in the range 0.75 to 1.0, p(x1,x2) = 2λ×2λ = 4 (1+b)2 , 64/49 to 1 The p(x1,x2) has a maximum value of 64/49 when b = 0.75.
2. This question concerns the multivariate normal distribution. a) Consider the data in the following table showing the height (x1) and arm span (x2) of a sample of 8 adults. x1 151.1 152.4 152.9 156.8 161.8 158.6 157.4 158.8 x2 154.5 162.2 151.5 158.2 165.3 165.6 159.8 162.0 The joint distribution of the two variables is to be modeled using a multivariate Gaussian with mean vector, µ and covariance matrix, Σ. (i) Calculate an appropriate value for the mean vector, µ. [5%] ANSWER: (156.2,159.9) (ii) Write down the formula for sample variance. Use it to calculate the unbiased variance estimate for both height and arm span. [10%] ANSWER: Var(x) = 1 N−1 N ∑ i=1 (xi−µx) 2 The variance of height and arm space are 13.9 and 24.9, respectively. (iii) Write down the formula for sample covariance. Use it to calculate the unbiased estimate of the covariance between height and arm span. [10%] ANSWER: Cov(x,y) = 1 N−1 N ∑ i=1 (xi−µx)(yi−µy) The covariance between height and arm span is 13.5. (iv) Write down the covariance matrix, Σ. [5%] ANSWER: ( 13.9 13.5 13.5 24.9 )
(v) Compute the inverse covariance matrix, Σ−1. [15%] ANSWER: ( 0.154 −0.0840 −0.0840 0.0860 ) b) Remember that the pdf of a multivariate Gaussian is given by p(x) =Ce− 1 2 (x−µ) T Σ−1(x−µ) where C is a scaling constant that does not depend on x. Using the answer to 2 (a) and the equation above, answer the following questions. (i) Who should be considered more unusual: • Ginny who is 162.1 cm tall and has arms 164.2 cm long, or • Cho who is 156.0 cm tall and has arms 153.1 cm long? Show your reasoning. [20%] ANSWER: Compute −(x− µ)T Σ−1(x− µ) for each person and see which is smaller. For Ginny it is -2.67 and for Cho it is -3.71. Therefore Cho’s proportions are more unusual. (ii) A large sample of women is taken and it is found that 120 have measurements similar to those of Ginny. How many women in the same sample would be ex- pected to have measurements similar to those of Cho? [15%] ANSWER: The samples should be in the ratio exp(−(x1−µ) T Σ−1(x1−µ)) : exp(−(x2−µ) T Σ−1(x2−µ)) Plugging in the numbers from the previous section 120∗ exp(−3.71)/exp(−2.67) = 42 COM 2004 7
c) A person’s ‘ape index’ is defined as their arm span minus their height. (i) Use the data in 2 (a) to estimate a mean and variance for ape index. [10%] ANSWER: µ = 3.66 σ2 = 11.6 σ = 3.41 (ii) The figure below shows a standard normal distribution, i.e., X ∼ N(0,1). The percentages indicate the proportion of the total area under the curve for each segment. −3 −2 −1 0 1 2 3 19.1%19.1% 15.0%15.0% 9.2%9.2% 4.4%4.4% 1.7%1.7% 0.5%0.5% Using the diagram estimate the proportion of the population who will have an ape index greater than 10.5? [5%] ANSWER: 10.5 is 2 σ greater than the mean. Therefore about 2.2% of the population will be expected to have an ape index of 10.5 or greater. (iii) Using the figure above estimate the mean-centred range of ape indexes that would include 99% of the population. [5%] ANSWER: This would be about + or - 2.5 sigma about the mean. So • smallest: 3.66−3.41×2.5 =−4.87 • biggest: 3.66+3.41×2.5 = 12.19
i.e., the range [−4.87,12.19].
3. This question concerns classifiers. a) Consider a Bayesian classification system based on a pair of univariate normal distribu- tions. The distributions have equal variance and equal priors. The mean of class 1 is less than the mean of class 2. For each case below say whether the decision threshold increases, decreases, remains unchanged or can move in either direction. (i) The mean of class 2 is increased. [5%]