50.1K

Verified Solution

Link Copied!

We want to confirm the theory behind the bias-variance decomposition with an empirical experiment that measures the bias and variance for polynomial models. In our experiment, we will reuse python codes in HW1 and repeatedly fit our hypothesis model to a random training set. We then find the expectation and variance of the fitted models generated from these training sets. Follow the instruction below. 1. Repeat the following steps by changing the polynomial degree d from 0 to 9. 2. i. Make a training data Xtrain, which is evenly spaced 21 numbers over [-7, 7], and Ytrain = f(Xtrain ) + e where e ~ N(0,0.52) is i.i.d. samples from Gaussian distribution. ii. Fit d-th order polynomial to the training data and estimate the optimal model parameters/coefficients w*. ii. Repeat [steps 2i-2ii] 100 times to obtain 100 different model parameters {wi}i=1:100 and model predictions {train,i }i=1:100. iv. Compute the sample mean u and standard deviation o of {train, }i=1:100. Note that pl, o is a vector of length 21. Use np.mean & np.std. v. Compute bias = (u ytrue) and variance = o. Note that bias and variance is a vector of length 21. vi. Make a subplot of the true model and 100 model predictions as follows: degree - 0 degree - 1 degree - 2 degree = 3 degree - 4 15 14 15 15 15 1.0 10 10 10 10 05 05 05 05 05 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -05 -10 -1.0 -10 -10 -101 -15 -15 -1.5 -15 -15 -2 2 2 2 degree - 5 degree - 6 15 degree - 8 degree degree - 7 15 15 1.5 15 1.0 10 10 10 1.0 05 05 05 05 05 0.0 0.0 0.0 0.0 0.0 -0.5 S -0.5 -0.5 -05 -0.5 -1.0 -10 -10 -10 -10 -1.5 -1.5 -15 -1.5 0 vii. Make a subplot of the true model and u, o as follows. Use fill_between to fill the area between u to. degree - 1 degree 2 degree = 3 degree - 4 1.5 1.5 1.5 1.5 1.5 degree - A) Exhix) | Sh =]] 10 1.0 1.0 1.0 1.0 0.5 0.5 0.5 0.5 0.5 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -1.5 SS SS S -1.5 -2 2 -2 - 2 -2 -2 0 degree - 6 degree = 5 degree - 7 o degree - 9 degree = 8 1.5 1.5 1.5 1.5 1.5 10 1.0 1.0 10 1.0 0.5 0.5 0.5 0.5 0.5 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -1.5 -1.5 -2 -2 0 2 0 2 -2 0 -2 0 3. Make a bar plot (use bar ) that takes an average of biasand variance over all values of Xtrain, together with the sum of two. 0.5 0.5 E(biasa) E(variance) 0.10 E(bias) E(variance) 0.4 0.4 0.08 0.31 0.3 0.06 0.2 0.2 0.04 0.14 0.02 0.1 0.0 0.00 0.0 0 2 6 8 02 6 8 02 6 8 4 degree 4 degree 4 degree We want to confirm the theory behind the bias-variance decomposition with an empirical experiment that measures the bias and variance for polynomial models. In our experiment, we will reuse python codes in HW1 and repeatedly fit our hypothesis model to a random training set. We then find the expectation and variance of the fitted models generated from these training sets. Follow the instruction below. 1. Repeat the following steps by changing the polynomial degree d from 0 to 9. 2. i. Make a training data Xtrain, which is evenly spaced 21 numbers over [-7, 7], and Ytrain = f(Xtrain ) + e where e ~ N(0,0.52) is i.i.d. samples from Gaussian distribution. ii. Fit d-th order polynomial to the training data and estimate the optimal model parameters/coefficients w*. ii. Repeat [steps 2i-2ii] 100 times to obtain 100 different model parameters {wi}i=1:100 and model predictions {train,i }i=1:100. iv. Compute the sample mean u and standard deviation o of {train, }i=1:100. Note that pl, o is a vector of length 21. Use np.mean & np.std. v. Compute bias = (u ytrue) and variance = o. Note that bias and variance is a vector of length 21. vi. Make a subplot of the true model and 100 model predictions as follows: degree - 0 degree - 1 degree - 2 degree = 3 degree - 4 15 14 15 15 15 1.0 10 10 10 10 05 05 05 05 05 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -05 -10 -1.0 -10 -10 -101 -15 -15 -1.5 -15 -15 -2 2 2 2 degree - 5 degree - 6 15 degree - 8 degree degree - 7 15 15 1.5 15 1.0 10 10 10 1.0 05 05 05 05 05 0.0 0.0 0.0 0.0 0.0 -0.5 S -0.5 -0.5 -05 -0.5 -1.0 -10 -10 -10 -10 -1.5 -1.5 -15 -1.5 0 vii. Make a subplot of the true model and u, o as follows. Use fill_between to fill the area between u to. degree - 1 degree 2 degree = 3 degree - 4 1.5 1.5 1.5 1.5 1.5 degree - A) Exhix) | Sh =]] 10 1.0 1.0 1.0 1.0 0.5 0.5 0.5 0.5 0.5 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -1.5 SS SS S -1.5 -2 2 -2 - 2 -2 -2 0 degree - 6 degree = 5 degree - 7 o degree - 9 degree = 8 1.5 1.5 1.5 1.5 1.5 10 1.0 1.0 10 1.0 0.5 0.5 0.5 0.5 0.5 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -1.5 -1.5 -2 -2 0 2 0 2 -2 0 -2 0 3. Make a bar plot (use bar ) that takes an average of biasand variance over all values of Xtrain, together with the sum of two. 0.5 0.5 E(biasa) E(variance) 0.10 E(bias) E(variance) 0.4 0.4 0.08 0.31 0.3 0.06 0.2 0.2 0.04 0.14 0.02 0.1 0.0 0.00 0.0 0 2 6 8 02 6 8 02 6 8 4 degree 4 degree 4 degree

Answer & Explanation Solved by verified expert

See Answer