PS 5841 Ashford University Mse Bias Variance of Statistical Learning Questions

ACTU PS5841 Data Science in Finance and Insurance – Autumn 2019Dr. Yubo Wang
Assignment-1
Assigned 9/5/19, Due 9/17/19 (Tue)
Problem 1. Statistical Learning
Suppose the observed data are generated by
𝑦 = 1 + 2𝑥 + 𝜖,
𝑥 ∈ [−50,50],
𝜖 ∈ 𝑁(𝜇 = 0, 𝜎 2 = 102 )
Use your preferred data analysis tool (a spreadsheet at this stage can be useful to many), demonstrate
numerically that a simple linear regression model 𝑦̂ = 𝛽̂0 + 𝛽̂1 𝑥 is able to learn.
[a] Specifically, use a test set of size 100 and training sets of various sizes (30, 100, 200, 300),
numerically estimate the corresponding expected test MSE and complete the following table.
Training Set size
Expected Test MSE
30
100
200
300
[b] Please also provide a plot of the expected test MSE against the training set size.
Problem 2. Bias-Variance Trade-off
Suppose the observed data are generated by
𝑥
𝑦=
+ 𝜖,
𝑥 ∈ [−25,25],
𝜖 ∈ 𝑁(𝜇 = 0, 𝜎 2 = 0.52 )
2
√1 + 𝑥
Suppose you use polynomial regressions 𝑦̂ = ∑𝑛𝑖=0 𝛽̂𝑖 𝑥 𝑖 , 𝑛 = 1, 2, … ,6 to learn from data and make
predictions.
Use your preferred data analysis tool (a spreadsheet at this stage can be useful to many), numerically
demonstrate the trade-off between bias and variance.
Specifically, use 300 training sets and test them on the test set associated with 𝑥 = −20, −10, 0, 10, 20.
[a] Please complete the following table with your estimates to demonstrate that the variance-bias
decomposition roughly holds for each model.
degree n
Expeted Test MSE
squred bias
variance
variance of error term
LHS – RHS
1
2
3
4
5
6
[b] Please provide a graph based on your estimates demonstrating the bias-variance trade-off.
Please see notes on linear model and on Excel on the next page.
ACTU PS5841 Data Science in Finance and Insurance – Autumn 2019
Dr. Yubo Wang
Assignment-1
Assigned 9/5/19, Due 9/17/19 (Tue)
Notes on linear model
̂ , the coefficients based on least squares estimation are
̂ = 𝛽̂0 + 𝒙𝑇 𝜷
For a linear model 𝒚
̂ = (𝑿𝑇 𝑿)−𝟏 𝑿𝑇 𝒚
𝜷
𝑇
̂ = (𝛽̂0 , 𝜷
̂ 𝑇 ) , 𝑿 = (𝟏, 𝒙1 , … , 𝒙𝑝 ) where 𝒙𝑗 = (𝑥1𝑗 , … , 𝑥𝑛𝑗 )𝑇 , and 𝒚 = (𝑦1 , … , 𝑦𝑛 )𝑇 .
where 𝜷
Notes on Excel
Transposition 𝑨𝑇 = 𝑇𝑅𝐴𝑁𝑆𝑃𝑂𝑆𝐸(𝑨)
Matrix multiplication 𝑨𝑩 = 𝑀𝑀𝑈𝐿𝑇(𝑨, 𝑩)
Inverse matrix 𝑨−1 = 𝑀𝐼𝑁𝑉𝐸𝑅𝑆𝐸(𝑨)
𝑅𝐴𝑁𝐷() returns a number randomly sampled [0,1)
𝑁𝑂𝑅𝑀. 𝐼𝑁𝑉(𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦, 𝑚𝑒𝑎𝑛, 𝑠𝑡𝑑𝑒𝑣) returns the inverse of the normal cumulative distribution for
the specified mean and standard deviation.
Data->What-if analysis->Data Table is a convenient tool for automating repetitive tasks.
Bias vs Variance (2)
High Bias
Low Variance
Low Bias
High Variance
Prediction Error
Test Sample
Training Sample
Low
High
Model Complexity
Bias vs Variance
3
(3)
E