Lead Data Scientist Interview Question:

Tell us how would you validate a model you created to generate a predictive model of a quantitative outcome variable using multiple regression?

Submitted by: Muhammad

Proposed methods for model validation:

☛ If the values predicted by the model are far outside of the response variable range, this would immediately indicate poor estimation or model inaccuracy.
☛ If the values seem to be reasonable, examine the parameters; any of the following would indicate poor estimation or multi-collinearity: opposite signs of expectations, unusually large or small values, or observed inconsistency when the model is fed new data.
☛ Use the model for prediction by feeding it new data, and use the coefficient of determination (R squared) as a model validity measure.
☛ Use data splitting to form a separate dataset for estimating model parameters, and another for validating predictions.
☛ Use jackknife resampling if the dataset contains a small number of instances, and measure validity with R squared and mean squared error (MSE).
Submitted by: Muhammad