CST383 - Module 6

What did I learn in the sixth week of CST383?

This week covered topics going over KNN regression, assessing regressors, linear regression, and hyperparameter tuning. The main theme behind them all was to understand how to not only build a model but how to measure whether it works or how it can be improved.

The most surprising aspect this week was how linear regression finds the best fitting line. Basically, the idea that training in context to the data is essentially finding the coefficients that produce the lowest possible MSE was both interesting and difficult, as it requires understanding why squaring the errors rather than taking their absolute values leads to a solution that the model arrives at.

A concept I found confusing was the computing of both MSE and RMSE. I understood from the lecture material that RMSE is more interpretable in comparison to MSE as it shares the same units as the target variable, but is there a specific situation where MSE is preferred in a real world data science setting?

Another aspect that I found confusing was linear regression over KNN in practice. Linear regression within the readings is depicted to be easy and fast, while KNN is also like that on smaller data sets but it starts to struggle in larger data sets compared to linear regression. Therefore, when would KNN be needed on a larger data set given its slowness?  

Some ideas I had when going over the material were that the baseline prediction can be a really useful reality check, as without it a model's RMSE number wouldn't mean much. Furthermore, a random search matching a grid search in results was interesting, as it suggested that every single hyperparameter combination through grid search may not always be necessary, especially if the data set gets too large.