CST383 - Module 7

What did I learn in the seventh week of CST383?

This week covered logistic regression, encoding categorical variables, and overfitting with learning curves. The main theme for this week was preparing data for machine learning, while choosing an appropriate model based on what is getting trained/tested. 

The most surprising aspect was how logistic regression works as a classification algorithm. Basically, logistic regression squishes a linear model's output using the sigmoid function, which makes it so that the probability is produced between 0 and 1. Therefore, does logistic regression simply inherit from linear regression, whether good or bad?

A concept I found confusing was the bias-Variance trade off. I understood from the lecture video and the lecture slide (overfitting1.pptx) that high variance means that the model is too sensitive to training data, which creates a high bias that won't capture a realistic shape of the data. However, how do you know when a gap between training and test error is acceptable?

Another area of confusion was encoding the target variable. The lecture was clear that the target variable should use ordinal encoding, mapping each unique value to a number starting with 0. However, does it matter which value gets 0 and which gets 1?

Some ideas I had were that learning curves is a useful diagnostic tool before hyperparameter tuning, as it illustrates whether one should get more data or use a simpler model. Furthermore, bias-variance tradeoff felt like it could go beyond machine learning, as it can help with not oversimplifying or overcomplicating everyday decision making.