California Housing Regression

A supervised-regression project that predicts the median house value of California districts (1990 US Census) from eight numeric features. The assignment's chosen algorithm is linear regression trained with gradient descent (SGDRegressor); a KNN regressor and a decision tree are run alongside as non-linear baselines, and the whole validation is reproduced in WEKA.

The full walk-through is in the build-log; the theory behind the line — cost function, gradients, R² — is in my linear regression series.

The California Housing dataset is 20,640 districts with a continuous target (median house value, ×$100k). Two constraints shaped the work:

• Stay linear, on purpose. The point is to see how far a straight-line model trained by gradient descent gets — and to measure what non-linear structure it leaves on the table.
• Validate honestly. No leakage in preprocessing, and every metric confirmed against an independent WEKA run.

• Feature selection: rank features by mutual information on the training set only, keeping the five that inform the target (MedInc, location, AveRooms, AveOccup) and dropping the rest.
• Scaling: MinMaxScaler to [0, 1] rather than standardization — California's extreme outliers make plain z-scoring diverge the SGD gradient to NaN; bounding the features keeps it stable.
• Modelling: a pipeline (scaler + regressor) on an 80/20 holdout, scored with R², MAE and RMSE. Linear regression via SGD is compared against KNN and a decision tree.

The linear model explains about half the variance (R² ≈ 0.53); the predicted-vs-actual plot exposes both the target's censoring ceiling at $500k and the model's growing error on expensive districts. KNN and the tree do better, quantifying the non-linear, geographic structure a line can't capture. Against WEKA every model scored a little higher — WEKA's LinearRegression solves least squares in closed form rather than by gradient descent — the same kind of method/implementation gap the classification leg surfaced.

This is the regression leg of a broader machine-learning project. The classification write-up (Acoustic Emotion Recognition) is done, and unsupervised learning (clustering / PCA) is in progress — each validated the same way, scikit-learn against WEKA.

Python · scikit-learn · pandas · NumPy · Matplotlib / Seaborn · WEKA · Jupyter.