House Prices: Advanced Regression Techniques

Ask a home buyer to describe their dream house, and they probably won't begin with the height of the basement ceiling or the proximity to an east-west railroad. But this playground competition's dataset proves that much more influences price negotiations than the number of bedrooms or a white-picket fence.

With 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa, this competition challenges you to predict the final price of each home, based on the values of the explanatory variables.

Practice Skills

• Creative feature engineering 
• Advanced regression techniques like random forest and gradient boosting

The Dataset is available here.

Python, Machine Learning, Advanced regression techniques, Lasso, Decision Tree, Random Forest, Gradient Boosting, XGBoost, LightGBM, Keras, Tensorflow, Neural Networks, Ensemble, Vertical Stacking

In this project, we will participate in a Kaggle competition, based on Advanced Regression Techniques. Using a training dataset of 1460 houses, each with 79 explanatory features, we will try to predict the house prices for 1459 houses in the test dataset. We achieved the greatest results with vertical stacking of XGBoost, LightGBM, Elastic Net and Lasso.

As part of the project, we did:

1. A comprehensive exploration of the data
2. Univariate and pairwise comparisons
3. Data cleaning (treating missing values, dealing with leverage points and wrong data types)
4. Transformations (Quadratic - Cubic - Square Root Transformations, Box-Cox Transformation to treat skewed features, One-hot encoding to create dummy variables for different categorical levels)
5. Model formulation (Linear Regression, Lasso, Elastic net, Decision Tree, Random Forest, Adaboost, XGBoost, LightGBM, Neural Networks with Keras)
6. Stacking (Ensemble, Vertical Stacking)

A comprehensive Jupyter Notebook is available here