My Kernel Series

For Kaggle House Pricing Competition solving with Advanced Linear Regression Under Construction…

1. Minimal Kernel LB: 0.60109
   • NaN => Median
   • LinearRegression
2. Minimal + Normalized X Kernel LB: 0.30013
   • LinearRegression(Normalized X)
3. Minimal + Normalized X,y Kernel LB: 0.14305
   • y = log2(y)
4. Minimal + Normalized X skew,y Kernel LB: 0.14104
   • X = log2(X) if abs(skew) > 1.7 & no Inf issues
5. Minimal + Normalized X skew,y + filter low Var Kernel LB: 0.13764
   • filter X if Variance < 0.2 and not correlated with target y

ElasticNet enters…

1. Normalized X,y + dummy Kernel LB: 0.13817
   • dummy categorical features
   • ElasticNetCV
2. Beginner ElasticNet Kernel Ver.1 LB: 0.13343
   • all previous things plus
   • ElasticNetCV alpha optimization
3. Beginner ElasticNet + Univar_models Kernel Ver.2 LB: 0.13101
   • ElasticNetCV L1_ratio = 1
   • ElasticNetCV alpha optimization
   • Bagged with 1/3 Simple Linear Regression using selected features by Univariate model performance TestMSE < 0.5
4. Beginner ElasticNet + Univar_models Kernel Ver.3 LB: 0.12867
   • Previous kernel with thresold 0.55 instead on 0.5
   • ver4 LB: 0.13976 didn’ improve
   • Bagged with 2/3 * ver5 LB: 0.13131
   • Bagged with 1/3
   • Thresold 0.55
   • ElasticNetCV L1_ratio = 0.2

5. Beginner ElasticNet + Univar_models Kernel ver.7 LB: 0.12860

6. Beginner ElasticNet Kernel Ver.5 LB 0.12811
   • ElasticNetCV L1_ratio = 0.1
   • ElasticNetCV alpha optimization
   • Bagged with 5. Minimal + Normalized X skew,y + filter low Var Kernel LB: 0.13764
7. Beginner ElasticNet Kernel Ver.7 LB 0.12408
   • ElasticNetCV L1_ratio = 1
   • ElasticNetCV alpha optimization
   • Bagged with
   1. Minimal + Normalized X skew,y + filter low Var Kernel LB: 0.13764 and
   2. Beginner ElasticNet Kernel Ver.5 LB 0.12811

Next things in the list are:

• max_iter=1000, tol=0.0001 ? optimize
• positive & Selection with t<1e-8
• NaN Imputation
• Outlier Remove
• Ensemble

Under Construction…