logo

Regression

The L1 penalty leads to sparse solutions, driving most coefficients to zero. The Elastic Net solves some deficiencies of the L1 penalty in the presence of highly correlated attributes.

  • Hinge: (soft-margin) Support Vector Machines.
  • Log: Logistic Regression.
  • Least-Squares: Ridge Regression.
  • Epsilon-Insensitive: (soft-margin) Support Vector Regression.
  • L2 norm: ,
  • L1 norm: , which leads to sparse solutions.
  • ElasticNet: Convex combination of L2 and L1; (1 - l1*ratio) * L2 + l1*ratio * L1.