Last Updated: 2021-11-19

The L1 penalty leads to sparse solutions, driving most coefficients to zero. The Elastic Net solves some deficiencies of the L1 penalty in the presence of highly correlated attributes.

  • Hinge: (soft-margin) Support Vector Machines.
  • Log: Logistic Regression.
  • Least-Squares: Ridge Regression.
  • Epsilon-Insensitive: (soft-margin) Support Vector Regression.
  • L2 norm: ,
  • L1 norm: , which leads to sparse solutions.
  • ElasticNet: Convex combination of L2 and L1; (1 - l1*ratio) * L2 + l1*ratio * L1.