Last Updated: 2021-11-19
The L1 penalty leads to sparse solutions, driving most coefficients to zero. The Elastic Net solves some deficiencies of the L1 penalty in the presence of highly correlated attributes.
- Hinge: (soft-margin) Support Vector Machines.
- Log: Logistic Regression.
- Least-Squares: Ridge Regression.
- Epsilon-Insensitive: (soft-margin) Support Vector Regression.
- L2 norm: ,
- L1 norm: , which leads to sparse solutions.
- ElasticNet: Convex combination of L2 and L1;
(1 - l1*ratio) * L2 + l1*ratio * L1.