Regression

The L1 penalty leads to sparse solutions, driving most coefficients to zero. The Elastic Net solves some deficiencies of the L1 penalty in the presence of highly correlated attributes.

• Hinge: (soft-margin) Support Vector Machines.
• Log: Logistic Regression.
• Least-Squares: Ridge Regression.
• Epsilon-Insensitive: (soft-margin) Support Vector Regression.
• L2 norm: ,
• L1 norm: , which leads to sparse solutions.
• ElasticNet: Convex combination of L2 and L1; (1 - l1*ratio) * L2 + l1*ratio * L1.