logo

Linear Regression

Hypothesis

h θ ( x ) = θ T x h_\theta(x) = \theta^T x

Parameter

θ \theta

Cost function:

J ( θ ) = 1 2 m i = 1 m ( h θ ( x ( i ) ) y ( i ) ) 2 J(\theta) = {1 \over 2m} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})^2

Goal

min θ J ( θ ) \underset{\theta}{\min} J(\theta)

Gradient Descent: simultaneously update all θ j \theta_j

θ j : = θ j α θ j J ( θ ) \theta_j:=\theta_j - \alpha {\partial \over \partial \theta_j} J(\theta)

Regularization

J ( θ ) = 1 2 m [ i = 1 m ( h θ ( x ( i ) ) y ( i ) ) 2 + λ i = 1 n θ j 2 ] J(\theta) = {1 \over 2m} {\Big [} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})^2 + \lambda \sum_{i=1}^n \theta_j^2 {\Big ]}