Linear Regression

Updated: 2019-01-03

Hypothesis

hθ(x)=θTxh_\theta(x) = \theta^T x

Parameter

θ\theta

Cost function:

J(θ)=12mi=1m(hθ(x(i))y(i))2J(\theta) = {1 \over 2m} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})^2

Goal

minθJ(θ)\underset{\theta}{\min} J(\theta)

Gradient Descent: simultaneously update all θj\theta_j

θj:=θjαθjJ(θ)\theta_j:=\theta_j - \alpha {\partial \over \partial \theta_j} J(\theta)

Regularization

J(θ)=12m[i=1m(hθ(x(i))y(i))2+λi=1nθj2]J(\theta) = {1 \over 2m} {\Big [} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})^2 + \lambda \sum_{i=1}^n \theta_j^2 {\Big ]}