Python Machine Learning

Logistic Regression

Sigmoid function

$$f(x) = \frac{1}{1+e^{-(x)}}$$

Logistic regression cost function

$$J(\theta)=\frac{1}{m}\sum_{i=1}^m(cost(h_\theta(x^{(i)}),y^{(i)}))$$

$$Cost(h_\theta(x),y)=\begin{cases}-log(h_\theta(x)) & \text{if y = 1}\\ -log(1 - h_\theta(x)) & \text{if y = 0}\end{cases}$$

Note y = 0 or y = 1 always

$$Cost(h_\theta(x),y)=-y*log(h_\theta(x))-(1-y)*log(1-h_\theta(x))$$

$$J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}{y^{(i)}log(h_\theta(x^{(i)}))+(1-y^{(i)})*log(1-h_\theta(x^{(i)}))}$$

$$\text{To fit parameter }\theta: min J(\theta)$$

Gradient Descent

$$\text{Repeat }\{\theta_j := \theta_j-\alpha\frac{\sigma}{\sigma\theta_j}J(\theta)\}$$

$$\text{Repeat }\{\theta_j := \theta_j-\alpha\sum_{i=1}^{m}(h_\theta(x^{(i)})- y^{(i)})* x_j^{(i)}\}$$

Data

HoursPass
.50
0.750
10
1.250
1.50
1.750
1.751
20
2.251
2.50
2.751
30
3.251
3.50
41
4.251
4.51
4.751
51
5.51