Logistic Regression in a simple way.

Jul 21, 20212 min read

Hello folks ,

Today i am going to talk about geometric intuition of Logistic Regression. Trust me it will going to be fun. Okay, first lets discuss our roadmap.

Understanding figure
Assumption
Derivation
Test cases
why using Sigmoid ?
Final Optimization Equation

Understanding Figure:-

This is how our classifier works when we have linearly separable data. Here our classifier try to fit a plane that separates our positive points from negative points.

If the plane passes through origin then b = 0 , equation of plane becomes wTx = 0.

Here x , w are d dimensional vector and b is a scalar value.

Assumption :- Here we assume classes are almost linearly separable.

Derivation:-

Di = wtxi > 0 (same direction of unit vector w)

Dj = wTxi < 0 (opp direction of unit vector w)

Now ,

Classifier says if wTxi >0 then it is positive point and when wTxi <0 then it is negative point.

Test Cases:-

Remember :- yiwTxi = it is known as signed distance.

Case 1 : xi is positive point means yi = +ve.

Yi * wTxi > 0 wTxi > 0 ; classifier is saying it is a positive point.

(+) (+)

Hence w is correctly classifying our point.

Case 2 : xi is negative point means yi = -ve.

Yi * wTxi > 0 wTxi < 0 ; classifier is saying it is a negative point.

(-) (-)

Hence w is correctly classifying our point.

Case 3 : xi is positive point means yi = +ve.

Yi * wTxi < 0 wTxi < 0 ; classifier is saying it is a negative point.

(+) (-)

Here yi is positive but our LR says it is a negative point.Hence w is incorrectly classifying our point.

Case 4 : xi is negative point means yi = -ve.

Yi * wTxi < 0 wTxi > 0 ; classifier is saying it is a positive point.

(-) (+)

Here yi is negative but our LR says it is a positive point.Hence w is incorrectly classifying our point.

Summary - we have seen from above , to classify every point correctly we need our signed distance to be positive. Hence all we want is max sigma(yiwTxi). So our task is to find suitable plane of equation