Logistic regression is one of the most widely used classification algorithms. Despite the name, it is a classification model — not regression. It predicts the probability of class membership using the sigmoid function. This guide covers it completely.
How Logistic Regression Works
Logistic regression applies a sigmoid function to a linear combination of features, mapping output to a probability between 0 and 1. A threshold (usually 0.5) converts the probability to a binary class label.
Binary Classification Example
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, roc_auc_score, roc_curve
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
y_prob = model.predict_proba(X_test)[:, 1]
print(classification_report(y_test, y_pred))
print(f'AUC-ROC: {roc_auc_score(y_test, y_prob):.4f}')
fpr, tpr, _ = roc_curve(y_test, y_prob)
plt.plot(fpr, tpr, label=f'AUC = {roc_auc_score(y_test, y_prob):.3f}')
plt.plot([0,1],[0,1],'k--')
plt.xlabel('False Positive Rate'); plt.ylabel('True Positive Rate')
plt.title('ROC Curve'); plt.legend(); plt.show()Multiclass Classification
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression(multi_class='auto', max_iter=500)
model.fit(X_train, y_train)
print(f'Accuracy: {model.score(X_test, y_test):.4f}')
print(classification_report(y_test, model.predict(X_test)))Important Hyperparameters
| Parameter | Options | When to use |
|---|---|---|
| C | 0.001 to 100 | Lower = more regularisation |
| penalty | l1, l2, elasticnet | l1 for feature selection |
| solver | lbfgs, saga, liblinear | saga for l1 + large datasets |
| class_weight | balanced, None | balanced for imbalanced data |
FAQ
Does logistic regression need feature scaling?
Yes — always standardise features. Logistic regression uses gradient-based optimisation which converges much faster and more reliably with scaled features.
Logistic regression vs SVM vs Random Forest?
✦ SUMMARIZE THIS ARTICLE WITH AI
ChatGPTUse logistic regression when you need probability outputs and interpretability. Random Forest when accuracy is the priority. SVM for high-dimensional data with clear margins.
