Logistic Regression Explained: Sigmoid Function, Binary Classification and When to Use It (Updated August 2026)

Logistic Regression Explained: Sigmoid Function, Binary Classification and When to Use It (Updated August 2026) (Updated August 2026)

If linear regression answers 'how much?', logistic regression answers 'which category?' — and it is the first classification algorithm every machine learning engineer learns. From spam detection to medical diagnosis to credit risk scoring, logistic regression powers binary decisions across industries. The NASSCOM-Deloitte report projects India needs 1.25 million AI professionals by 2027, and logistic regression appears in data science interviews more consistently than almost any other algorithm. This guide, grounded in the ABC Trainings Proficient ML curriculum, explains logistic regression from the sigmoid function through to the assumptions, types and evaluation metrics you'll use in real projects.

What Is Logistic Regression and How Is It Different from Linear Regression?

Logistic regression is a classification algorithm — despite 'regression' being in its name. Where linear regression predicts a continuous number, logistic regression predicts the probability that an input belongs to a specific class, then assigns the input to the class with the higher probability. It is used for binary classification (two classes: yes/no, spam/not-spam, fraud/legitimate) and can be extended to multi-class problems. The name comes from the logistic function (sigmoid) used internally. It is an extension of linear regression: instead of outputting a raw score, it passes the linear combination of features through the sigmoid function to produce a probability output between 0 and 1. The class boundary is where probability = 0.5 — everything above is Class 1, everything below is Class 0.

The Sigmoid Function: How Logistic Regression Maps Inputs to Probabilities

The sigmoid function is the mathematical heart of logistic regression. Its formula is σ(z) = 1 / (1 + e^(-z)), where z is the linear combination of features: z = β0 + β1x1 + β2x2 + ... + βnxn. No matter what value z takes — positive, negative, very large or very small — σ(z) always outputs a number between 0 and 1. This makes it interpretable as a probability. The output curve is S-shaped (the 'sigmoid' or 'logistic' curve): for very negative z values it approaches 0; for very positive z values it approaches 1; at z = 0 it outputs exactly 0.5. In Python: from sklearn.linear_model import LogisticRegression; model = LogisticRegression(); model.fit(X_train, y_train). The model.predict_proba() method returns the class probabilities; model.predict() returns the class label based on the 0.5 threshold.

Type	Classes	Function used	Example
Binary	2	Sigmoid	Spam vs not-spam, fraud vs legitimate
Multinomial	3+ (unordered)	Softmax	Document topic, product category
Ordinal	3+ (ordered)	Cumulative logit	Satisfaction rating, severity level

Binary, Multinomial and Ordinal: Three Types of Logistic Regression

Logistic regression has three main types determined by the nature of the output variable. Binary (binomial) logistic regression handles exactly two classes — the most common type. Examples: email spam/not-spam, customer churned/stayed, transaction fraud/legitimate, loan default/no-default. The sigmoid function is used. Multinomial logistic regression handles three or more classes with no natural ordering between them. Examples: document topic classification (sports/politics/technology/entertainment), image classification (cat/dog/bird), product category prediction. The softmax function replaces the sigmoid to output probabilities across all classes that sum to 1. Ordinal logistic regression handles three or more ordered categories where the ranking matters. Examples: customer satisfaction rating (low/medium/high), severity classification (mild/moderate/severe). The ordering constraint is built into the model.

Key Assumptions and Characteristics of Logistic Regression

Logistic regression works best when four conditions hold. First, the dependent variable must be categorical — binary for standard logistic regression, with exactly two possible values. If you have more than two categories without ordering, use multinomial logistic regression. Second, observations should be independent — each row in your dataset should represent a different, unrelated individual or event. Third, there should be minimal multicollinearity among predictor variables — when two features are highly correlated, the model struggles to separate their individual effects. Fourth, there should be a linear relationship between the independent variables and the log-odds of the dependent variable — not the probability itself, but its logit transformation. Violating these assumptions doesn't always break the model, but it reduces reliability and makes coefficient interpretation invalid.

Maximum Likelihood Estimation and the Softmax Function

Maximum likelihood estimation (MLE) is how logistic regression finds its coefficients. Unlike linear regression which uses the closed-form OLS solution, logistic regression optimises the likelihood function — the probability of observing the training data given the model parameters. MLE finds the coefficient values that maximise this likelihood. In practice, the log-likelihood is maximised (equivalent, but numerically more stable), using iterative optimisation algorithms like gradient descent or Newton-Raphson. The softmax function is the generalisation of sigmoid for multinomial classification. For K classes, softmax outputs K probabilities that sum to 1: P(class k | x) = e^(zk) / Σ e^(zj) for all j. Each class gets a score, and softmax converts these scores to probabilities. The class with the highest probability is the prediction. Sklearn's LogisticRegression with multi_class='multinomial' and solver='lbfgs' implements softmax automatically.

When to Use Logistic Regression vs Other Classification Algorithms

Logistic regression is the right first choice when your problem is binary classification with a relatively small number of features, you need interpretable results (the coefficients are log-odds, which can be converted to odds ratios), you need fast training and prediction, and your features have approximately linear relationships with the log-odds of the outcome. It may underperform compared to decision trees or random forests when features have complex non-linear relationships or many interactions, when there are many categorical features requiring extensive encoding, or when the classes are highly imbalanced without additional resampling. Use logistic regression as your baseline classifier — if a more complex model doesn't significantly outperform it, use logistic regression in production for its simplicity and interpretability.

Logistic Regression Training at ABC Trainings Pune: What You Build

In the ABC Trainings Proficient ML programme, logistic regression is taught immediately after linear regression, building on the same linear combination concept but adding the sigmoid function and classification framing. Students implement binary logistic regression from scratch (computing sigmoid manually in NumPy), then switch to sklearn for multi-class examples using softmax. Evaluation metrics — accuracy, precision, recall, F1 score and the confusion matrix — are covered alongside the algorithm. Students build a complete spam detection or loan default prediction project before moving to decision trees. This project-first approach means every student graduates with a live working classification model in their portfolio. Batches run at Wagholi and Hadapsar. Call 7039169629 or WhatsApp 7774002496 to check current dates and CMKPY eligibility.

Eligible students can apply for CMKPY (Chief Minister Yuva Karyaprasaran Yojana) skill training reimbursement of ₹6,000–₹10,000 toward approved ML and data science courses. Ask ABC Trainings whether the current ML batch is CMKPY-empanelled when you enquire.

FAQs

What is logistic regression and why is it called regression if it classifies?

Logistic regression is a classification algorithm that predicts the probability of a categorical outcome. It is called 'regression' because it extends linear regression — it uses a linear combination of features (like linear regression) but passes the result through the sigmoid function to output a probability between 0 and 1, which is then thresholded to produce a class prediction.

What is the sigmoid function in logistic regression?

The sigmoid function σ(z) = 1 / (1 + e^(-z)) takes any real number z and maps it to a value between 0 and 1. In logistic regression, z is the linear combination of features (β0 + β1x1 + ...). The sigmoid output is interpreted as the probability that the input belongs to Class 1. If the probability is above 0.5, the model predicts Class 1; otherwise Class 0. The S-shaped curve of the sigmoid is what gives logistic regression its characteristic probability output.

What is the difference between binary and multinomial logistic regression?

Binary logistic regression handles two classes (e.g., spam vs not-spam) and uses the sigmoid function. Multinomial logistic regression handles three or more unordered classes (e.g., cat/dog/bird) and uses the softmax function, which outputs a probability for each class that sums to 1 across all classes. Ordinal logistic regression handles ordered categories (e.g., low/medium/high severity) with additional constraints that respect the ordering.

When should I use logistic regression instead of a decision tree or random forest?

Use logistic regression when you need interpretable coefficients (log-odds) for regulatory or explainability requirements, when your dataset is small to medium and training speed matters, or when features have approximately linear relationships with the log-odds. Use decision trees or random forests when features have complex non-linear interactions, when you have many categorical features, or when you need higher accuracy and interpretability is less critical.