Linear Regression in Machine Learning: How It Works, Formula and Step-by-Step Examples (Updated August 2026)

Linear Regression in Machine Learning: How It Works, Formula and Step-by-Step Examples (Updated September 2026)

Linear regression is where every machine learning engineer's journey with supervised learning begins. It is the simplest algorithm that actually does something useful — predict a continuous number from input features — and it introduces every concept you will carry into more advanced models: learning from labelled data, minimising error, evaluating with metrics, and understanding the bias-variance trade-off. The NASSCOM-Deloitte report projects India needs 1.25 million AI professionals by 2027, and linear regression remains the first algorithm asked about in data science interviews. This guide, grounded in the ABC Trainings Proficient ML programme, explains linear regression from first principles to practical evaluation.

How to Build a Linear Regression Model in Python: Step-by-Step (Updated September 2026)

Building a linear regression model in Python takes five steps: import the data, split it with train_test_split, instantiate LinearRegression(), call fit(X_train, y_train), then evaluate with r2_score and mean_squared_error — a working model in under 25 lines. Here is the complete walkthrough using a house-price prediction example, the same exercise covered in ABC Trainings' Proficient ML programme.

Step 1 — Import libraries and load data. import numpy as np, pandas as pd and from sklearn.model_selection import train_test_split. Load your CSV with df = pd.read_csv('house_prices.csv'). Inspect with df.head() and df.info() to spot missing values.

Step 2 — Prepare features and target. Separate X = df[['area_sqft','location_score','age_years']] (features) from y = df['price_lakhs'] (target). Handle missing values: X = X.fillna(X.mean()). Then split: X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42).

Step 3 — Train the model. from sklearn.linear_model import LinearRegression; model = LinearRegression(); model.fit(X_train, y_train). After fitting, model.coef_ gives the slope for each feature (e.g., ₹12.4 lakhs per 100 sq ft) and model.intercept_ gives β0.

Step 4 — Predict. y_pred = model.predict(X_test). Compare a few: print(y_pred[:5], y_test[:5].values).

Step 5 — Evaluate. from sklearn.metrics import mean_squared_error, r2_score; rmse = np.sqrt(mean_squared_error(y_test, y_pred)); r2 = r2_score(y_test, y_pred); print(f'RMSE: {rmse:.2f}L | R²: {r2:.3f}'). An R² above 0.75 and an RMSE below 10L is a solid baseline for a 3-feature house price model. If R² is below 0.5, add more features or check for outliers with df.describe().

The most common mistake beginners make: forgetting to check for multicollinearity (when two features are highly correlated, their individual coefficients become unreliable). Use df.corr() to spot it. If two features correlate above 0.85, drop one before training. This is exactly the kind of real-world detail covered in the ABC Trainings ML programme — not just the API calls but the judgment calls.

What Is Linear Regression in Machine Learning?

Linear regression is a supervised machine learning algorithm that learns a linear relationship between one or more input features and a continuous output variable. Given a labelled dataset — historical examples where both inputs and correct outputs are known — linear regression finds the straight line (or hyperplane in multiple dimensions) that best fits the data. It then uses that line to predict outputs for new inputs. The word 'linear' means the relationship between inputs and output is assumed to be a straight line — as the input increases by one unit, the output changes by a fixed amount (the slope). Linear regression is both the simplest and the most widely understood ML algorithm: it is fully interpretable, fast, and the mathematical foundation for logistic regression, ridge regression, lasso regression and many other methods.

Simple vs Multiple Linear Regression: Key Differences

Simple linear regression uses one independent variable to predict the dependent variable — for example, predicting a student's exam score from the number of hours studied. It produces one slope coefficient and one intercept. Multiple linear regression uses two or more independent variables — for example, predicting house price from size (sq ft), location score and number of rooms simultaneously. Each independent variable gets its own slope coefficient, and there is one intercept. When to use which: if only one predictor is relevant or you're building a baseline model, use simple linear regression first. In most real-world problems, multiple features matter — use multiple linear regression. Both are implemented identically in sklearn's LinearRegression() class; the difference is only in how many feature columns you pass.

Metric	Formula	What it tells you	Better when
MSE	Mean of squared errors	Average squared prediction error	Lower
RMSE	√MSE	Error in original units (interpretable)	Lower
R²	1 – (SS_res / SS_tot)	% of variance explained by model	Closer to 1

The Linear Regression Formula: Slope, Intercept and the Equation

The simple linear regression equation is: y = β0 + β1x, where y is the predicted value (dependent variable), x is the input feature (independent variable), β0 is the intercept (the value of y when x = 0, where the line crosses the y-axis), and β1 is the slope (how much y changes when x increases by one unit). The multiple regression equation extends this: y = β0 + β1x1 + β2x2 + ... + βnxn. In Python with sklearn: from sklearn.linear_model import LinearRegression; model = LinearRegression(); model.fit(X_train, y_train). After fitting, model.coef_ gives you the slope coefficients (β1, β2, ...) and model.intercept_ gives β0. These numbers tell you the exact effect of each feature on the prediction — one of linear regression's key advantages over black-box models.

How Linear Regression Finds the Best Fit Line: Least Squares Method

The best fit line minimises the total prediction error across all training examples. The standard method is Ordinary Least Squares (OLS), which minimises the sum of squared residuals — where a residual is the difference between the actual value (y) and the predicted value (ŷ) for each training point. Squaring the residuals serves two purposes: it makes all errors positive (so positive and negative errors don't cancel out) and it penalises large errors more heavily than small ones. The OLS solution has a closed-form mathematical solution — it can be calculated directly from the training data without iterative optimisation. In practice, sklearn's LinearRegression uses this closed-form solution, which is why linear regression training is nearly instantaneous even on large datasets compared to gradient-descent-based methods.

Evaluating a Linear Regression Model: MSE, RMSE and R-Squared

Three metrics evaluate linear regression model quality. Mean Squared Error (MSE) is the average of squared prediction errors: MSE = (1/n) Σ(yi - ŷi)². Lower is better, but the unit is the square of the target variable (e.g., rupees²), making it hard to interpret directly. Root Mean Squared Error (RMSE) is the square root of MSE — it gives the average error in the original unit of the target variable (rupees, kg, °C). A house price model with RMSE of ₹2 lakhs means predictions are typically off by about ₹2 lakhs. R-squared (R²) measures the proportion of variance in the target explained by the model, ranging from 0 (model explains nothing) to 1.0 (perfect). An R² of 0.85 means the model explains 85% of the variation in the target variable. Always report both RMSE and R² — RMSE tells you error magnitude; R² tells you explanatory power.

Real-World Applications of Linear Regression in Industry

Linear regression is used wherever a continuous output depends on measurable inputs. In Pune's real estate market, builders and banks use regression models to estimate property values from area, floor, location, age and amenity proximity. In manufacturing plants at Chakan and Talegaon, regression models predict product defect rates from process parameters like temperature, pressure and machine speed. In HR, regression estimates expected salary ranges from experience, skills, location and role seniority — the basis for compensation benchmarking tools used by HR consultancies. In agriculture, regression models predict crop yield from rainfall, temperature and soil quality data. In finance, regression forecasts quarterly revenue from marketing spend, seasonality and economic indicators. Every domain that collects numerical data about inputs and outcomes has a potential linear regression application.

Linear Regression Training at ABC Trainings Pune: What You Build

In the ABC Trainings Proficient ML programme, linear regression is the first algorithm implemented in Python. Students start by understanding the formula, then implement it from scratch using NumPy to calculate slope and intercept manually, then use sklearn's LinearRegression for production-ready code. They build a complete project — data loading, preprocessing, splitting, training, prediction, and RMSE/R² evaluation — before moving to polynomial regression and classification algorithms. This from-scratch-to-framework approach means students understand what sklearn is doing, not just how to call it. Batches run in Wagholi and Hadapsar. Programme fees start at ₹25,000; CMKPY-eligible students can reclaim ₹6,000–₹10,000. Call 7039169629 or WhatsApp 7774002496 to check current batch dates.

Eligible students can apply for CMKPY (Chief Minister Yuva Karyaprasaran Yojana) skill training reimbursement of ₹6,000–₹10,000 toward approved ML and data science courses. Ask ABC Trainings whether the current ML batch is CMKPY-empanelled when you enquire.

FAQs

What is linear regression in machine learning in simple terms?

Linear regression is a supervised ML algorithm that learns a straight-line relationship between input features and a continuous numerical output. Given training data (inputs + correct outputs), it finds the slope and intercept that minimise prediction error. Once trained, it predicts outputs for new inputs by plugging values into the equation y = β0 + β1x.

What is the difference between simple and multiple linear regression?

Simple linear regression uses one independent variable (one feature) to predict the target. Multiple linear regression uses two or more independent variables simultaneously. In real-world ML projects, multiple regression is far more common because most outcomes are influenced by several factors at once — a house price depends on size, location, age and amenities, not just one factor.

What is R-squared and what is a good R-squared value?

R-squared (R²) measures the proportion of variance in the target variable explained by the model, on a scale from 0 to 1. An R² of 0.80 means the model explains 80% of the variation in the target. There is no universal 'good' R² — it depends on the domain. In highly variable social science data, 0.5 may be excellent. In precise engineering measurements, 0.95 might be expected. Always pair R² with RMSE for a complete picture of model quality.

Is linear regression still used in industry in 2026?

Yes. Linear regression is used in production systems at major companies in 2026. It is the baseline model in every regression project because it is fast, interpretable and often competitive with more complex models on well-structured data. Even when a random forest or XGBoost ultimately outperforms it, linear regression provides the interpretability that finance, healthcare and HR domains often require for regulatory compliance.

5 Assumptions of Linear Regression You Must Check Before Building a Model

The 5 assumptions of linear regression are linearity, independence of errors, homoscedasticity, normality of residuals, and no multicollinearity — violating any of these produces coefficients that are biased, inconsistent, or have inflated standard errors, making your model unreliable for inference or prediction.

Here is what each assumption means in practice and how to test it. Linearity: the relationship between each predictor and the target must be linear. Test with scatter plots of each feature vs. the target or with partial regression plots. If you see a curve, try adding polynomial features or a log transformation. Independence of errors: residuals must not be correlated with each other — critical for time-series data. Test with the Durbin-Watson statistic (values near 2 indicate no autocorrelation). Homoscedasticity: residual variance should be constant across all predicted values. Test with a residuals vs. fitted values plot — a funnel shape indicates heteroscedasticity; fix with log-transforming the target or using weighted least squares. Normality of residuals: residuals should be approximately normally distributed. Test with a Q-Q plot (points should fall on the diagonal line) or Shapiro-Wilk test. Note: this assumption matters most for small samples; for large datasets (n > 100), CLT makes the model robust to mild violations. No multicollinearity: predictors should not be highly correlated with each other. Test with Variance Inflation Factor (VIF) — VIF > 10 indicates problematic collinearity; fix by removing correlated features or using Ridge regression.

In practice, assumption checking is not a blocker but a diagnostic. Real-world data frequently violates one or more assumptions. The question is not whether the assumptions are perfectly met but whether the violations are severe enough to invalidate the model for your use case. In a production system at a finance or HR company where interpretability is required, assumption checking is a compliance step. In an exploratory model where prediction accuracy is the only goal, mild violations are often acceptable. Python libraries that help: statsmodels (OLS summary with residual diagnostics), scipy.stats (Shapiro-Wilk, Durbin-Watson), statsmodels.stats.outliers_influence (VIF calculation).

When Linear Regression Fails: Limitations and Better Alternatives in 2026

Linear regression fails when the relationship between features and target is genuinely non-linear, when the target variable is categorical or count-based rather than continuous, when there is severe multicollinearity, or when outliers dominate the least-squares solution — in each of these cases, a different model will outperform it, often dramatically.

Specific failure modes and their solutions: Non-linear relationships — if a scatter plot shows a curve, polynomial regression (adding squared/cubed terms), decision trees, or random forests will capture it better. Binary or multi-class target variable — linear regression will output probabilities outside [0,1] and is conceptually wrong; use logistic regression or a classification model. Count data (number of events, purchases) — use Poisson regression or negative binomial regression; linear regression will predict negative counts and underfit the variance structure. Severe outliers — a single outlier can pull the regression line significantly; use Huber regression or RANSAC regression which are robust to outliers. High-dimensional data with multicollinearity — use Ridge regression (L2 regularisation), Lasso (L1, which zeroes out less important features), or Elastic Net. Time-series with autocorrelated errors — use ARIMA or add lag features explicitly; standard OLS will have inflated t-statistics. Interaction effects between features — add explicit interaction terms (feature1 × feature2) or use gradient boosting which captures interactions automatically.

The practical decision rule for data science students building a regression model: start with linear regression as your baseline (it is interpretable, fast, and often competitive on clean structured data), check the 5 assumptions above, and then try Ridge/Lasso if you have multicollinearity or many features, XGBoost or Random Forest if you see non-linearity in residual plots, and logistic regression if your business question is actually a classification problem in disguise. Linear regression's greatest strength — interpretability of coefficients — is also its most important differentiator in regulated industries where model explainability is required by compliance.

Linear regression vs logistic regression vs Ridge regression: which model should you use in your machine learning project?

Use linear regression when your target variable is continuous (price, temperature, sales volume) and the relationship with your features is approximately linear — it gives you interpretable coefficients and is required as a baseline in any regression task. Use logistic regression when your target is binary (yes/no, churn/no-churn, fraud/no-fraud) or multi-class — it models the log-odds of each class and outputs calibrated probabilities, making it the standard baseline for classification. Use Ridge regression (linear regression with L2 regularisation) when you have many correlated features, when your dataset has more features than observations, or when plain linear regression is overfitting — Ridge penalises large coefficients and reduces variance at the cost of a small bias increase; it is strictly better than plain linear regression when multicollinearity is present. The decision rule: continuous target + approximately linear relationship = linear regression; binary or multi-class target = logistic regression; continuous target + many correlated features or high-dimensional data = Ridge or Lasso regression.

How to Build a Linear Regression Model in Python — 5 Steps (Updated October 2026)

To build a linear regression model in Python, you need five steps: prepare your dataset, split it 80/20 with train_test_split, call LinearRegression().fit(X_train, y_train), then evaluate with R² and RMSE on the test set — the entire workflow runs in under 20 lines of scikit-learn code. This is the exact pipeline that Infosys, TCS, and KPIT data science panels test in interviews, so knowing it cold is non-negotiable.

1. Prepare the data — Load with pandas, check for nulls (df.isnull().sum()), separate features X from the continuous target y.
2. Split — X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42). The 80/20 split is the industry standard starting point.
3. Fit the model — model = LinearRegression(); model.fit(X_train, y_train). OLS (Ordinary Least Squares) finds the slope and intercept that minimise Mean Squared Error automatically.
4. Predict — y_pred = model.predict(X_test). Always predict on the held-out test set, never the training set.
5. Evaluate — r2_score(y_test, y_pred) (target ≥ 0.75 for structured business data); mean_squared_error(y_test, y_pred, squared=False) gives RMSE in original units like rupees or units.

What most students don’t realise: model.coef_ gives each feature’s weight, making linear regression directly interpretable — this is why KPIT, Bajaj Tech, and Bosch India still use it for salary benchmarking and production quality modelling in 2026, even with deep learning available. ABC Trainings covers this complete sklearn pipeline in the AI Powered Application Development batch — centres in Pune, Chh. Sambhajinagar, Sangli, and Latur. Call +91 7039169629 to book a free demo this October.