Linear Regression in Machine Learning: How It Works, Formula and Step-by-Step Examples (Updated August 2026)

Linear Regression in Machine Learning: How It Works, Formula and Step-by-Step Examples (Updated August 2026) (Updated August 2026)

Linear regression is where every machine learning engineer's journey with supervised learning begins. It is the simplest algorithm that actually does something useful — predict a continuous number from input features — and it introduces every concept you will carry into more advanced models: learning from labelled data, minimising error, evaluating with metrics, and understanding the bias-variance trade-off. The NASSCOM-Deloitte report projects India needs 1.25 million AI professionals by 2027, and linear regression remains the first algorithm asked about in data science interviews. This guide, grounded in the ABC Trainings Proficient ML programme, explains linear regression from first principles to practical evaluation.

What Is Linear Regression in Machine Learning?

Linear regression is a supervised machine learning algorithm that learns a linear relationship between one or more input features and a continuous output variable. Given a labelled dataset — historical examples where both inputs and correct outputs are known — linear regression finds the straight line (or hyperplane in multiple dimensions) that best fits the data. It then uses that line to predict outputs for new inputs. The word 'linear' means the relationship between inputs and output is assumed to be a straight line — as the input increases by one unit, the output changes by a fixed amount (the slope). Linear regression is both the simplest and the most widely understood ML algorithm: it is fully interpretable, fast, and the mathematical foundation for logistic regression, ridge regression, lasso regression and many other methods.

Simple vs Multiple Linear Regression: Key Differences

Simple linear regression uses one independent variable to predict the dependent variable — for example, predicting a student's exam score from the number of hours studied. It produces one slope coefficient and one intercept. Multiple linear regression uses two or more independent variables — for example, predicting house price from size (sq ft), location score and number of rooms simultaneously. Each independent variable gets its own slope coefficient, and there is one intercept. When to use which: if only one predictor is relevant or you're building a baseline model, use simple linear regression first. In most real-world problems, multiple features matter — use multiple linear regression. Both are implemented identically in sklearn's LinearRegression() class; the difference is only in how many feature columns you pass.

Metric	Formula	What it tells you	Better when
MSE	Mean of squared errors	Average squared prediction error	Lower
RMSE	√MSE	Error in original units (interpretable)	Lower
R²	1 – (SS_res / SS_tot)	% of variance explained by model	Closer to 1

The Linear Regression Formula: Slope, Intercept and the Equation

The simple linear regression equation is: y = β0 + β1x, where y is the predicted value (dependent variable), x is the input feature (independent variable), β0 is the intercept (the value of y when x = 0, where the line crosses the y-axis), and β1 is the slope (how much y changes when x increases by one unit). The multiple regression equation extends this: y = β0 + β1x1 + β2x2 + ... + βnxn. In Python with sklearn: from sklearn.linear_model import LinearRegression; model = LinearRegression(); model.fit(X_train, y_train). After fitting, model.coef_ gives you the slope coefficients (β1, β2, ...) and model.intercept_ gives β0. These numbers tell you the exact effect of each feature on the prediction — one of linear regression's key advantages over black-box models.

How Linear Regression Finds the Best Fit Line: Least Squares Method

The best fit line minimises the total prediction error across all training examples. The standard method is Ordinary Least Squares (OLS), which minimises the sum of squared residuals — where a residual is the difference between the actual value (y) and the predicted value (ŷ) for each training point. Squaring the residuals serves two purposes: it makes all errors positive (so positive and negative errors don't cancel out) and it penalises large errors more heavily than small ones. The OLS solution has a closed-form mathematical solution — it can be calculated directly from the training data without iterative optimisation. In practice, sklearn's LinearRegression uses this closed-form solution, which is why linear regression training is nearly instantaneous even on large datasets compared to gradient-descent-based methods.

Evaluating a Linear Regression Model: MSE, RMSE and R-Squared

Three metrics evaluate linear regression model quality. Mean Squared Error (MSE) is the average of squared prediction errors: MSE = (1/n) Σ(yi - ŷi)². Lower is better, but the unit is the square of the target variable (e.g., rupees²), making it hard to interpret directly. Root Mean Squared Error (RMSE) is the square root of MSE — it gives the average error in the original unit of the target variable (rupees, kg, °C). A house price model with RMSE of ₹2 lakhs means predictions are typically off by about ₹2 lakhs. R-squared (R²) measures the proportion of variance in the target explained by the model, ranging from 0 (model explains nothing) to 1.0 (perfect). An R² of 0.85 means the model explains 85% of the variation in the target variable. Always report both RMSE and R² — RMSE tells you error magnitude; R² tells you explanatory power.

Real-World Applications of Linear Regression in Industry

Linear regression is used wherever a continuous output depends on measurable inputs. In Pune's real estate market, builders and banks use regression models to estimate property values from area, floor, location, age and amenity proximity. In manufacturing plants at Chakan and Talegaon, regression models predict product defect rates from process parameters like temperature, pressure and machine speed. In HR, regression estimates expected salary ranges from experience, skills, location and role seniority — the basis for compensation benchmarking tools used by HR consultancies. In agriculture, regression models predict crop yield from rainfall, temperature and soil quality data. In finance, regression forecasts quarterly revenue from marketing spend, seasonality and economic indicators. Every domain that collects numerical data about inputs and outcomes has a potential linear regression application.

Linear Regression Training at ABC Trainings Pune: What You Build

In the ABC Trainings Proficient ML programme, linear regression is the first algorithm implemented in Python. Students start by understanding the formula, then implement it from scratch using NumPy to calculate slope and intercept manually, then use sklearn's LinearRegression for production-ready code. They build a complete project — data loading, preprocessing, splitting, training, prediction, and RMSE/R² evaluation — before moving to polynomial regression and classification algorithms. This from-scratch-to-framework approach means students understand what sklearn is doing, not just how to call it. Batches run in Wagholi and Hadapsar. Programme fees start at ₹25,000; CMKPY-eligible students can reclaim ₹6,000–₹10,000. Call 7039169629 or WhatsApp 7774002496 to check current batch dates.

Eligible students can apply for CMKPY (Chief Minister Yuva Karyaprasaran Yojana) skill training reimbursement of ₹6,000–₹10,000 toward approved ML and data science courses. Ask ABC Trainings whether the current ML batch is CMKPY-empanelled when you enquire.

FAQs

What is linear regression in machine learning in simple terms?

Linear regression is a supervised ML algorithm that learns a straight-line relationship between input features and a continuous numerical output. Given training data (inputs + correct outputs), it finds the slope and intercept that minimise prediction error. Once trained, it predicts outputs for new inputs by plugging values into the equation y = β0 + β1x.

What is the difference between simple and multiple linear regression?

Simple linear regression uses one independent variable (one feature) to predict the target. Multiple linear regression uses two or more independent variables simultaneously. In real-world ML projects, multiple regression is far more common because most outcomes are influenced by several factors at once — a house price depends on size, location, age and amenities, not just one factor.

What is R-squared and what is a good R-squared value?

R-squared (R²) measures the proportion of variance in the target variable explained by the model, on a scale from 0 to 1. An R² of 0.80 means the model explains 80% of the variation in the target. There is no universal 'good' R² — it depends on the domain. In highly variable social science data, 0.5 may be excellent. In precise engineering measurements, 0.95 might be expected. Always pair R² with RMSE for a complete picture of model quality.

Is linear regression still used in industry in 2026?

Yes. Linear regression is used in production systems at major companies in 2026. It is the baseline model in every regression project because it is fast, interpretable and often competitive with more complex models on well-structured data. Even when a random forest or XGBoost ultimately outperforms it, linear regression provides the interpretability that finance, healthcare and HR domains often require for regulatory compliance.