Linear Regression: A Complete Guide for Beginners

Linear Regression is one of the most fundamental algorithms in machine learning. It is widely used for predicting a continuous numerical value, like predicting house prices, salary based on experience, or temperature forecasts. In this guide, we will cover: What Linear Regression is How it works Key concepts (loss function, gradient descent) Implementation in PyTorch with examples How to train, evaluate, and use a model 1. What is Linear Regression? Linear Regression is a method to model the relationship between one or more input features (independent variables) and a continuous output (dependent variable). The simplest form is Simple Linear Regression, where there is only one input feature: Predicted output (y) equals the weight (w) multiplied by the input (x) plus the bias (b). In other words: y = w * x + b x = input feature y = predicted output w = weight (slope) b = bias (intercept) For multiple features, we use Multiple Linear Regression: Predicted output y equals the sum of each weight multiplied by its corresponding feature plus the bias. In other words: y = w1x1 + w2x2 + ... + wn*xn + b 2. How Linear Regression Works The main goal of Linear Regression is to find the values of weights and bias that minimize the difference between the predicted output and the actual output. 2.1 Loss Function We measure the error using Mean Squared Error (MSE): MSE is calculated as the average of the squared differences between predicted values and actual values. In words: MSE = (1 / number of samples) times the sum of (predicted value minus actual value) squared for all samples. 2.2 Optimization We use Gradient Descent to update weights and bias: Each weight is updated by subtracting the learning rate multiplied by the derivative of the loss function with respect to that weight. Similarly, the bias is updated by subtracting the learning rate multiplied by the derivative of the loss function with respect to the bias. PyTorch can automatically compute gradients using autograd, so we don’t have to calculate them manually. 3. Implementing Linear Regression in PyTorch Let's implement a simple Linear Regression example to predict house prices based on square footage. 3.1 Import Libraries

import torch
import torch.nn as nn
import torch.optim as optim

3.2 Prepare Data

# Sample data (square footage vs price)
x_train = torch.tensor([[50.0], [60.0], [70.0], [80.0], [90.0]])
y_train = torch.tensor([[150.0], [180.0], [210.0], [240.0], [270.0]])

3.3 Define the Model

class LinearRegressionModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(1, 1)  # 1 input, 1 output

    def forward(self, x):
        return self.linear(x)

model = LinearRegressionModel()

3.4 Define Loss and Optimizer

criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.0001)

3.5 Train the Model

num_epochs = 1000

for epoch in range(num_epochs):
    model.train()
    
    # Forward pass
    y_pred = model(x_train)
    loss = criterion(y_pred, y_train)
    
    # Backward pass
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if (epoch+1) % 100 == 0:
        print(f'Epoch {epoch+1}, Loss: {loss.item():.4f}')

4. Evaluate the Model After training, you can test predictions

model.eval()
with torch.no_grad():
    x_test = torch.tensor([[100.0]])
    predicted = model(x_test)
    print(f"Predicted price for 100 sq.ft: {predicted.item():.2f}")

5. Key Notes and Best Practices Feature Scaling: For multiple features, normalize or standardize features for faster convergence. Learning Rate: Choose a proper learning rate; too high leads to divergence, too low leads to slow training. Evaluation: Use MSE or MAE to check model accuracy. Real-World Use: Linear Regression works best for linear relationships. For non-linear patterns, use more advanced models like polynomial regression or neural networks. Summary: - Linear Regression predicts continuous outputs. - Uses weights and bias to fit a line through data. - Loss function (MSE) measures prediction error. - Gradient Descent updates weights automatically in PyTorch. - Simple to implement and understand; foundational for learning more complex models.

Confirm Delete