Building Your First Neural Network with PyTorch

PyTorch is one of the most widely used frameworks for Deep Learning.

In this tutorial, we'll go through the basic components required to create and train a neural network with PyTorch.

By the end, you'll understand the relationship between:

• Tensors
• Models
• Loss functions
• Optimizers
• Backpropagation
• Training loops

1. Install PyTorch

You can install PyTorch using the official installation instructions for your operating system and hardware.

Once installed, verify it:

import torch

print(torch.__version__)

2. Create Some Data

Let's create a simple regression problem.

import torch

X = torch.randn(100, 1)

y = 3 * X + 2

Our model should learn the relationship:

y = 3x + 2

3. Create the Model

We'll use nn.Module.

import torch.nn as nn

class LinearModel(nn.Module):

    def __init__(self):
        super().__init__()

        self.linear = nn.Linear(1, 1)

    def forward(self, x):
        return self.linear(x)

Create the model:

model = LinearModel()

print(model)

4. Define the Loss Function

Because this is a regression problem, we'll use Mean Squared Error.

criterion = nn.MSELoss()

The loss tells us how far the predictions are from the target values.

5. Create an Optimizer

We'll use Adam:

optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.01
)

The optimizer is responsible for updating the model parameters using the gradients.

6. Training Loop

Now we can train the model.

for epoch in range(1000):

    optimizer.zero_grad()

    prediction = model(X)

    loss = criterion(prediction, y)

    loss.backward()

    optimizer.step()

    if epoch % 100 == 0:
        print(
            f"Epoch: {epoch}, Loss: {loss.item():.4f}"
        )

Let's understand what happens here.

optimizer.zero_grad()

PyTorch accumulates gradients by default.

Therefore, we clear the previous gradients before calculating new ones.

prediction = model(X)

We perform the forward pass.

The input goes through the neural network and produces predictions.

loss = criterion(prediction, y)

We calculate how wrong the predictions are.

loss.backward()

This performs backpropagation and calculates gradients.

optimizer.step()

The optimizer uses the gradients to update the model parameters.

This cycle is repeated many times.

7. Test the Model

After training:

test_x = torch.tensor([[5.0]])

prediction = model(test_x)

print(prediction)

The expected output should be close to:

17

because:

3 × 5 + 2 = 17

The Complete Training Process

The entire process can be summarized as:

Input
  ↓
Forward Pass
  ↓
Prediction
  ↓
Loss Calculation
  ↓
Backward Pass
  ↓
Gradients
  ↓
Optimizer
  ↓
Updated Parameters

This pattern appears again and again in Deep Learning.

Once you understand it, more advanced architectures become much easier to understand.

What's Next?

After mastering a simple neural network, you can move to:

1. Dataset and DataLoader
2. Classification
3. CNNs
4. GPU training
5. Attention
6. Transformers
7. Large Language Models

My tutorials cover PyTorch, Transformers, Machine Learning frameworks, and modern AI, with practical implementations from scratch.

YouTube tutorials: https://www.youtube.com/@Tahahussein-Ai

Source code: https://github.com/Taha2hussein

If you found this tutorial useful, feel free to share your questions or improvements in the comments.