Custom Models (nn.Module) in Python

nn.Sequential is the simplest way to build a PyTorch model: stack layers in a list and data flows through them in a straight line, one after another. It works well for simple feedforward architectures, but it has a hard structural limitation — it only supports a single input flowing through a single linear chain of layers, with no branching.

Modern architectures routinely need more than that. A ResNet needs skip connections, where the output of an earlier layer is added back in several layers later, bypassing the layers in between. A Transformer needs multiple inputs combined at specific points, like combining token embeddings with positional encodings. Some architectures even need conditional logic — different layers activated depending on the input.

None of that fits nn.Sequential's straight-line model. To express architectures with branches, multiple inputs, or dynamic behavior, PyTorch needs a more general mechanism — and that mechanism is writing your own Python class using standard object-oriented programming.

Every custom PyTorch model starts the same way: a Python class that inherits from torch.nn.Module. This isn't just a naming convention — nn.Module is the base class that gives your model access to PyTorch's core machinery: automatic parameter tracking, the .to(device) method for moving everything to a GPU at once, .parameters() for handing weights to an optimizer, and integration with Autograd's computation graph.

Inheriting from nn.Module means your class gets all of that behavior for free, as long as you follow two conventions PyTorch expects: calling the parent class's constructor, and assigning any layers you create directly as attributes on self (rather than, say, storing them in a plain Python list) so nn.Module's internal bookkeeping can find and track them.

An empty class CustomNetwork(nn.Module): pass is a valid nn.Module, but not yet a useful one — it has no layers and no forward pass defined. Both of those come next.

PyTorch enforces this through its entire ecosystem: optimizers expect a .parameters() method to know what to update, .to(device) expects to know which attributes are movable tensors, and model.eval() / model.train() expect to know which submodules (like Dropout or BatchNorm) behave differently between training and evaluation. All of that machinery is implemented on nn.Module — a class that doesn't inherit from it doesn't get any of it automatically.

nn.Network and torch.DeepLearning aren't real PyTorch classes; the actual base class is specifically named nn.Module, found in the torch.nn package that's conventionally imported as import torch.nn as nn. Layers you already know — nn.Linear, nn.Conv2d — are themselves nn.Module subclasses, which is exactly why they can be assigned as attributes and automatically discovered by a parent module's .parameters() call.

This is also why nn.Module composes so naturally: since a layer is a Module and your custom network is also a Module, you can nest Modules inside Modules indefinitely to build arbitrarily deep, structured architectures.

A working custom model needs exactly two methods defined, each with a distinct job. __init__(self) is where you instantiate the layers the model will use — self.layer1 = nn.Linear(10, 20) creates a linear layer with 10 input features and 20 output features, and assigns it as an attribute so nn.Module can track its parameters.

forward(self, x) is where you describe how input data actually flows through those layers — it's the function that gets called (indirectly) whenever you run model(x). In the simplest case, forward just calls each layer in sequence and returns the result: return self.layer1(x).

The crucial detail is that __init__ only declares which layers exist — it says nothing about the order or logic of how data passes through them. That's entirely forward's job. Two models could have byte-for-byte identical __init__ methods and completely different behavior if their forward methods route the data differently.

forward(self, x) is the single place where a model's actual computation is defined — every operation the input tensor x goes through, in whatever order you write them, from the first layer to the final output. When you call model(x) in your code, Python's nn.Module.__call__ (inherited automatically) invokes forward(x) for you, along with some bookkeeping like hooks — which is why you almost never call model.forward(x) directly.

It's worth being precise about what forward does not do. It doesn't initialize weights — that happens automatically when each layer (like nn.Linear) is constructed in __init__, using PyTorch's default initialization scheme. It also doesn't run backpropagation; that's Autograd's job, triggered separately by calling .backward() on the loss after forward has already produced a prediction.

forward's only responsibility is the forward pass: take an input, apply the model's layers and logic in order, and return an output. Everything else — initialization, gradient computation, weight updates — is handled by other parts of the PyTorch stack.

Because forward(self, x) is an ordinary Python method, it can contain any Python control flow you'd use anywhere else — if statements, for loops, while loops, even print() calls for debugging. PyTorch doesn't require forward to be a static, declarative description of the architecture; it's executed as real code, once per forward pass.

This has a direct architectural consequence: if x.sum() > 0: x = self.layer2(x) means the model can route data through different layers depending on the actual values in that specific input. Two different inputs to the same model instance can take genuinely different paths through the network.

This is the essence of what's called a 'dynamic computation graph' — the graph Autograd builds is whatever path the code actually took for this particular forward call, rebuilt fresh every time. It's a sharp contrast to frameworks with 'static' graphs, which require the full computation structure to be defined once, ahead of time, before any data flows through it.

The concrete advantage nn.Sequential can't offer is exactly what the previous section demonstrated: the ability to write real Python control flow — if/else branches, loops, conditional logic based on the input's actual values — directly inside the model's computation. nn.Sequential can only express a fixed, linear chain of layers; there's no way to make step 3 conditional on what step 1 produced.

This flexibility is what makes architectures like ResNets (conditionally adding a skip connection), recurrent networks (looping over a sequence with shared weights), and attention mechanisms (combining multiple tensors at specific points) expressible at all. None of them fit a strictly linear pipeline.

The distractor options aren't real properties of this pattern: writing a custom forward doesn't inherently make CPU training 10x faster (the speed depends on the operations themselves, not how they're organized in code), and PyTorch has no built-in mechanism that automatically saves a model to AWS — model persistence and cloud storage are handled by separate, explicit code you write yourself.

Every custom nn.Module subclass so far has started its __init__ with one specific line: super().__init__(). It's easy to treat as boilerplate and skip, but it's doing essential setup work, not a formality — and the next two sections explain exactly what breaks when it's missing.

Python classes that inherit from a parent class don't automatically run the parent's __init__ just because you defined your own __init__ — if your subclass defines __init__, it fully overrides the parent's version unless you explicitly call it. For an ordinary Python class, skipping the parent's __init__ might just mean some parent attributes never get set. For nn.Module specifically, the consequences are more severe.

nn.Module's __init__ sets up internal dictionaries the class uses to track every layer and parameter you assign to self later — the exact mechanism that makes .parameters(), .to(device), and .state_dict() work automatically. Skip that setup, and none of that bookkeeping exists yet when your own __init__ tries to use it.

Skipping super().__init__() causes an immediate, specific failure the moment you try to assign a layer inside your own __init__. nn.Module overrides Python's attribute-assignment behavior (via __setattr__) so that whenever you write self.layer1 = nn.Linear(10, 20), it doesn't just set a normal attribute — it also registers that layer in an internal dictionary of tracked submodules.

That internal dictionary is created inside nn.Module.__init__(). If super().__init__() was never called, the dictionary doesn't exist yet, and the very first layer assignment inside your class's __init__ raises an AttributeError, because nn.Module's custom __setattr__ tries to look up a dictionary that was never created.

This is why the crash happens 'instantly' — it's not a subtle bug that surfaces later during training; it fails the first time you try to construct a layer, which at least makes it easy to diagnose once you know what super().__init__() is actually responsible for.

This exact error message — 'cannot assign module before Module.__init__() call' — is PyTorch's explicit, named check for the missing super().__init__() case described in the last two sections. PyTorch could have let this fail with a generic AttributeError, but it specifically detects the missing setup and raises a clear message pointing at the real cause, because this is one of the most common mistakes beginners make with custom nn.Module classes.

The fix is a single line, placed as the very first statement inside __init__, before any layers are created: super().__init__(). Only after that call has run does self.layer1 = nn.Linear(...) (or any other layer assignment) have a properly initialized tracking dictionary to register itself into.

The distractor answers don't cause this specific error: forgetting to import Pandas is unrelated to nn.Module entirely, and requires_grad=True is a tensor-level setting, not something that affects whether a Module's own initialization succeeded.

Threat neutralized. OOP architecture secured. Proceeding to the Training Loop.

Building a custom PyTorch model comes down to three consistent rules covered in this lesson: inherit from nn.Module so the class gets parameter tracking, device management, and Autograd integration for free; call super().__init__() as the very first line of your own __init__, before assigning any layers; and define forward(self, x) as real Python code describing exactly how data moves through the layers you declared, using ordinary control flow when the architecture needs it.

With a properly structured model now in place — one that can hold layers, accept a batch from a DataLoader, and produce predictions through forward — the remaining piece is turning that into actual learning: running batches through the model, computing a loss, calling backward(), and stepping the optimizer, over and over, across epochs. That end-to-end training loop is what ties everything from this course together.

While nn.Sequential is easy, professional Deep Learning models are complex. They have multiple inputs, skipped connections, and dynamic loops.

To build a custom PyTorch model, you create a Python Class that inherits from nn.Module. This unlocks the full power of the PyTorch engine.

To build a professional, custom Neural Network in PyTorch, your Python class MUST inherit from which parent class?

• →nn.Network
• →nn.Module
• →torch.DeepLearning

Inside your custom class, you must define two functions: __init__ and forward. In __init__, you define the layers. In forward, you define how data moves through them.

In a custom nn.Module class, what is the specific purpose of the forward(self, x) function?

• →It defines the exact mathematical sequence and logic of how the input data 'x' passes through the layers to generate the final prediction.
• →It initializes the weights and biases to zero.
• →It runs the Backpropagation algorithm.

Because forward is just standard Python, you can use if statements, for loops, and print statements directly inside the network architecture.

What is a major advantage of using nn.Module with a custom forward function over nn.Sequential?

• →It trains 10x faster on the CPU.
• →You can use dynamic Python logic (like if/else loops) to route data differently depending on the input values.
• →It automatically saves the model to AWS.

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand class initialization.

If you forget to call super().__init__() inside your custom class, PyTorch will crash instantly when you try to use it.

ADA DEFENSE: You create a perfect custom Neural Network class. However, PyTorch throws an error saying "cannot assign module before Module.__init__() call". What did you forget?

• →You forgot to import Pandas.
• →You forgot to call super().__init__() at the very beginning of your __init__ function to initialize the parent PyTorch class.
• →You forgot to set requires_grad=True.

Threat neutralized. OOP architecture secured. Proceeding to the Training Loop.

Run a Real Forward Pass. Finish linear_layer_forward(): each output neuron computes its own weighted sum plus its own bias.