Introduction to PyTorch in Python

Everything so far in this course — linear regression, decision trees, Random Forests, clustering — has lived inside Scikit-Learn's world of classical statistical learning. This module marks a hard turn into Deep Learning, and PyTorch is the tool that takes you there. Built and open-sourced by Meta (Facebook) AI Research, PyTorch is the framework behind most modern neural network research and a huge share of production AI systems.

The shift isn't just a new library with a similar API. Scikit-Learn models like RandomForestClassifier() are self-contained black boxes: you call .fit() and the algorithm's internal structure is fixed by the library. PyTorch instead gives you the raw building blocks — tensors, layers, gradients — and expects you to assemble the architecture yourself. That flexibility is exactly what deep learning needs, since neural networks come in wildly different shapes depending on the problem: convolutional networks for images, transformers for text, recurrent networks for sequences.

Over this module you'll learn PyTorch's core vocabulary: tensors (the GPU-aware replacement for NumPy arrays), autograd (automatic differentiation), nn.Module (how models are structured), and the training loop that ties them together. By the end you'll understand why PyTorch, rather than Scikit-Learn, is the tool reached for whenever the problem calls for a neural network.

Deep Learning abandons the decision boundaries and split-based logic of Decision Trees and Random Forests. Instead, it builds Artificial Neural Networks (ANNs): layered structures of simple computational units ('neurons') loosely inspired by how biological neurons connect and fire, though the resemblance is really just an analogy — a neural network is, mechanically, a chain of matrix multiplications and non-linear functions.

A basic network has three kinds of layers. The Input Layer receives your raw features (pixel values, word embeddings, tabular columns). One or more Hidden Layers transform that input through learned weights and an activation function, extracting increasingly abstract patterns. The Output Layer produces the final prediction — a class probability, a regression value, or a sequence of tokens. Data flows forward through these layers during inference; during training, error signals flow backward to update the weights.

Unlike a Random Forest, where the 'shape' of the model is fixed by hyperparameters like n_estimators and max_depth, a neural network's shape — how many layers, how wide each one is, how they connect — is something you design explicitly in PyTorch. That's the trade-off: more control, more responsibility for getting the architecture right.

This checkpoint asks you to pin down the real architectural gap between something like Scikit-Learn's RandomForestClassifier and a PyTorch model. A Random Forest is an ensemble of decision trees: the algorithm itself decides how splits happen, and your only real design choices are hyperparameters like the number of trees or their maximum depth. The internal logic is opaque and fixed by the library.

A PyTorch model is different in kind, not just degree. It's exclusively built from stacked, multi-layered Artificial Neural Networks — you choose the number of layers, their widths, their activation functions, and how tensors flow between them, then PyTorch's autograd engine computes the gradients needed to train whatever architecture you assembled. There's no equivalent of RandomForestClassifier() that hands you a working model with one function call; you construct the network yourself, typically by subclassing nn.Module.

That's the correct answer to the checkpoint: PyTorch exclusively relies on multi-layered ANNs, while Scikit-Learn offers a toolbox of distinct, traditional statistical algorithms (trees, SVMs, linear models) that don't share a common neural architecture at all.

PyTorch's first massive advantage over Scikit-Learn is GPU acceleration. Scikit-Learn's algorithms run entirely on the CPU; there's no built-in path to a graphics card. PyTorch, by contrast, lets you move any tensor or model onto an NVIDIA GPU with a single call — model.to('cuda') — and every subsequent matrix multiplication executes on thousands of GPU cores in parallel instead of a handful of CPU cores in sequence.

This matters because neural network training is, at its core, an enormous number of matrix multiplications repeated over many batches and epochs. A GPU is purpose-built for exactly that kind of massively parallel numeric work, which is why the same training job that takes hours on a CPU can finish in minutes on a GPU. For the small tabular datasets Scikit-Learn typically handles, that speedup barely matters; for the million- or billion-parameter models deep learning builds, it's the difference between a feasible experiment and one that never finishes.

Critically, PyTorch code doesn't fork into 'CPU version' and 'GPU version' — the same model and training loop run on either device, and switching is a matter of where your tensors live, not what code you write.

Models like the ones behind ChatGPT have billions of parameters and are trained on datasets far too large to fit any single machine's memory. Training them without hardware acceleration isn't just slow — it's practically impossible within a human timescale. This checkpoint tests whether you understand why GPU support is the deciding factor for that scale of work.

The answer is that PyTorch natively supports CUDA, NVIDIA's parallel computing platform, which lets it dispatch the millions of matrix multiplications and additions inside a forward and backward pass to run simultaneously across thousands of GPU cores. Frameworks or libraries without native GPU support are stuck doing that same math sequentially on a CPU with, at most, a few dozen cores — a difference of orders of magnitude at the scale of a large language model.

This is also why large training runs use multiple GPUs (or even multiple machines, each with several GPUs) working in parallel — PyTorch's distributed training utilities build directly on the same CUDA foundation that makes single-GPU acceleration possible in the first place.

PyTorch's second massive advantage is Autograd, its automatic differentiation engine. Training a neural network means repeatedly asking: 'if I nudge this weight slightly, how does the final loss change?' That question is answered by a derivative — and a real network can have millions of weights, each needing its own derivative, recomputed after every batch of data.

Doing that calculus by hand, layer by layer, is exactly the kind of tedious, error-prone bookkeeping that made training deep networks impractical before automatic differentiation existed. Autograd solves it by recording every tensor operation you perform (multiplications, additions, activation functions) into a dynamic computation graph as your code runs. When you call .backward() on the final loss, PyTorch walks that graph backward, applying the chain rule automatically to compute the gradient of every parameter with respect to the loss.

The practical effect is that you write the forward pass — the ordinary math that turns input into a prediction — and PyTorch derives the backward pass for you. You never hand-write a gradient formula; you just define the computation, and autograd figures out how to differentiate it.

This checkpoint asks directly what Autograd is for. The short answer: it automates calculus. Every parameter in a neural network needs a gradient — the partial derivative of the loss with respect to that parameter — before an optimizer can update it, and computing those derivatives by hand for a network with millions of weights simply isn't feasible.

Autograd removes that burden entirely. As long as your tensors are created with requires_grad=True (the default for a model's learnable parameters), PyTorch tracks every operation performed on them behind the scenes. Calling loss.backward() then walks that recorded graph in reverse, applying the chain rule at each step to populate each parameter's .grad attribute with the exact derivative needed for that training step.

The distractors in this question (grading exams, generating a DataFrame) aren't just wrong, they're a reminder that Autograd has one job: turning a forward computation into the gradients that make backpropagation — and therefore gradient descent — possible.

Before wrapping up this introduction, the lesson stress-tests your understanding of where PyTorch sits relative to its biggest rival, TensorFlow. Both frameworks solve the same underlying problem — defining, training, and deploying neural networks — so the interesting question isn't 'which one works' but 'why did the ML research community converge so heavily on one of them.'

That distinction matters practically, too: the framework you choose affects debugging workflow, how naturally your code reads, and how much friction there is between writing an idea and testing it. Getting this checkpoint right means you understand PyTorch's design philosophy well enough to explain, not just recall, why it won over researchers.

Keep the two frameworks' origins in mind as you answer: TensorFlow was built by Google with production deployment as a first priority, while PyTorch was built by Meta AI with research iteration speed as a first priority. That difference in priorities shaped everything else about how each framework behaves.

TensorFlow is Google's deep learning framework; PyTorch is Meta's. Both let you define layers, run forward passes, and backpropagate gradients — but PyTorch has become the dominant choice in academic and research settings because it is, in the community's own word, 'Pythonic.'

Concretely, that means PyTorch code executes eagerly: each line runs immediately, tensor by tensor, exactly like ordinary Python. Early TensorFlow required you to first build a static computation graph and then run it inside a separate session, which meant standard debugging tools like pdb or a simple print(tensor) often couldn't show you what was actually happening mid-computation. PyTorch's dynamic graph is rebuilt on every forward pass, so you can inspect any intermediate tensor's shape or values the same way you'd inspect any other Python variable.

That difference sounds small, but for researchers iterating on novel architectures — trying an idea, watching it fail, tweaking it, trying again — it removes an entire category of friction. TensorFlow has since added eager execution too, but PyTorch's research-first design gave it a head start that stuck.

This defense checkpoint asks you to pick out the real reason PyTorch overtook TensorFlow in research labs, versus two plausible-sounding but false distractors. Neither GPU support nor TensorFlow being discontinued is the answer — both frameworks support CUDA GPU acceleration, and TensorFlow is very much still maintained and widely used in production today.

The actual answer is PyTorch's dynamic computation graph. Because PyTorch builds its graph on the fly as code executes (define-by-run) rather than requiring you to define a static graph upfront (define-and-run, as early TensorFlow did), it behaves exactly like normal Python: you can set breakpoints inside a forward pass, print tensor shapes mid-computation, and use control flow like if statements and loops that depend on tensor values without any special graph-compilation machinery.

For researchers who spend most of their time experimenting with novel architectures rather than deploying finished ones, that ease of debugging and iteration outweighs almost everything else — which is exactly why PyTorch became the default choice for published papers and academic labs.

This introduction covered the landscape: PyTorch's place relative to Scikit-Learn, its two headline advantages of GPU acceleration and Autograd, and why it out-competed TensorFlow for research use through its dynamic, Pythonic computation graph. That's the conceptual map — but none of it sticks until you write actual PyTorch code.

The next lessons get hands-on with the object every PyTorch program is built from: the tensor. You'll learn how to create tensors, move them between CPU and GPU, reshape them, and perform the operations that Autograd tracks. Tensors are the direct successor to NumPy arrays, sharing most of the same API, but with the two additions that make deep learning possible — GPU residency and gradient tracking.

Everything covered here — why PyTorch exists, what makes it different, why the field adopted it — is the context you'll keep coming back to as the mechanics get more concrete starting with the next lesson on Tensors.

Module 04: Welcome to Deep Learning. Scikit-Learn was for traditional statistics. PyTorch is the engine of modern AI, built by Meta (Facebook).

Deep Learning abandons Decision Trees and Lines. Instead, it uses Artificial Neural Networks (ANNs) inspired by the human brain.

What is the primary architectural difference between Scikit-Learn algorithms (like Random Forest) and PyTorch models?

• →PyTorch exclusively relies on multi-layered Artificial Neural Networks, whereas Scikit-Learn uses a variety of traditional statistical algorithms.
• →PyTorch uses SQL databases natively.
• →Scikit-Learn uses Neural Networks, but PyTorch uses Trees.

PyTorch has two massive advantages over Scikit-Learn. First: GPU Acceleration. PyTorch can run math on your graphics card, making it 100x faster for massive matrices.

Why do Deep Learning practitioners strongly prefer PyTorch over traditional Python libraries for training massive models like ChatGPT?

• →Because PyTorch natively supports GPU hardware acceleration (CUDA), which allows for parallel processing of millions of mathematical operations simultaneously.
• →Because PyTorch is written in HTML.
• →Because PyTorch automatically cleans missing data in Pandas.

The second massive advantage is Autograd (Automatic Differentiation). PyTorch automatically calculates the complex calculus derivatives needed to train Neural Networks.

What is the purpose of PyTorch's "Autograd" engine?

• →It automatically grades student exams.
• →It automatically calculates derivatives and gradients, completely automating the brutal calculus required for Backpropagation.
• →It automatically generates a Pandas DataFrame.

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand the difference between TensorFlow and PyTorch.

TensorFlow is Google's library. PyTorch is Meta's library. Both do the same thing, but PyTorch dominates academic research because it is "Pythonic".

ADA DEFENSE: Why did PyTorch overtake TensorFlow as the dominant framework in AI research labs worldwide?

• →Because PyTorch feels exactly like writing standard Python (Dynamic Computation Graphs), making it much easier to debug and experiment with.
• →Because PyTorch is the only one that supports GPUs.
• →Because TensorFlow was discontinued in 2015.

Threat neutralized. Landscape understood. Proceeding to Tensor mechanics.

Compute a Real Neuron's Output. Finish neuron_output(): every neuron computes a weighted sum of its inputs plus a bias.