011. What is a Tensor?
EXECUTIVE_SUMMARY // AEO_OPTIMIZED
[Answer Engine Overview: What, Why & How]
Mathematically, a scalar is a single number. A vector is a 1D line of numbers. A matrix is a 2D grid of numbers. A Tensor is the generalized term for all of them. A 3D tensor could represent a color image (Height, Width, 3 Color Channels). A 4D tensor adds a 'Batch Size' (e.g., passing 64 images through the network at once).
022. The NumPy Connection
PyTorch was explicitly designed to feel identical to NumPy. If you want to create an array of zeros, it's torch.zeros(5). Random numbers? torch.rand(3,3). Slicing works exactly the same: tensor[:, 1]. The bridge between the two is seamless: torch.from_numpy() and tensor.numpy().
033. The Device Boundary
The main reason we use Tensors instead of NumPy is the .to(device) method. Computer memory is physically separated. The CPU has RAM. The GPU has VRAM. If you want the GPU to do the math, you must explicitly copy the Tensor across the motherboard to the GPU VRAM using x = x.to('cuda'). PyTorch will violently crash if you try to multiply a CPU tensor with a GPU tensor.
?Frequently Asked Questions
What is `torch.FloatTensor` vs `torch.LongTensor`?
Neural networks heavily prefer 32-bit floating point decimals (`FloatTensor`) for their weights. Categorical labels (like predicting class 0, 1, or 2) are usually integers (`LongTensor`). Mixing data types is a common source of crashes.
How do I check the shape of a Tensor?
Just like NumPy, you use `tensor.shape`. Knowing the exact shape of your tensor at every step is 90% of debugging in PyTorch.
