Linear Algebra is the engine of AI. Neural Networks are essentially giant matrix multiplications. NumPy provides a specialized `linalg` module that makes this complex math incredibly fast and accessible.
1Vectors and Dot Products
A 1D NumPy array is a vector. The dot product—the sum of the products of corresponding entries—is a foundational operation for calculating network weights and similarity scores between datasets.
2Matrix Multiplication
Unlike element-wise math, matrix multiplication requires strict dimension alignment. NumPy uses the @ operator (or np.matmul()) to perform these operations, which are the backbone of almost all modern AI algorithms.
3Step-by-Step Breakdown
Linear Algebra is the engine of AI. Neural Networks are essentially giant matrix multiplications. NumPy makes this math incredibly fast.
Let's start with Vectors. A 1D NumPy array is a vector. We can compute the dot product—the sum of the products of corresponding entries.
The result is a scalar value (1*4 + 2*5 + 3*6 = 32). This operation is foundational for calculating network weights.
Checkpoint: What is the resulting data type of a dot product between two 1D arrays?
Now for Matrices (2D arrays). Matrix multiplication is not element-wise. We use the '@' operator or np.matmul() to multiply them.
Notice that for A @ B to work, the inner dimensions must match! A (m x n) * B (n x p) = C (m x p).
Checkpoint: If Matrix A is shape (3, 2) and Matrix B is shape (2, 4), what is the shape of A @ B?
Finally, inverse matrices. Solving linear equations like Ax = b requires finding the inverse of A. We use the np.linalg module for this.
Log in below to unlock advanced AI tasks and write your own linear algebra operations!
Compute a Real Dot Product. Finish computing the dot product of the two vectors and confirm the result.
Level Up 🚀
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported.
Fully supported.
Fully supported.
Fully supported.
Accessibility (A11y)
1Describe Matrix Shapes Verbally in Explanations
A statement like 'A(m x n) @ B(n x p) = C(m x p)' is precise but relies on visually scanning notation — spell out the concrete meaning in prose too ('a 3-row, 2-column matrix multiplied by a 2-row, 4-column matrix produces a 3-row, 4-column result'), so screen reader users get the same understanding without parsing symbolic shorthand.
// (3, 2) @ (2, 4) = (3, 4): 3 rows in, 4 columns outSEO Implications
- 1
Linear Algebra Results Are Computational, Not Page Content
A dot product or an inverted matrix is a value inside a running script, never a rendered page — the SEO value of this page rests entirely on its own written explanation of vector/matrix operations, distinct from any output a reader might generate by running the example code themselves.
Best Practices
Avoid Explicitly Computing a Matrix Inverse When You Just Need to Solve Ax=b
Computing inv(A) and then multiplying by b is numerically less stable and slower than calling np.linalg.solve(A, b) directly, which uses a more efficient and accurate factorization method under the hood. Reserve explicit inversion for cases where you genuinely need the inverse matrix itself.
Check a Matrix's Determinant Before Attempting to Invert It
A matrix with a determinant of (or very close to) zero is singular or near-singular and cannot be reliably inverted — np.linalg.inv() may raise an error or return a numerically garbage result. Check np.linalg.det(A) first, or use np.linalg.pinv() (pseudo-inverse) for a more robust fallback.
Frequent Bugs
Using the * operator expecting matrix multiplication, when it actually performs element-wise multiplication on NumPy arrays.
A * B on two NumPy arrays multiplies corresponding elements together (Hadamard product), not true matrix multiplication — that requires the @ operator or np.matmul(). Using * where @ was intended silently produces a result of the same shape with completely wrong values, no error raised.
Real-World Examples
Computing a Neural Network Layer's Forward Pass
A single dense neural network layer's forward pass is literally one matrix multiplication plus a bias vector: output = inputs @ weights + bias, where inputs has shape (batch_size, features) and weights has shape (features, neurons) — this exact @ operation, repeated across layers, is what 'training a neural network' mechanically consists of.
inputs = np.random.rand(32, 784) # batch of 32, 784 features each
weights = np.random.rand(784, 128) # 128 neurons
bias = np.zeros(128)
output = inputs @ weights + bias # shape: (32, 128)