NumPy Linear Algebra: The Math Behind AI
Pascual Vila
AI & Data Science Architect
"Under the hood, deep learning is just a series of matrix multiplications. If you master NumPy's linear algebra capabilities, you master the foundation of Artificial Intelligence."
Vectors and Dot Products
A vector is a one-dimensional array representing data features. In machine learning, a prediction $y$ is often calculated as the dot product of a weight vector $W$ and an input vector $x$.
In NumPy, we use np.dot(). This operation takes two arrays of equal length, multiplies their corresponding elements, and sums them into a single scalar value.
Matrix Multiplication
When passing hundreds of inputs through a neural network layer, we use matrices. A matrix is a 2D array. To multiply matrices $A$ and $B$, the number of columns in $A$ must equal the number of rows in $B$.
In Python 3.5+, the @ operator was introduced specifically for matrix multiplication, replacing the more verbose np.matmul().
Inverse & Determinants
Solving a linear system like $A \cdot x = b$ is a classic data science problem. If matrix $A$ represents your features and $b$ represents your targets, you can find the optimal weights $x$ by computing the inverse of $A$ ($A^-1$).
You can calculate this using np.linalg.inv(A). Note that a matrix must be square (e.g., $3 \times 3$) and have a non-zero determinant (computed via np.linalg.det()) to be invertible.
View Performance Benchmarks+
Why not use standard Python lists? Python lists are heavily unoptimized for math. A simple matrix multiplication of two $1000 \times 1000$ matrices using nested for loops in pure Python can take over 10 minutes. Because NumPy offloads calculations to underlying C/C++ BLAS libraries, the exact same operation using A @ B takes a fraction of a second.
❓ Algorithm FAQ
Difference between np.dot() and * ?
The * operator performs element-wise multiplication. If you do `[1, 2] * [3, 4]`, you get `[3, 8]`.np.dot() or @ performs true algebraic dot product/matrix multiplication, resulting in a scalar (for 1D arrays) or a transformed matrix.
What does "LinAlgError: Singular matrix" mean?
It means you are trying to compute the inverse of a matrix using `np.linalg.inv()`, but the matrix's determinant is exactly 0. In linear algebra, singular matrices do not have an inverse.