Network Pruning in AI & Artificial Intelligence

The most common technique is Magnitude-Based Pruning. It assumes that weights with small absolute values (close to zero) contribute the least to the model's final prediction. By setting these weights to zero, we create a Sparse Weight Matrix. While the number of parameters remains the same, the sparsity allows for significantly better compression (e.g., using Gzip or specialized hardware kernels) and reduces the total amount of data that needs to be moved between memory and the processor.

Pruning isn't a one-step process. If you remove 50% of a model's weights instantly, its accuracy will likely crash. The industry-standard workflow is the Prune-and-Fine-tune Cycle: you gradually increase the sparsity during training (using a Sparsity Schedule). This allows the remaining 'Active' weights to adapt and take over the features previously handled by the removed connections, effectively 'concentrating' the intelligence into a smaller subset of the network.

Pruning can be Unstructured (removing individual weights anywhere) or Structured (removing entire neurons, channels, or layers). Unstructured pruning leads to the highest sparsity but requires specialized software/hardware to see a speedup. Structured pruning directly reduces the dimensions of the tensors, meaning the model becomes physically smaller and runs faster on any standard CPU or GPU without needing special sparse-math support.