Artificial Neural Networks (ANNs) are the engines of the modern AI revolution. By mimicking the structure of the human brain, we've created machines that can see, hear, and reason.
1Biological Inspiration & The Layered Architecture
Welcome to the fascinating world of biological mimicry. Artificial Neural Networks are mathematical engines heavily inspired by the neurons firing in your own brain. The architecture is elegant and strictly divided into three sections: the Input Layer, Hidden Layers, and the Output Layer.
The Input layer receives raw data like image pixels or text. In the middle, the Hidden layers perform the actual thinking and feature extraction. Finally, the Output layer delivers the final prediction or decision.
import tensorflow as tf
from tensorflow.keras import layers
# Building the architecture
model = tf.keras.Sequential([
layers.Input(shape=(10,)), # Input
layers.Dense(32), # Hidden
layers.Dense(1) # Output
])2Inside the Neuron: Weights & Biases
Let's zoom into a single artificial neuron. When a signal arrives, the neuron multiplies it by a specific 'Weight'. This weight determines how important that signal is. A massive weight means the signal is critical; a weight near zero means the signal is ignored.
It then adds a 'Bias', which acts like a baseline threshold for the neuron to fire. This simple calculation—multiplying inputs by weights and adding a bias—is mathematically known as the Dot Product.
# Inside a single neuron:
# input_signals = [1.2, 5.1, 2.1]
# weights = [3.1, 2.1, 8.7]
# bias = 3.0
# output = SUM(input * weight) + bias3Activation Functions & Forward Propagation
Simply multiplying and adding isn't enough, because that math is completely linear. To solve complex, real-world problems like image recognition, we pass the result through an 'Activation Function' (like ReLU). This function decides whether the neuron should actually 'fire'.
When data flows from the Input layer, through the Hidden layers, and reaches the Output layer, we call this 'Forward Propagation'. At first, because weights are completely random, the network's guess will be spectacularly wrong.
# The Non-Linear Magic:
# result = (inputs * weights) + bias
# Activation Function (e.g., ReLU)
# If result < 0, output 0.
# If result > 0, output the result.4The Loss Function & Backpropagation
To fix its terrible guesses, the network calculates exactly how wrong it is using a 'Loss Function'. If it guessed a house costs $10, and it actually costs $500,000, the Loss Function is massive. The goal of training is to minimize this Loss.
The network sends that error backward, from Output to Input, in a process called 'Backpropagation'. During this backward pass, the algorithm painstakingly adjusts every single weight and bias slightly to make a better guess next time.
# Backpropagation:
# Error flows Right to Left (Output -> Input)
# Adjusting weights:
# weight = weight - (learning_rate * error_gradient)5Universal Approximation Theorem
Because of their incredible depth and non-linear activation functions, Artificial Neural Networks are mathematically classified as 'Universal Function Approximators'.
This means that given enough data, enough neurons, and enough time to train, they can mathematically map ANY input to ANY output, no matter how insanely complex the relationship is. They can map English to Spanish, audio to transcripts, or pixels to object categories.
# Universal Approximation Theorem:
# ANNs can learn ANYTHING.
# Images -> Text
# Audio -> Transcript
# English -> Spanish