Introduction to AI and ML in Python

Machine Learning flips the direction of traditional software development. Instead of a programmer encoding explicit rules ('if income > X and credit score > Y, approve loan'), you feed an algorithm a large set of examples together with the correct answers, and it works backward to discover the pattern connecting them. The 'model' that comes out the other end is just a set of learned parameters — weights, thresholds, coefficients — tuned until its predictions match the training examples closely enough to generalize to new data.

This matters because it changes where the engineering effort goes. In traditional programming you spend your time writing and debugging logic; in ML you spend it curating data, choosing an appropriate algorithm and set of features, and validating that what the model learned actually reflects the real-world pattern rather than noise or artifacts in your dataset. The rest of this course builds on that shift: scikit-learn handles the classical statistical algorithms (regressions, trees, clustering), and PyTorch handles the neural-network side of the field once your data and problem call for it.

In a traditional program, the logic is the input: you write if x > 10: return 'high' and the computer executes exactly that branch, every time, deterministically. Machine Learning inverts this — you supply the inputs (features) and the desired outputs (labels), and the training process searches for a function that maps one to the other as accurately as possible. A spam filter built the traditional way needs a human to keep writing new keyword rules forever; a spam filter built with ML looks at thousands of labeled emails and statistically learns which word patterns, sender behaviors, and structural cues correlate with 'spam'.

This 'answers + data -> rules' framing is why the quality and quantity of your training data matters more in ML than in traditional programming. A bug in traditional code is usually a logic error you can trace line by line. A 'bug' in an ML model is far more often a data problem — biased sampling, mislabeled examples, or features that leak information the model shouldn't have at prediction time — and those are much harder to spot by reading code alone.

This checkpoint exercise tests whether you can articulate the paradigm shift in your own words, not just recognize it. The correct framing is that in Machine Learning the computer generates the rules by analyzing data and known answers, rather than a human writing those rules by hand — the wrong options either confuse ML with a specific language (it isn't tied to Python or HTML) or confuse it with rote memorization (a model that only memorizes has failed to learn a useful pattern, a problem you'll revisit shortly as 'overfitting').

Getting this distinction solid now pays off later: every time you evaluate a scikit-learn or PyTorch model, you're really asking 'did it learn the underlying rule, or did it just memorize the examples it saw?' That question is the throughline connecting supervised learning, train/test splits, and overfitting — all covered next.

Supervised learning is the branch you'll use most in this course: every training example comes with a known correct answer (a label), and the algorithm's job is to learn the mapping from input to that label — classifying emails as spam/not-spam, or predicting a continuous number like a house price. Unsupervised learning removes the labels entirely; instead of predicting a known answer, algorithms like k-means clustering or PCA look for structure — natural groupings, dimensions of variance — hidden inside unlabeled data, which is useful for tasks like customer segmentation where you don't know the 'right' groups in advance.

Reinforcement learning is a different paradigm again: there's no fixed dataset of correct answers at all. An agent takes actions in an environment, receives a reward or penalty, and gradually learns a policy that maximizes cumulative reward through trial and error — the approach behind game-playing AI and robotics. This course focuses on supervised and unsupervised learning with scikit-learn and neural networks with PyTorch; reinforcement learning is a related but separate specialty.

The house-price example is a textbook supervised learning problem: each row of your dataset (a house's square footage, location, number of bedrooms) is paired with a known label — the historical sale price. Because every training example already has its 'correct answer' attached, the model can directly measure how wrong its predictions are during training and adjust itself accordingly. This is what makes it supervised: you are supervising the learning process by grading its output against ground truth.

Contrast this with an unsupervised approach to the same raw data — clustering houses by similarity without ever looking at price — which would group similar properties together but couldn't tell you a dollar figure, because there's no label to regress toward. Recognizing whether your target variable is present in the data is usually the fastest way to decide which branch of ML a problem falls into.

Deep Learning is not a separate field from Machine Learning — it's a specific technique within it, built on Artificial Neural Networks: layered structures of simple mathematical units ('neurons') that each apply a weighted sum and a nonlinearity, then pass their output to the next layer. Stacking many such layers ('deep' networks) lets the model learn increasingly abstract representations of raw input — early layers in an image classifier might detect edges, later layers might detect shapes, and the final layers detect whole objects.

The practical dividing line in this course is tooling: scikit-learn implements classical ML algorithms (linear/logistic regression, decision trees, k-means) that work well on structured, tabular data with modest dataset sizes and don't require a GPU. PyTorch implements neural networks and the automatic differentiation machinery needed to train them, and is the tool of choice once you're working with unstructured data (images, text, audio) or need the representational power of deep architectures — including the large language models behind tools like ChatGPT.

The distinguishing feature is architecture, not just scale: classical algorithms in scikit-learn — linear regression, decision trees, random forests, SVMs — are built from explicit statistical or geometric rules (minimize squared error, split on the feature that best reduces impurity, maximize the margin between classes). Deep Learning, in contrast, relies exclusively on stacked layers of artificial neurons whose weights are learned via backpropagation and gradient descent, with no hand-designed rule for what each layer should compute.

This has practical consequences you'll feel throughout the course: scikit-learn models tend to need less data, train in seconds on a CPU, and are easier to interpret (you can often inspect a decision tree's splits directly). PyTorch models typically need much more data and benefit heavily from GPU acceleration, but can automatically learn feature representations from raw, unstructured input that you'd otherwise have to hand-engineer for a classical model.

Splitting data into a training set and a testing set exists to answer one question honestly: did the model learn a generalizable pattern, or did it just memorize the specific examples it was shown? If you train and evaluate on the exact same data, a model can score perfectly by essentially looking up answers it already 'saw' during training — that score tells you nothing about how it will perform on new, unseen inputs, which is the only thing that matters in production.

The standard practice, and what you'll do constantly with scikit-learn's train_test_split, is to hold out a portion of your labeled data — commonly 20-30% — before training even starts, and never let the model see it until final evaluation. Violating this rule, even accidentally (for example, by scaling your features using statistics computed from the full dataset before splitting), is called data leakage, and it's one of the most common and hardest-to-detect bugs in real ML pipelines.

Overfitting is the single most common failure mode you'll encounter once you start training real models, which is why it gets its own checkpoint here before you touch scikit-learn or PyTorch code. It happens when a model's capacity is high enough — relative to the amount and diversity of training data — that instead of learning the general pattern connecting inputs to outputs, it starts memorizing the noise and idiosyncrasies specific to the training examples themselves.

The telltale sign is a widening gap between training performance and test performance: accuracy that looks excellent on data the model has already seen, but drops sharply on anything new. Keep that signature in mind for the exercise that follows — it's exactly the pattern you're being asked to diagnose.

A model that scores 100% on the training set but only 30% on the testing set is showing the classic signature of overfitting: it didn't learn 'what makes a dog a dog' in any general sense — it effectively memorized the pixel patterns of the specific training images, including noise and irrelevant details unique to those exact photos. When it's shown new dogs it has never seen, none of that memorized detail transfers, so its accuracy collapses.

The fix is not more training on the same data (that would make overfitting worse) — it's techniques that force the model to generalize: gathering more diverse training examples, regularization, simplifying the model, or stopping training earlier. You'll apply several of these directly once you start building models with scikit-learn's estimators and PyTorch's neural networks later in this course.

You now have the vocabulary the rest of this course assumes: the shift from writing rules to learning them from data, the three branches of ML (supervised, unsupervised, reinforcement), how Deep Learning relates to classical ML through neural network architecture, why train/test splits exist, and what overfitting looks like when a model fails to generalize. None of these are abstract trivia — they're the lens you'll use to debug every model you train from here on.

From this point forward the course gets hands-on: the next modules move into scikit-learn for classical supervised and unsupervised algorithms, followed by PyTorch for building and training neural networks from scratch, including the autograd engine that makes backpropagation possible.

Welcome to the Machine Learning pipeline. You have mastered data with Pandas and math with SciPy. Now, we teach computers how to learn from that data.

In traditional programming, you write the logic. In Machine Learning, you provide the data and the expected outcome, and the algorithm figures out the logic itself.

What is the fundamental difference between Traditional Programming and Machine Learning?

• →In ML, the computer generates the rules by analyzing the data and the answers, rather than a human writing the rules manually.
• →Traditional programming uses Python, while ML uses HTML.
• →Machine Learning is just a database that memorizes everything.

Machine Learning is divided into three main branches: Supervised Learning, Unsupervised Learning, and Reinforcement Learning.

If you want to train an algorithm to predict house prices, and you provide a dataset where every house already has its historical sale price attached, which branch of ML is this?

• →Supervised Learning (because the data has 'labels' or known answers).
• →Unsupervised Learning (because you want it to learn on its own).
• →Reinforcement Learning.

Deep Learning is a specialized sub-branch of Machine Learning that uses Artificial Neural Networks to simulate how the human brain processes information.

What is the primary difference between general Machine Learning (like Scikit-Learn) and Deep Learning (like PyTorch)?

• →Deep Learning exclusively uses multi-layered Artificial Neural Networks, whereas general ML uses statistical algorithms like Trees or Regressions.
• →Deep Learning is just Machine Learning on a faster computer.
• →Deep Learning requires the data to be in SQL.

To train any ML model, you must split your data into a "Training Set" (to teach the model) and a "Testing Set" (to see if it actually learned, or just memorized).

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand the concept of "Overfitting".

ADA DEFENSE: You train an AI to recognize dogs. It gets 100% accuracy on the Training Set, but when you show it new dogs (the Testing Set), it gets 30% accuracy. What happened?

• →Overfitting. The model memorized the exact pixels of the training dogs instead of learning the abstract concept of a dog.
• →Underfitting. The model needs more epochs to learn.
• →Hardware failure. The GPU overheated during testing.

Threat neutralized. Concept validated. Welcome to the world of Artificial Intelligence.

Classify a Real Learning Type. Finish classify_learning_type(): supervised learning always includes labeled answers.