Scikit-Learn Basics in Python

Scikit-learn is the library that turned classical Machine Learning in Python from a scattered collection of academic implementations into a single, consistent, production-ready toolkit. It wraps decades of statistical learning research — linear models, decision trees, support vector machines, clustering algorithms — behind one uniform interface, so switching from a RandomForestClassifier to a LogisticRegression is a one-line change rather than a rewrite.

It's built directly on top of NumPy, SciPy, and matplotlib, which is why everything you learned about ndarrays and vectorized operations transfers immediately: scikit-learn estimators expect NumPy arrays (or pandas DataFrames) as input and return NumPy arrays as output. That shared foundation is also what makes it easy to move data between scikit-learn, PyTorch, and the rest of the Python data stack without conversion headaches.

Every estimator in scikit-learn — whether it's a linear regression, a decision tree, or a k-means clusterer — follows the exact same three-step contract: instantiate the class (model = Algorithm()), call .fit(X, y) to train it on labeled data, and call .predict(X_new) to generate predictions on new data. This consistency is deliberate design, not coincidence — it's what lets you swap one algorithm for another with almost no code changes while you experiment.

Under the hood, .fit() is where all the actual math happens: computing coefficients for a regression, choosing split points for a tree, or finding cluster centroids. .predict() never re-learns anything — it simply applies the parameters that .fit() already computed to new input. Understanding that split (learning happens in fit, application happens in predict) is the foundation for everything else in this module.

The three-step workflow — instantiate, fit, predict — isn't just a convenience for beginners; it's the contract that every scikit-learn estimator promises to honor, no matter how different the underlying algorithm is. A RandomForestClassifier and a LogisticRegression compute wildly different things internally, but both expose exactly the same three calls, which is why swapping one for another during experimentation is a one-line change rather than a rewrite.

This uniformity is scikit-learn's actual design philosophy, not an accident. Every estimator class follows the same constructor conventions (hyperparameters as keyword arguments), the same .fit(X, y) signature, and the same .predict(X) signature. Once you internalize this pattern, reading unfamiliar scikit-learn code — or documentation for an algorithm you've never used — becomes far faster because you already know the shape of the API.

Getting this sequence backwards is one of the most common beginner errors: calling .predict() before .fit() raises a NotFittedError, because the model object has no learned parameters yet. The three steps are strictly ordered, and scikit-learn will refuse to skip ahead.

Scikit-learn enforces a single, unbending convention across every estimator: X is always a 2D array-like object of shape (n_samples, n_features), and y is a 1D array-like object of shape (n_samples,) holding the target you want to predict. Even when you only have one feature, X still needs to stay 2D — a common beginner error is passing a flat 1D array of features and hitting a ValueError: Expected 2D array, got 1D array instead.

The reasoning behind the shape requirement is that scikit-learn is designed around tabular data: each row of X is one observation (a house, a patient, a transaction), and each column is one measured feature of that observation (square footage, blood pressure, transaction amount). y lines up row-for-row with X, so y[i] is the correct label for the observation described by X[i].

This convention is what lets model.fit(X, y) work identically whether X came from a NumPy array, a pandas DataFrame, or a SciPy sparse matrix — scikit-learn only cares about the shape and dtype, not the container. That's also why so much of the preprocessing work in a real pipeline (encoding categoricals, scaling numeric columns) is really just reshaping raw data into a valid X.

It's worth internalizing the X/y naming convention precisely because it's not scikit-learn-specific — it comes from the classic statistical notation for a supervised learning problem, where you're trying to learn a function f such that f(X) ≈ y. You'll see the exact same convention in TensorFlow, PyTorch's dataset APIs, and most ML textbooks, so getting comfortable with it here pays off across the entire ecosystem.

In practice, X is built by selecting your predictor columns from a DataFrame (X = df.drop(columns=['price'])) and y is the single column you're trying to predict (y = df['price']). Keeping this separation explicit — rather than passing the whole DataFrame into fit() — is what prevents the target column from accidentally leaking into the features, which would let the model 'cheat' by seeing the answer during training.

A subtle but important detail: y can be continuous (regression, e.g. predicting a price) or categorical (classification, e.g. predicting a class label). Scikit-learn doesn't ask you to declare which one upfront — it infers the task from the estimator class you chose (LinearRegression versus LogisticRegression), not from the shape of y itself.

train_test_split exists to answer a question you can't answer any other way: does this model actually generalize, or did it just memorize the training examples? By randomly withholding a slice of the data — commonly 20-30% — before training even starts, you create a held-out test set the model never sees during .fit(), so evaluating on it afterward is a genuine test of generalization rather than a rehearsed answer.

The split has to happen before any fitting, including preprocessing steps like scaling. If you fit a StandardScaler on the entire dataset and then split, statistics from the test set (its mean and variance) have already leaked into the transformation applied to the training set — a subtle form of data leakage that inflates your reported accuracy without you realizing it.

train_test_split also accepts a random_state parameter, which is worth setting explicitly in any real project: without it, every run produces a different random split, making your results impossible to reproduce or compare across experiments. For classification problems with imbalanced classes, the stratify=y argument keeps the class proportions consistent between the train and test sets.

The core failure mode train_test_split protects against is overfitting: a model with enough capacity (a deep decision tree, a high-degree polynomial regression) can achieve near-perfect accuracy on data it was trained on simply by memorizing noise, while performing poorly on anything new. Without a held-out test set, that memorization is invisible — the training accuracy looks great, and you'd ship a model that fails in production.

A useful mental model: training accuracy tells you how well the model fits what it has already seen; test accuracy tells you how well it's likely to perform on data it hasn't seen, which is the only number that actually matters once the model is deployed. A large gap between the two — high training accuracy, much lower test accuracy — is the textbook signature of overfitting.

In projects with enough data, teams typically go a step further and carve out a third split (validation) used for tuning hyperparameters, keeping the test set completely untouched until the very final evaluation. That way, even hyperparameter choices — which are themselves a form of fitting to data — can't leak information from the test set.

Before diving into what .fit() actually returns and mutates, it's worth pausing on a distinction that trips up a lot of newcomers coming from a purely functional programming background: scikit-learn estimators are stateful objects, not pure functions. Calling .fit(X, y) doesn't just compute an output — it permanently changes the internal state of the model object you called it on.

That internal state is what scikit-learn calls 'fitted attributes,' conventionally named with a trailing underscore — things like model.coef_ for a linear model's learned weights, or model.feature_importances_ for a tree-based model. Before .fit() is called, these attributes don't exist at all; trying to access them raises an AttributeError, and trying to call .predict() raises a NotFittedError.

Understanding this state transition — unfitted object, to .fit() call, to fitted object with learned parameters — is the foundation for everything that comes next, including why you must never call .fit() again on the same object with test data, and why cross-validation clones a fresh, unfitted estimator for every fold.

.fit() is where all of scikit-learn's actual mathematics happens. For LinearRegression, that means solving for the coefficients that minimize squared error. For a DecisionTreeClassifier, it means recursively choosing the feature and threshold that best splits the data at each node. The algorithms are completely different under the hood, but the calling convention — model.fit(X, y) — never changes.

Crucially, .fit() mutates the estimator object in place rather than returning a new one. model.fit(X, y) does return a value, but that value is just self — a reference back to the same object, now populated with learned attributes — which is why method chaining like predictions = LinearRegression().fit(X, y).predict(X_new) works.

This in-place mutation has a practical consequence: if you call .fit() a second time on the same object with different data, the previous learned parameters are discarded and overwritten, not accumulated. There's no 'incremental training' by default — for that you need estimators that explicitly support .partial_fit(), like SGDClassifier.

The exact return value of .fit() is a detail that matters more than it looks. Because .fit() returns self rather than a prediction or a metric, a line like model = LinearRegression().fit(X_train, y_train) works in a single expression — you're not discarding a meaningful return value, you're just capturing the same object with its new internal state attached.

This matters for pipelines. A Pipeline object internally calls .fit() on each of its steps in sequence, relying on the fact that each step's .fit() returns the fitted object itself so the pipeline can move on to calling .transform() or .predict() on it. If .fit() returned predictions instead of self, the entire Pipeline abstraction would need to work completely differently.

A common beginner assumption is that .fit() must return something like accuracy or loss, the way training loops in other frameworks sometimes print a running loss value. Scikit-learn deliberately keeps .fit() silent — no printed metrics, no returned score — because evaluating a model is a separate, explicit step you perform afterward with .score(), .predict(), or a metric function from sklearn.metrics.

With the fit/predict contract and the X/y convention in hand, you now have the two pieces of vocabulary that every other scikit-learn topic builds on. Whether the next module covers linear models, tree ensembles, or unsupervised clustering, the calling pattern won't change — only the algorithm computing what happens inside .fit() will.

The train/test split habit is equally foundational: from here on, treating the test set as strictly off-limits until final evaluation should become automatic, the same way you'd never merge directly to a production branch without review. Skipping it doesn't just risk a wrong number — it risks shipping a model that looks great in development and fails silently on real-world data.

From this point, the course moves into concrete algorithms: how LinearRegression and LogisticRegression actually compute their coefficients, how tree-based models split data, and how to evaluate each one with the right metric for the task. The API shape you just learned is the constant thread running through all of it.

Module 01: Scikit-Learn Basics. Scikit-Learn (sklearn) is the undisputed industry standard library for traditional Machine Learning in Python.

Every model in Scikit-Learn follows the exact same three-step API: Initialize the model, Fit the data, Predict the outcome.

What is the standard three-step workflow for virtually every algorithm in Scikit-Learn?

• →Initialize -> Fit -> Predict
• →Download -> Compile -> Execute
• →Train -> Test -> Delete

In Scikit-Learn, data is always separated into X (the features/inputs) and y (the target/labels).

When calling model.fit(X, y), what do the variables X and y conventionally represent in Machine Learning?

• →X represents the target labels (the answers), and y represents the input features (the data).
• →X represents the input features (the data), and y represents the target labels (the answers).
• →X and y are just random variables with no specific meaning.

Before fitting a model, we must split our dataset. We use train_test_split to randomly reserve e.g. 20% of our data for testing later.

Why do we use train_test_split to hold back a portion of our data before training?

• →Because the algorithm cannot handle 100% of the data.
• →To evaluate the model's accuracy on unseen data, proving it didn't just memorize the training set.
• →To make the training process take longer.

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand the state of the model after fit() is called.

The fit() method performs all the heavy mathematical calculus. It modifies the internal state of the model object itself.

ADA DEFENSE: When you run model.fit(X_train, y_train), what exactly is the method returning?

• →It returns a Pandas DataFrame with the predictions.
• →It modifies the model object in-place, updating its internal mathematical weights. It returns a reference to the updated self.
• →It returns an integer representing accuracy.

Threat neutralized. Model architecture understood. Welcome to the Scikit-Learn ecosystem.

Run a Real Initialize-Fit-Predict Cycle. Finish full_workflow(): every Scikit-Learn model follows the same three steps.