Listen up. If you're going to process data in Python, you need to understand Cleaning Empty Cells in Python. This is where data engineers separate themselves from script kiddies. It's about writing code that scales.
1Detecting and Removing Missing Values
Real-world datasets are rarely complete. A sensor drops a reading, a survey respondent skips a question, a join produces a row with no match ā and Pandas represents all of these gaps the same way: as NaN (Not a Number), a special floating-point value from NumPy. Before you can clean a dataset you have to find the NaNs, and df.isna() (or the older alias df.isnull()) returns a same-shaped DataFrame of booleans marking exactly where they are, which you can sum per column with df.isna().sum() to get a quick missing-value report.
The blunt tool for dealing with NaN is dropna(). Called with no arguments it drops any row that contains at least one missing value, which is fast but can throw away good data sitting next to a single empty cell. That's why dropna() accepts subset=['Age'] to restrict the check to specific columns, and how='all' to only drop a row when every value in it is missing. Like most Pandas mutation methods, dropna() returns a new DataFrame by default and leaves df untouched ā you either reassign the result (df = df.dropna()) or pass inplace=True to modify the original object directly.
Dropping data isn't always acceptable, especially when every row is expensive to collect. fillna() is the alternative: it replaces NaN with a value you choose, whether that's a constant, the column's mean(), median(), or the previous/next valid value via method='ffill' / method='bfill'. Which strategy is correct depends entirely on the column ā filling a missing age with the column average is reasonable, filling a missing customer ID with the average is nonsense ā so cleaning empty cells is really a per-column judgment call, not a single blanket operation.
# Example
import pandas as pd
print("Running Pandas...")Data processed and aggregated.
2Step-by-Step Breakdown
Empty cells in Pandas are represented as NaN (Not a Number). The fastest way to deal with them is to simply delete any row that contains a NaN value.
Which Pandas method is used to instantly remove all rows that contain at least one missing (NaN) value?
- ādelete_nan()
- āremove_empty()
- ādropna()
By default, dropna() returns a NEW DataFrame and leaves the original untouched. If you want to change the original DataFrame, you must use the inplace=True argument.
If you want dropna() to alter the existing DataFrame rather than returning a new copy, which argument must you pass?
- āmodify=True
- āinplace=True
- āoverwrite=True
If you cannot afford to delete rows, you can replace the NaN values with a specific number using fillna(). Often, we replace it with the mean (average) of the column.
Which method allows you to replace NaN values with a designated replacement value?
- āfillna()
- āreplace_nan()
- āimpute()
Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand how to target specific columns.
ADA DEFENSE: You want to drop rows that have NaN values ONLY in the "Age" column, but keep rows if they have NaN values in other columns. How do you do this?
- ādf['Age'].dropna()
- ādf.dropna(subset=['Age'])
- ādf.dropna(column='Age')
Threat neutralized. NaN values purged from the system. Your dataset is structurally sound.
Threat neutralized. Concept validated. Proceed to the next section.
Drop Real Missing Rows. Finish drop_missing(): remove every row containing at least one NaN.
Level Up š
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported.
Fully supported.
Fully supported.
Fully supported.
Accessibility (A11y)
1Document Missing-Data Decisions
When you drop or impute rows, leave a comment or log entry explaining the strategy ā future readers of a notebook or report need to know a mean-imputed column isn't raw observed data.
# Note: 'Age' imputed with column mean (n=12 missing)
df['Age'].fillna(df['Age'].mean(), inplace=True)SEO Implications
- 1
High Search Volume for Data-Cleaning Basics
"pandas dropna vs fillna" and "handle missing values pandas" are among the most searched beginner data-science queries, since nearly every real dataset requires this step before analysis.
Best Practices
Inspect Before You Drop
Run df.isna().sum() before calling dropna() so you know how much data you're about to lose ā dropping rows blindly on a column with 40% missing values can gut your dataset.
Impute Per-Column, Not Globally
Choose a fill strategy for each column individually (mean for a continuous measurement, a sentinel value or mode for a category) rather than calling fillna() once for the whole DataFrame.
Frequent Bugs
Calling dropna() or fillna() without inplace=True or reassignment, then acting on a DataFrame that still contains the original NaN values.
Either reassign the result (df = df.dropna()) or pass inplace=True, and always print df.isna().sum() afterward to confirm the cleanup actually happened.
Real-World Examples
Cleaning a Survey Export Before Analysis
A CSV export from a survey tool has blank cells wherever a respondent skipped an optional question, and downstream aggregate functions like mean() are returning NaN for entire columns.
# Diagnose
print(df.isna().sum())
# Drop rows missing required fields, impute optional ones
df.dropna(subset=['respondent_id'], inplace=True)
df['age'].fillna(df['age'].median(), inplace=True)