Handling Missing Data in Python DataFrames | Lecture 4
Handling Missing Data in Python DataFrames
Missing
data is a common issue in data analysis, and Pandas provides several ways to
handle it. Here's an overview with examples:
1. Identifying Missing Data
Missing
data in a DataFrame is often represented as NaN (Not a
Number). You can use these methods to detect them:
- isnull(): Returns True for missing values.
- notnull(): Returns False for missing values.
Example:
import pandas as pd
import numpy as np
# Creating a DataFrame with missing values
data = {
"Name": ["Alice", "Bob",
"Charlie", None],
"Age": [25, np.nan, 30, 22],
"Score": [85, 90, np.nan, 88]
}
df = pd.DataFrame(data)
# Detect missing values
print(df.isnull())
# Check for non-missing values
print(df.notnull())
2. Dropping Missing Data
- dropna(): Removes rows or columns
with missing values.
Example:
# Drop rows with any missing values
df_dropped_rows = df.dropna()
# Drop columns with any missing values
df_dropped_columns = df.dropna(axis=1)
print(df_dropped_rows)
print(df_dropped_columns)
3. Filling Missing Data
- fillna(): Replaces missing values
with a specified value or strategy.
Example:
# Fill missing values with a fixed value
df_filled_fixed = df.fillna(0)
# Fill missing values with the column mean
df["Age"] =
df["Age"].fillna(df["Age"].mean())
print(df)
4. Interpolating Missing Data
- interpolate(): Fills missing values using
interpolation.
Example:
# Interpolate missing values
df_interpolated = df.interpolate()
print(df_interpolated)
5. Replacing Specific Values
- replace(): Replaces specific values,
including NaN.
Example:
# Replace NaN with a specific value
df_replaced = df.replace(np.nan,
"Unknown")
print(df_replaced)
6. Checking for Missing Data in the Entire
DataFrame
You can
check if there are any missing values:
# Check if any values are missing
print(df.isnull().any().any())
# Count missing values per column
print(df.isnull().sum())
Summary Table of Methods
|
Method |
Purpose |
|
isnull() |
Detect
missing values |
|
notnull() |
Detect
non-missing values |
|
dropna() |
Drop
rows/columns with missing |
|
fillna() |
Fill
missing values |
|
interpolate() |
Interpolate
missing values |
|
replace() |
Replace
specific values |
These
tools help clean and prepare data effectively for analysis. Let me know if you
need help applying any of these!
International finance requires a solid understanding of exchange rates, global markets, and financial management concepts. Many learners benefit from international finance assignment answer support to understand difficult topics through structured explanations. This guidance encourages better learning, improves assignment quality, and helps students apply theoretical concepts to practical business situations. It also reduces academic pressure during busy semesters.
ReplyDelete