Data Cleaning & Missing Values in Pandas for Pharmacy Students (Clinical Dataset Challenges & Solutions)

Real-world healthcare datasets are often incomplete or inconsistent. In pharmaceutical sciences, data cleaning is essential to ensure accurate analysis of patient records, ADR datasets, and PK studies.

🔷 What is Data Cleaning?

Data cleaning is the process of identifying and correcting errors, missing values, and inconsistencies in datasets.

🔷 What are Missing Values?

Missing values occur when data is not available for certain entries.

import pandas as pd

df = pd.read_csv("clinical_data.csv")
print(df)

Missing values are often represented as NaN.

🔷 Detecting Missing Values

print(df.isnull())
print(df.isnull().sum())

👉 Helps identify which columns have missing data.

🔷 Removing Missing Values

df_clean = df.dropna()

Removes rows with missing values.

🔷 Filling Missing Values

📘 Fill with Mean

df["Dose"].fillna(df["Dose"].mean(), inplace=True)

📘 Fill with Constant

df.fillna(0, inplace=True)

💊 Clinical Dataset Example

df = pd.read_csv("adr_data.csv")

# Fill missing dose
df["Dose"].fillna(df["Dose"].mean(), inplace=True)

# Remove rows with missing drug name
df.dropna(subset=["Drug"], inplace=True)

👉 Ensures dataset is usable for analysis.

🔷 Handling Inconsistent Data

# Replace incorrect values
df["Reaction"].replace("sev", "Severe", inplace=True)

🧠 Memory Tricks

• isnull() → Find missing
• dropna() → Remove data
• fillna() → Fill data

🧪 Practice Exercise

Load dataset and:

• Find missing values
• Fill missing dose with average
• Remove incomplete rows

🧪 Mini Project

Clean patient dataset:

import pandas as pd

df = pd.read_csv("clinical_data.csv")

df["Dose"].fillna(df["Dose"].mean(), inplace=True)
df.dropna(inplace=True)

print("Cleaned Data:")
print(df)

📝 MCQs

1. Missing values are represented as:
   a) 0
   b) NaN
   c) None
   d) Blank
   Answer: b


2. Which function removes missing values?
   a) fillna()
   b) dropna()
   c) replace()
   d) remove()
   Answer: b


3. fillna() is used for:
   a) Delete data
   b) Fill missing data
   c) Print data
   d) Sort data
   Answer: b

❓ FAQs

Why is data cleaning important in pharmacy?

It ensures accurate clinical analysis and decision-making.

What is best method to handle missing data?

Depends on context—use mean, median, or remove rows.

➡ Next Topic: Filtering & Selecting Data →