Merging DataFrames with duplicate Keys in Pandas

Last update on April 09 2025 12:52:11 (UTC/GMT +8 hours)

9. Merge Duplicate Keys

Write a Pandas program to merge DataFrames with duplicate Keys.

Following program shows how to merge DataFrames that contain duplicate keys, resulting in a Cartesian product of matching rows.

Sample Solution :

Code :

import pandas as pd

# Create two sample DataFrames with duplicate keys
df1 = pd.DataFrame({
    'ID': [1, 1, 2],
    'Name': ['Annabel', 'Annabel', 'Selena']
})

df2 = pd.DataFrame({
    'ID': [1, 2],
    'Age': [25, 30]
})

# Merge the DataFrames with duplicate keys
merged_df = pd.merge(df1, df2, on='ID')

# Output the result
print(merged_df)

Output:

   ID     Name  Age
0   1  Annabel   25
1   1  Annabel   25
2   2   Selena   30        

Explanation:

• Created two DataFrames df1 and df2 with duplicate keys in df1.
• Used pd.merge() to merge on the 'ID' column.
• The result is a Cartesian product where matching keys from both DataFrames are combined.

For more Practice: Solve these Related Problems:

• Write a Pandas program to merge two DataFrames with duplicate keys and then group by the key to aggregate duplicate rows.
• Write a Pandas program to merge DataFrames with duplicate keys and then filter the result to keep only unique key combinations.
• Write a Pandas program to merge DataFrames with duplicate keys and sort the result based on the frequency of key occurrences.
• Write a Pandas program to merge DataFrames with duplicate keys and create a new column indicating the count of duplicates per key.

Python-Pandas Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.