Optimize string operations in Pandas: str accessor vs. apply

Pandas: Performance Optimization Exercise-15 with Solution

Write a Pandas program to optimize the performance of string operations on a DataFrame column by using the str accessor vs. applying a custom function with apply.

Sample Solution :

Python Code :

# Import necessary libraries
import pandas as pd
import time

# Create a sample DataFrame
num_rows = 1000000
df = pd.DataFrame({
    'text': ['example_string'] * num_rows
})

# Measure time for str accessor method
start_time = time.time()
str_accessor_result = df['text'].str.upper()
end_time = time.time()
str_accessor_time = end_time - start_time

# Define a custom function to apply
def to_upper(text):
    return text.upper()

# Measure time for apply method
start_time = time.time()
apply_result = df['text'].apply(to_upper)
end_time = time.time()
apply_time = end_time - start_time

# Print the time taken for each method
print(f"Time taken using str accessor: {str_accessor_time:.6f} seconds")
print(f"Time taken using apply method: {apply_time:.6f} seconds")

Output:

Time taken using str accessor: 0.181023 seconds
Time taken using apply method: 0.139029 seconds

Explanation:

• Import Libraries:
• Import pandas, numpy, and time.
• Create DataFrame:
• Generate a sample DataFrame with 1,000,000 rows, each containing a string.
• Time Measurement for str Accessor:
• Measure the time taken to convert strings to uppercase using the str.upper accessor.
• Define Custom Function:
• Define a custom function to convert strings to uppercase.
• Time Measurement for apply Method:
• Measure the time taken to apply the custom function using the apply method.
• Finally print the time taken for each method.

Python-Pandas Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Performance comparison of cumulative Sum calculation in Pandas.
Next: Reshaping DataFrame in Pandas: pivot_table vs. manual Loop.