Extract Dictionary Value from Column in Data Frame

Data Frames in Python, especially with libraries like Pandas, are powerful tools for data manipulation and analysis. Sometimes, data stored within a Data Frame's column may be in the form of dictionaries. Extracting values from such dictionaries is a common operation in data preprocessing. This post explores various methods to extract dictionary values from Data Frame columns efficiently in Python.

Extracting Dictionary Values from Columns in a Dataframe

Below are some of the ways by which we can extract dictionary values from columns in a DataFrame in Python:

1. Using Apply and Lambda Functions
2. Using List Comprehension

Using Apply and Lambda Functions

In this method, we use the pandas apply() function to a column. It takes a function as an argument which will be applied to each value of that column. The function to be passed to apply() will be a lambda function, which will extract each iterable from a particular column.

Example: In this example, we create a data frame with two columns, one of them is of dictionary type. To extract the dictionary values, we then used apply() and lambda function to each keys of the dictionary.

# import pandas
import pandas as pd
# Sample DataFrame
data = {
  'ID': [1, 2, 3],
  'Details': [{'Name': 'Monu', 'Age': 30},
              {'Name': 'Sonu', 'Age': 25},
              {'Name': 'Golu', 'Age': 35}]
}

df = pd.DataFrame(data)

# Accessing and extracting values from 'Details' column
df['Name'] = df['Details'].apply(lambda x: x['Name'])
df['Age'] = df['Details'].apply(lambda x: x['Age'])

print(df)

   ID                      Details  Name  Age
0   1  {'Name': 'Monu', 'Age': 30}  Monu   30
1   2  {'Name': 'Sonu', 'Age': 25}  Sonu   25
2   3  {'Name': 'Golu', 'Age': 35}  Golu   35

Using List Comprehension

Another method to extract dictionary values from data frame column is by using python list comprehension, where using a for loop list comprehension, each value of the dictionary is extracted and stored in the form of a list.

Example: In this example, we simply use list comprehension on the column with dictionary values and extract each value through list comprehension.

# import pandas
import pandas as pd
# Sample DataFrame
data = {
  'ID': [1, 2, 3],
  'Details': [{'Name': 'Monu', 'Age': 30},
              {'Name': 'sonu', 'Age': 25},
              {'Name': 'Golu', 'Age': 35}]
}

df = pd.DataFrame(data)

# Accessing and extracting values from 'Details' column
df['Name'] = [i['Name'] for i in df['Details']]
df['Age'] = [i['Age'] for i in df['Details']]

print(df)

Output:

    ID                                   Details   Name  Age
0   1   {'Name': 'Monu', 'Age': 30}   Monu    30
1    2    {'Name': 'sonu', 'Age': 25}    Sonu     25
2   3    {'Name': 'Golu', 'Age': 35}     Golu     35