Create PySpark dataframe from dictionary Last Updated : 30 May, 2021 Comments Improve Suggest changes Like Article Like Report In this article, we are going to discuss the creation of Pyspark dataframe from the dictionary. To do this spark.createDataFrame() method method is used. This method takes two argument data and columns. The data attribute will contain the dataframe and the columns attribute will contain the list of columns name. Example 1: Python code to create the student address details and convert them to dataframe Python3 # importing module import pyspark # importing sparksession from # pyspark.sql module from pyspark.sql import SparkSession # creating sparksession and giving # an app name spark = SparkSession.builder.appName('sparkdf').getOrCreate() # list of college data with dictionary data = [{'student_id': 12, 'name': 'sravan', 'address': 'kakumanu'}] # creating a dataframe dataframe = spark.createDataFrame(data) # show data frame dataframe.show() Output: Example2: Create three dictionaries and pass them to the data frame in pyspark Python3 # importing module import pyspark # importing sparksession from # pyspark.sql module from pyspark.sql import SparkSession # creating sparksession and giving # an app name spark = SparkSession.builder.appName('sparkdf').getOrCreate() # list of college data with dictionary # with three dictionaries data = [{'student_id': 12, 'name': 'sravan', 'address': 'kakumanu'}, {'student_id': 14, 'name': 'jyothika', 'address': 'tenali'}, {'student_id': 11, 'name': 'deepika', 'address': 'repalle'}] # creating a dataframe dataframe = spark.createDataFrame(data) # show data frame dataframe.show() Output: Comment More infoAdvertise with us Next Article Create PySpark dataframe from dictionary sravankumar_171fa07058 Follow Improve Article Tags : Python Python-Pyspark Practice Tags : python Similar Reads Create PySpark dataframe from nested dictionary In this article, we are going to discuss the creation of Pyspark dataframe from the nested dictionary. We will use the createDataFrame() method from pyspark for creating DataFrame. For this, we will use a list of nested dictionary and extract the pair as a key and value. Select the key, value pairs 2 min read Create pandas dataframe from lists using dictionary Pandas DataFrame is a 2-dimensional labeled data structure like any table with rows and columns. The size and values of the dataframe are mutable, i.e., can be modified. It is the most commonly used pandas object. Creating pandas data-frame from lists using dictionary can be achieved in multiple way 2 min read PySpark - Create dictionary from data in two columns In this article, we are going to see how to create a dictionary from data in two columns in PySpark using Python. Method 1: Using Dictionary comprehension Here we will create dataframe with two columns and then convert it into a dictionary using Dictionary comprehension. Python # importing pyspark # 3 min read PySpark - Create DataFrame from List In this article, we are going to discuss how to create a Pyspark dataframe from a list. To do this first create a list of data and a list of column names. Then pass this zipped data to spark.createDataFrame() method. This method is used to create DataFrame. The data attribute will be the list of da 2 min read How to create DataFrame from dictionary in Python-Pandas? The task of converting a dictionary into a Pandas DataFrame involves transforming a dictionary into a structured, tabular format where keys represent column names or row indexes and values represent the corresponding data.Using Default ConstructorThis is the simplest method where a dictionary is dir 3 min read Convert PySpark DataFrame to Dictionary in Python In this article, we are going to see how to convert the PySpark data frame to the dictionary, where keys are column names and values are column values. Before starting, we will create a sample Dataframe: Python3 # Importing necessary libraries from pyspark.sql import SparkSession # Create a spark se 3 min read Create Pandas Dataframe from Dictionary of Dictionaries In this article, we will discuss how to create a pandas dataframe from the dictionary of dictionaries in Python. Method 1: Using DataFrame() We can create a dataframe using Pandas.DataFrame() method. Syntax: pandas.DataFrame(dictionary) where pandas are the module that supports DataFrame data struct 2 min read Convert Python Dictionary List to PySpark DataFrame In this article, we will discuss how to convert Python Dictionary List to Pyspark DataFrame. It can be done in these ways: Using Infer schema.Using Explicit schemaUsing SQL Expression Method 1: Infer schema from the dictionary We will pass the dictionary directly to the createDataFrame() method. Syn 3 min read How to create an empty PySpark DataFrame ? In PySpark, an empty DataFrame is one that contains no data. You might need to create an empty DataFrame for various reasons such as setting up schemas for data processing or initializing structures for later appends. In this article, we’ll explore different ways to create an empty PySpark DataFrame 4 min read Create a Pandas DataFrame from List of Dicts Pandas DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. It is generally the most commonly used Pandas object. Pandas DataFrame can be created in multiple ways using Python. Let’s discuss how to create a Pandas DataFrame from the List of Dictionaries. C 3 min read Like