union and union all in pyspark | unionByName in pyspark | merge two dataframes in pyspark
union and union all in pyspark | unionByName in PySpark | merge two dataframes in pyspark | databricks
"union and union all in pyspark" | "unionByName in PySpark" | "merge two dataframes in pyspark" | "databricks"
Creating dataframe 1
data_1 = [Row(employee_name="Alice", employee_gender="F", employee_salary=70000),
Row(employee_name="Bob", employee_gender="M", employee_salary=80000),
Row(employee_name="Charlie", employee_gender="M", employee_salary=55000),
Row(employee_name="David", employee_gender="M", employee_salary=45000),
Row(employee_name="Eve", employee_gender="F", employee_salary=50000),
Row(employee_name="Eve", employee_gender="F", employee_salary=50000)
]
df_1 = spark.createDataFrame(data_1)
df_1.show()
-------------------------------------------------------------------------------------------------------------
creating dataframe 2
data_2 = [Row(employee_name="Frank", employee_gender="M", employee_salary=60000),
Row(employee_name="Grace", employee_gender="F", employee_salary=65000),
Row(employee_name="Hannah", employee_gender="F", employee_salary=70000),
Row(employee_name="Ian", employee_gender="M", employee_salary=48000),
Row(employee_name="Jill", employee_gender="F", employee_salary=53000),
Row(employee_name="Eve", employee_gender="F", employee_salary=50000)
]
df_2 = spark.createDataFrame(data_2)
df_2.show()
---------------------------------------------------------------------------------------------------------------------
Creating DataFrame data_1
data_1 = [[10,20,30]]
schema_1 = "col_0 int , col_1 int , col_2 int"
df_1 = spark.createDataFrame(data = data_1, schema = schema_1)
df_1.show()
-------------------------------------------------------------------------------------------------------------------
Creating DataFrame data_2
data_2 = [[100,200,300]]
schema_2 = "col_0 int , col_1 int , col_2 int"
df_2 = spark.createDataFrame(data = data_2, schema = schema_2)
df_2.show()
Need Help ? Connect With me 1:1 - https://topmate.io/dewithdhairy
Let's connect on LinkedIn : / dhirajgupta141
top interview question and answer in pyspark :
• top interview questions and answers in pys...
PySpark Installation and Setup : • Spark Installation | PySpark Installation ...
DSA In Python Interview Series : • dsa for data engineer | dsa for data analy...
PySpark Interview Series : • pyspark interview questions and answers
Pandas Interview Series : • pandas interview questions and answers
SQL Interview Series : • sql interview questions and answers
#pwc #fang #pyspark #sql #interview #dataengineers #dataanalytics #datascience #StrataScratch #Facebook #data #dataengineeringinterview #codechallenge #datascientist #pyspark #CodingInterview
#dsafordataguy #dewithdhairy #DEwithDhairy #dhiarjgupta #leetcode #topinterviewquestion