Q11. Realtime Scenarios Interview Question | PySpark | Header in PySpark
Here is the link to the project course:
https://www.geekcoders.co.in/courses/...
If you haven't taken my Azure End-to-End Project course then here is the link(More than 450 users have already taken it)
https://www.geekcoders.co.in/courses/...
"""
path='/FileStore/tables/csv_with_no_header.csv'
df=spark.read.option('header',True).option('inferschema',True).csv(f'{path}').limit(1)
if(df.schema==schema):
df.write.mode('overwrite').save('/FileStore/tables/csv_with_header_output/')
else:
df=spark.read.option('header',False).schema(schema).csv(f'{path}')
df.write.mode('overwrite').save('/FileStore/tables/csv_with_header_output/')
"""
Please visit my website also for more info
https://www.geekcoders.co.in/
Looking for 1:1 Long mentorship
https://app.preplaced.in/profile/saga...
HASHTAGS
TAGS
#tags #hashtags
YouTube Link: / @geekcoders
GeekCoders: https://www.geekcoders.co.in/courses
LinkedIn: / sagar-prajapati-86073b155
Telegram: https://lnkd.in/dKz7s3y8
Instagram: https://lnkd.in/dKz7s3y8
#pyspark #pysparkinterview #spark