Create A Empty or Dummy Pyspark Dataframe in Databricks | dr.dataspark
Hello,
In this video, I have showcased how to create dummy or empty pyspark dataframe associated with some schema. You can use this dataframe to store the schema information in Azure Blob or Azure ADLS to re-use in future or any other databricks notebook. By doing this you can help the production jobs to first infer the schema and store it first and then infer this schema may be later and reduce the SLA of business.
Do like, share and comment your thoughts.
Follow me on Instagram: / dr.dataspark
Connect me on LinkedIn: / bigdatamanoj
#pyspark #databricks #drdataspark