Spark Basics | Partitions
Spark is a distributed computing system that is used within Foundry to run data transformations at scale. This series covers the core Spark concepts you need to know for working with data in Foundry.
In this video we introduce partitions, discuss the importance of partition sizing, demonstrate how to find the count and size of partitions for a dataset in Foundry, and describe methods for changing the number of partitions in a Spark DataFrame.