How to handle "Memory Error" while loading a huge file in Python-Pandas
Learn how to solve the memory error while working with a huge file in Pandas-Python.
#MemoryError #Python #Pandas
How to read a sample data from csv file without opening it in Excel or Python?
The data to load
f = "telco_churn.csv"
Count the lines
num_lines = sum(1 for l in open(f))
Sample size - in this case ~50%
size = int(num_lines // 2)
The row indices to skip - make sure 0 is not included to keep the header!
random.seed(100)
skip_idx = random.sample(range(1, num_lines), num_lines - size)
Read the data
data = pd.read_csv(f, skiprows=skip_idx)
Connect with me here:
LinkedIn: / ssumitshukla
Twitter: / swssumit
Facebook: / sumitshuklasws