Process HUGE Data Sets in Pandas
34,826
Published 2022-10-12
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
🐍 The Python Bible Book: www.neuralnine.com/books/
💻 The Algorithm Bible Book: www.neuralnine.com/books/
👕 Programming Merch: www.neuralnine.com/shop
🌐 Social Media & Contact 🌐
📱 Website: www.neuralnine.com/
📷 Instagram: www.instagram.com/neuralnine
🐦 Twitter: twitter.com/neuralnine
🤵 LinkedIn: www.linkedin.com/company/neuralnine/
📁 GitHub: github.com/NeuralNine
🎙 Discord: discord.gg/JU4xr8U3dm
🎵 Outro Music From: www.bensound.com/
All Comments (21)
-
I can't always follow everything he says, cause he moves pretty quick and throws a lot at you, but he's always straight to the point, no fluff, and innovative. I always glean more things to look up after hearing it from NeuralNine first.
-
Thanks a ton ! This is very helpful !
-
thanks but how you deal with depending row like times series data or observations like text where context correletead to row?
-
wie immer top content perfekt präsentiert!
-
why would you use csv format instead of parquet or hdf5 for large datasets?
-
Can we use each chunk to spawn a new process and do it in parallel?
-
Great video thanks
-
I like the simplicity. Wonder if a similar thing could be done with sql queries given they usually store incredibly large datasets.
-
it works! thanks!
-
i was litteraly watch a video when you post a new video...i like that!(8)
-
Your explanation is very good can you do a video on the Python project that else the position of an eye
-
In excel file, method "pd.read_excel" has no parameter "chunksize", how to handling the big data in many sheet in excel? Please help me!
-
Thank you.. Could you please make a tutorial on how you would stip out certain elements from a file that is not your typical "list", "csv" or "json".. Find this task to be the most confusing and difficult things you can do in Python. If needed, I can provide you with a text file which include information about airports such as runways, elevation, etc. Perhaps there are some way to clean such file up or even convert it to a json/excel/csv etc.
-
Thanks!
-
Awesome!
-
brilliant!
-
How can I connect database in python, and how to optimise it if I have 60L+ records in it
-
OMG tnx im trying to open csv file with million data then my pc collapse so i find some i9 computer with 16gb ram to open it thanks now i can open big files using pandas.
-
how we can further work on it. Suppose if want to use groupby function on column [ 'A '].
-
i'm a simple man, i see vim, i press like
