How to Solve Python Memory Problems with SQLite UDFs
Have you ever run into MemoryError in Python? How can SQLite help you with large datasets?
In this video, we will show you how to use SQLite and write User Defined Functions (UDF) to solve resource problems when you deal with data.
1:23 Why SQLite?
3:24 Pass data between Python and SQLite
5:09 Why not bring it into memory?
7:29 What is serialization?
8:59 Explanation of code
11:09 json_each, and SQLite built-ins
13:42 User Defined Functions (UDF) examples
15:39 Why not memory? Revisited.
16:57 **Turning Python function into SQLite function**
18:25 How long does a UDF live?
19:37 Why you should remember to close the connection.
CODE: https://github.com/SeanTater/software-for-science
MUSIC
Nimbus by Eveningland https://www.youtube.com/audiolibrary/music
SPECIAL THANKS
– Dave Harned from Junior Developer Toolbox https://juniordevelopertoolbox.com/
– Stephen Welch from Welch Labs
https://www.youtube.com/channel/UConVfxXodg78Tzh5nNu85Ew
– Ben Drozdoff
Comments are closed.