SQLite by default supports only SQL tables with a maximum of 2000 columns. This number can be increased to around 32K but you need to recompile SQLite to do this because the SQLITE_MAX_COLUMN setting is a #pragma in the sqlite3.c file.
Below you find the steps to get the latest sqlite3 source code, edit it and rebuild it such that you can use it in your virtualenv as a regular sqlite3 package.
First get the source code and build the sqlite3.h header:
- wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
- tar xzf sqlite.tar.gz
- cd sqlite/
- ./configure
- make sqlite3.c
Then create a virtualenv, or use an existing one (where you need the new sqlite3)
- cd ~/.venv/<virtualenv>/
- source bin/activate
- git clone https://github.com/coleifer/pysqlite3
- cd pysqlite3
- cp /path/to/sqlite/sqlite3.[ch] .
Then edit sqlite3.c and change the line
# define SQLITE_MAX_COLUMN 2000to
# define SQLITE_MAX_COLUMN 5000or some other number.
- python setup.py build_static
- python setup.py install
The last command may raise an error like “Download error on https://pypi.org/simple/pysqlite3/: [SSL: CERTIFICATE_VERIFY_FAILED]”. In that case, upgrade setuptools as follows:
- pip install –upgrade setuptools
and rerun python setup.py build_static build and python setup.py install
Then deactivate virtualenv, reopen terminal, activate virtual env and type:
- python
- >>> from pysqlite import dbapi as sqlite3
References
https://charlesleifer.com/blog/compiling-sqlite-for-use-with-python-applications/