[QGIS-Developer] Geopackage: ERROR: Too many connections: max 64 ?

[Richard Duivenvoorde]

Hi Devs,

I created a geopackage from 144 individual (hourly data) shapefiles as 
individual tables (in one gpkg).

To load all those tables in QGIS I can 'load vector data', point to the 
gpkg, choose 'select all' and then OK.

BUT: it looks like both in 2.18 and in master it for every table QGIS 
(or OGR?) creates a connection?
In my Debug builds the loading takes several minutes showing me msg 
like:

src/providers/ogr/qgsogrprovider.cpp: 3196: (OGRDestroyWrapper) [167ms] 
GPKG: Trying to return to delete mode
src/providers/ogr/qgsogrprovider.cpp: 3214: (OGRDestroyWrapper) [0ms] 
Return: In ExecuteSQL(): sqlite3_step(PRAGMA journal_mode = delete):
   database is locked
src/providers/ogr/qgsogrprovider.cpp: 3224: (OGRDestroyWrapper) [3ms] 
GPKG: Trying again
ERROR: Too many connections: max 64

And when I close the project or try to create a new project this happens 
again:

src/providers/ogr/qgsogrprovider.cpp: 3224: (OGRDestroyWrapper) [4ms] 
GPKG: Trying again
src/providers/ogr/qgsogrprovider.cpp: 3242: (OGRDestroyWrapper) [163ms] 
Return: wal
src/providers/ogr/qgsogrprovider.cpp: 3196: (OGRDestroyWrapper) [5ms] 
GPKG: Trying to return to delete mode
src/providers/ogr/qgsogrprovider.cpp: 3214: (OGRDestroyWrapper) [1ms] 
Return: In ExecuteSQL(): sqlite3_step(PRAGMA journal_mode = delete):
   database is locked

In general both processes will finally end, but I think this is/will be 
a problem in the future?

Note that loading the 144 shapefiles take a lot less time.

I'm not sure if I'm stretching the usability of GeoPackage too much 
here?
(my plan is/was to create a grouplayer-player, a group(p)layer: some 
buttons for given layergroup to show/hide the next layer thereby having 
a poor-man's 'time'-manager...

Is it possible (at least for the loading of several tables at once) to 
do the opening/closin sequential over the same connections.

Or should we 'just' make 'max 64' 'max 6400' ;-)

Is there a solution thinkable for this, should I create an issue?

Regards,

Richard Duivenvoorde

PS: Happy to provide the (1.1 Gb) Geopackage I created for testing.
(note that the 144 shapefile were also almost 1Gb, BUT zipped it is 
about 250Mb).

PS2: Also trying to discuss this in the light of the future of gpkg in 
QGIS.