[Qgis-user] Limit to Geodatabase File Size?

[Even Rouault]

> I've never used GPKG but I do use spatialite all the time.  I always create
> without the index then create the index after I'm done.  It makes sense
> because if you start with the spatial index then it has to reshuffle the
> RTree with each new item.
> 
> In c++ code I set SPATIAL_INDEX=DEFFERED so that the index is built at the
> end.  Even would this work from the command line too?

The GPKG driver implements this deffered behaviour when you set SPATIAL_INDEX to YES (or 
doesn't specifiy it since it is the default behaviour). 

Basically it is 2 statements :

CREATE VIRTUAL TABLE rtree_tablename_geomcol
		USING rtree(id, minx, maxx, miny, maxy)

--> instantaneous


Populate the spatial index from the bounding box of geometries:

INSERT OR REPLACE INTO rtree_tablename_geomcol
 SELECT fid, st_minx(geom), st_maxx(geom), st_miny(geom), st_maxy(geom)
        FROM tablename

--> takes hours. Had to stop it since it made my computer slow.

I'm not sure of the reason of this very slow performance. Perhaps the I/O performance of my 
disk sucks, or there's something intrinsically slow to build the spatial index given the dataset.

> Also maybe this
> isn't available anymore I'm still using 1.11.x.

Yes spatial index support in GPKG requires at least GDAL 2.0

Even

> 
> 
> > Jonathon
> > 
> > On Tue, Mar 7, 2017 at 9:38 PM, Even Rouault <even.rouault at spatialys.com>
> > 
> > wrote:
> > > On mardi 7 mars 2017 14:30:13 CET Parker, Jonathon wrote:
> > > > New user to QGIS here. Went through some tutorials, had ESRI
> > > > experience
> > > 
> > > in
> > > 
> > > > distance past. Starting a demonstration project using data from the US
> > > > 
> > > > Census Bureau https://www.census.gov/geo/maps-data/data/tiger.html
> > > > 
> > > > 
> > > > 
> > > > I successfully added shapefiles and small geodatabases
> > > > 
> > > > ftp://ftp2.census.gov/geo/tiger/TGRGDB16// as layers. When I tried the
> > > > 
> > > > same process with a large geodatabase (tlgdb_2016_a_us_edges.gdb), it
> > > > did
> > > > 
> > > > not render. The feature count was reported as 0. I contacted the US
> > > 
> > > Census
> > > 
> > > > bureau and they verified it was not corrupted and worked on ESRI
> > > 
> > > products.
> > > 
> > > > Is there a size limit that prevents QGIS from working with large
> > > > files?
> > > > 
> > > > The table in the geodatabase is 13GB.
> > > 
> > > I've downloaded the file and reproduced the issue. The OGR OpenFileGDB
> > > driver had a bug in the particular situation where the attribute
> > > description section of the .gdbtable is beyond the 4 GB offset. I've
> > > just
> > > fixed it per
> > > 
> > > https://trac.osgeo.org/gdal/ticket/6830
> > > 
> > > 
> > > 
> > > The feature count now reported is ~68 million and it displays fine in
> > > QGIS
> > > (against the fixed GDAL version)
> > > 
> > > 
> > > 
> > > Best regards,
> > > 
> > > 
> > > 
> > > Even
> > > 
> > > 
> > > 
> > > --
> > > 
> > > Spatialys - Geospatial professional services
> > > 
> > > http://www.spatialys.com
> 
>  
>  


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20170308/f9a5346c/attachment.html>