[Qgis-psc] [QGIS-Developer] (QGIS 2020 Grant Report) Use of FileGeodatabase spatial index in OpenFileGDB driver

[Andreas Neumann]

@Even - also a big "thank you" from my side! I won't appreciate it as much
as others though - as I never have to deal with ESRI formats ... except an
occasional shapefile. Luckily, most governments in Switzerland aren't
distributing their data in ESRI formats ...

@Tim: I will do my best, but some things to consider:
- bug fixes often happen until the last hour/minutes
- if I do it too early, we'll miss out on some entries
- if the process is automated - then we can publish it 2 or 3 times when
there are late entries.

It would be great if we could manage to publish the visual changelog in
parallel with the release. We don't need for all entries to be ready, just
publish it shortly before the release, and then an update after the
release. I think that in reality, we'll need to publish at least twice.

Andreas

On Tue, 29 Sep 2020 at 00:15, Tim Sutton <tim at kartoza.com> wrote:

> Wow, thank you for the update, great work!
>
> @Andreas do you think it would be possible to accummulate the paid bugfix
> and grant work in the changelog a week or so before we release?  We want to
> try for the first time to have the visual changelog go out at the same time
> as the release...
>
> Regards
>
> Tim
>
> On Mon, 28 Sep 2020, 21:29 Even Rouault, <even.rouault at spatialys.com>
> wrote:
>
>> Hi,
>>
>>
>>
>> I want to report about the outcome of
>>
>> https://github.com/qgis/QGIS-Enhancement-Proposals/issues/172
>>
>> "Use of FileGeodatabase spatial index in OpenFileGDB driver"
>>
>>
>>
>> This work has been successfully completed into GDAL master (for upcoming
>> GDAL 3.2) per https://github.com/OSGeo/gdal/pull/2771 , and
>> automatically benefits to QGIS when it uses the OpenFileGDB driver.
>>
>>
>>
>> Some figures on a version from of few years ago of the dataset at
>>
>> https://data.linz.govt.nz/layer/50772-nz-primary-parcels/, featuring 2
>> 484 646 polygons, 773 MB:
>> "ogrinfo -spat 174.949909 -41.143842 175.684529 -40.594600 -al -so
>> -noextent" (thus counting the number of features intersecting the spatial
>> filter), which returns 81 046 polygons, nows runs in 400 ms with GDAL
>> 3.2dev and the OpenFileGDB driver, versus 6.7 s before (full scan), vs 890
>> ms with the FileGDB driver (with FileGDB SDK 1.5).
>>
>> All figures given with debug builds, and on a hot run (so with most pages
>> cached by the OS).
>>
>>
>>
>> Interactive display of this dataset in QGIS with the OpenFileGDB driver
>> is as fluid as with the FileGDB one. Comparing behaviour of OpenFileGDB and
>> FileGDB drivers with strace shows that they read a similar amount of data
>> in the .spx file, which confirms it is uses correctly.
>>
>>
>>
>> The filegdb reverse engineered specification was also updated in
>>
>>
>> https://github.com/rouault/dump_gdbtable/wiki/FGDB-Spec#specification-of-spx-files
>>
>>
>>
>> Thanks to qgis.org grant program for making this improvement possible.
>>
>>
>>
>> Best regards,
>>
>>
>>
>> Even
>>
>>
>>
>> --
>>
>> Spatialys - Geospatial professional services
>>
>> http://www.spatialys.com
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> _______________________________________________
> Qgis-psc mailing list
> Qgis-psc at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/qgis-psc



-- 

--
Andreas Neumann
QGIS.ORG board member (treasurer)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20201002/14f8f33d/attachment.html>