[gdal-dev] Request for advice in unexpected slowdown working with LVBAG

[Even Rouault]

Actually, you might have meant 3.0.3 and 3.5.3. When using 3.0.3, LVBAG 
files are read through the generic GML driver. When using 3.5.3, they 
are read with the new LVBAG driver 
(https://gdal.org/en/latest/drivers/vector/lvbag.html). When reviewing 
it, I saw that it ran a IsValid() check, even if 
AUTOCORRECT_INVALID_DATA is not set, which bears some cost. I've 
submitted an enhancement in https://github.com/OSGeo/gdal/pull/11234 to 
avoid doing that check when not requested.  You can still use the 
generic GML driver by setting the OGR_SKIP=LVBAG environment 
variable/configuration option. But the LVBAG driver should better 
preserve information.

Le 10/11/2024 à 13:33, Even Rouault via gdal-dev a écrit :
> Jan,
>
> What are those 3.03 and 3.53 versions ? They don't match GDAL official 
> version numbers. I thought you meant 3.0.3 and 3.5.3, but the LVBAG 
> driver is new to 3.2.0, so that can't be it.  Xerces is *not* a 
> required DLL for that driver. It uses Expat. Do you use the 
> AUTOCORRECT_INVALID_DATA=YES open option? If so, the difference could 
> come from the GEOS version
>
> Even
>
> Le 10/11/2024 à 09:58, Jan Heckman via gdal-dev a écrit :
>> My c++ utility-app collects Kadaster BAG data in postgresql tables 
>> for use in "populatie service".
>> It has functioned for several years, with a typical runtime of 40 
>> minutes.
>> In the process of rounding this off, I updated the gdal binaries from 
>> 3.03 to 3.53, as well as added some threaded workloads for analyzing 
>> the tables.
>> Confusingly, the runtime went up to 4 hours and more. This is after 
>> the download, so internet speed has no role here. It's all local, no 
>> network and my PC has not degraded. I have tried to reconstruct an 
>> older setup, but failed to get it back up to speed.
>> Profiling and random breaks during waiting point conclusively to the 
>> function getting features from a layer.
>> LVBAG processes lots of xml of a specific (but not highly particular) 
>> structure.
>> My app is compiled, if so indicated, against expat. Xerces is also a 
>> required dll.
>> So my first idea was that there might be a serious performance 
>> difference between xml libs, but without direct evidence.
>> I am using gdal windows binaries from Gisinternals.
>> All suggestions are welcome!
>> Thanks in advance,
>> Jan
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-- 
http://www.spatialys.com
My software is free, but my time generally not.
Butcher of all kinds of standards, open or closed formats. At the end, this is just about bytes.