[Qgis-developer] python/ogr problems reading large gml file

[Even Rouault]

Le lundi 02 mai 2016 16:37:13, Richard Duivenvoorde a écrit :
> Thanks Even!
> 
> so
>  os.utime(os.path.join('/tmp/foo/data.gfs'), None)
> fixed all my strange problems :-)
> 
> apparently it was a some race-problem of the closing time of the gml
> file on disk and the modification time of the gfs: smaller files just
> worked etc etc..
> 
> anyway: thanks!
> Maybe a warning about ignoring the 'old' gfs would be nice...
> 
> I tried:
> $ ogrinfo --debug data1.gml
> but that did not show me more info
> 
> Then I did:
> $ ogrinfo --debug
> ERROR 1: --debug option given without debug level.

The syntax is "--debug on" to enable all debug messages. And "--debug 
component_name" to enable only the ones in component_name

> 
> Should I be able to use --debug from cli anyway?
> 
> Regards,
> 
> Richard
> 
> On 02-05-16 15:50, Even Rouault wrote:
> > Le lundi 02 mai 2016 15:38:45, Richard Duivenvoorde a écrit :
> >> Hi Devs,
> >> 
> >> (below all via python/plugin on Debian/Linux):
> >> 
> >> Via a WFS I retrieve 10000 features as GML as data.gml
> >> To be able to read (attr+geometry) the GML I copy a data.gfs next to it.
> >> 
> >> Now all seems ok, untill the number of features returned is > 1500.
> >> Strange errors like:
> >> 
> >> ogrinfo -ro data.gml reports a (None) geometry
> >> BUT if I 'touch' the data.gfs ogrinfo, then suddenly all is ok
> >> Seems like ogrinfo does not like the copied gfs at first?
> >> Permissions are ok. A 'touch /tmp/foo/data.gfs' is enough.
> > 
> > The OGR GML driver only uses the .gfs file if its modification time is
> > equal or greater than the the one of the .gml file (to avoid using an
> > outdated .gfs file). If you run ogrinfo with --debug on, you'll see a
> > message about that. Should probably be a more verbose warning.
> > 
> >> Off course the same holds for QGIS, if I open the retrieved and saved
> >> gml file as a gml-vectorlayer, a 1000 feature containing one is probably
> >> ok, but <1500 often fails and the 10000 one ALWAYS fails (untill I
> >> 'touch' the gfs file).
> >> IF something goes wrong: there is no exception, and 'layer.isValid()'
> >> returns true, but geometry/attributes are None at such moment.
> >> 
> >> I'm pretty carefull with closing/opening the files (I think), but it
> >> smells like either files that are still open, OR python loosing it's
> >> handles or so?
> >> 
> >> Anybody some idea on what I am doing wrong?
> >> Sorry, cannot sent you a wfs-url it's an internal service here.
> >> 
> >> Thansk for any pointers
> >> 
> >> Regards,
> >> 
> >> Richard Duivenvoorde
> >> _______________________________________________
> >> Qgis-developer mailing list
> >> Qgis-developer at lists.osgeo.org
> >> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com