Fwd: [gdal-dev] Can OGR WFS driver take advantage of WFS Server streaming?

[Yingqi Tang]

Resend to the mailing list.

---------- Forwarded message ----------
From: Even Rouault <even.rouault at mines-paris.org>
Date: Thu, May 17, 2012 at 12:05 PM
Subject: Re: [gdal-dev] Can OGR WFS driver take advantage of WFS Server
streaming?
To: Yingqi Tang <anakintang at gmail.com>


Le jeudi 17 mai 2012 20:52:07, Yingqi Tang a écrit :
> Even,
>
> Just to clarify, when you say:
>
> "...server allows partial downloading of arbitrary ranges of bytes within
> the file...", what does it really mean? If I simply put a static GML file
> in IIS and share it through HTTP, will it support partial downloading?

Yes, static pages should support partial downloading. But WFS servers that
generate response on the fly generally don't.

>
> Thanks,
> Yingqi
>
> On Thu, May 17, 2012 at 4:19 AM, Even Rouault
>
> <even.rouault at mines-paris.org>wrote:
> > Le jeudi 17 mai 2012 00:25:52, Yingqi Tang a écrit :
> > > Thanks for confirming it, Even. I have two follow up questions though:
> > >
> > > 1. Does the GML driver (reading local big GML file) also load the
whole
> >
> > GML
> >
> > > in memory too?
> >
> > No, it will read it by small chunks. If no matching XML schema is
> > available (or that the parser of the GML driver cannot manage to
> > understand it because
> > it is too complex (when the GML data is not limited to the Simple
Feature
> > convention)), there will be an initial full scan of the file to
determine
> > the
> > schema of features.
> >
> > > 2. Does the GML driver take url pointing to a GML?
> >
> > It can work with /vsicurl/http://something, provided that the server
> > allows
> > partial downloading of arbitrary ranges of bytes within the file, which
> > is rarely the case with GetFeature() requests.
> >
> > In theory the WFS driver could be improved to use the capability of the
> > GML driver to read chunk by chunk, but there is some work to make that
> > happen. This would probably require a /vsicurl_streaming/ virtual file
> > system where basically only Read() requests would be available, thus not
> > requiring the server to have download range capabilities.
> >
> > > Thanks
> > >
> > > On Wed, May 16, 2012 at 3:20 PM, Even Rouault
> > >
> > > <even.rouault at mines-paris.org>wrote:
> > > > Le mercredi 16 mai 2012 23:58:29, Yingqi Tang a écrit :
> > > > > Hi all,
> > > > >
> > > > > I am communicating against a WFS server which supports data (GML)
> > > > > streaming, which means that once I send a GetFeature quest for a
> > > > > big GML (2.0GB+), server will take only seconds to start streaming
> > > > > GML back to client while at the same time it's still preparing the
> > > > > rest
> >
> > of
> >
> > > > > the big
> > > >
> > > > GML.
> > > >
> > > > > Now I wonder if GDAL/OGR WFS client can take advantage of that?
> > > > > Will
> >
> > I
> >
> > > > > be able to access the first few features while the streaming is
> > > > > still going on? It seems to me that it doesn't because it always
> > > > > waiting
> >
> > the
> >
> > > > > whole
> > > >
> > > > big
> > > >
> > > > > GML response is loaded in memory.
> > > >
> > > > Yes indeed, the OGR WFS driver waits for the whole response to be
> > > > downloaded
> > > > before parsing it. It could be desirable to have on-the-fly parsing
> > > > indeed, but
> > > > we currently lack the capability of doing it with the way we use the
> >
> > Curl
> >
> > > > library.
> > > >
> > > > > Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20120517/41d06d92/attachment.html