[gdal-dev] GDAL Progressive Data Support RFC Discussion

[Aron Rubin]

On Tue, Mar 16, 2010 at 4:31 PM, Frank Warmerdam <warmerdam at pobox.com>wrote:

> With regard to the fixed vs. mixed resolution results, it is true
> that the current API provides no mechanism to request particular
> levels of completeness, such as passes in an interleaved progressive
> display.  I imagine this issue was not addressed in part because
> it has no applicability to the JPIP case.  I'm not at all clear on
> how important it is.
>

You might have guessed that it a technique I use in my own applications.
This could also allow notification for tile/block completion among other
things. I am not sure how JPIP is implemented in GDAL but it seems that only
a fixed window and resolution are allowed at a time I am not sure if the
image data itself can be progressive. However JPEG (maybe JPEG in Tiff?) and
PNG support progressive data. Loading a GeoTiff I would want to know when a
tile read was complete.

>
> I still didn't quite grasp why an event/callback approach is better
> than GetNextUpdateRegion() style polling to handle distinct levels
> of completeness, nor what you mean by mangaging the I/O budget.


The timeout passed to GetNextUpdateRegion is abstractly thought of as an I/O
budget. In this case it is constrained in time. On a related note the RFC
does not specify when GetNextUpdateRegion would return besides the
expiration of the timeout and end of stream.

 2. There is no consideration or mention of how this would work with
> user managed I/O if that ever happens. I would like to eventually see
> a GDAL/OGR user managed I/O in both push and pull forms. Asynchronous
> I/O and user managed I/O have implications for each other.
>

I'm not sure what you mean by user managed I/O.  Do you mean applications
> providing their own low level IO handlers - for instance to the VSI*L
> layer?  I'm not sure I see the relationship between the progressive data
> api and managing IO.
>

Right, I mean when the user feeds data (push) to a driver or a I/O handling
object (pull) like VSIVirtualHandle (preferably with a C interface). This
would interact with async I/O because it would affect the I/O state and
enable a graceful read interruption. For example if a synchronous read does
not return the expected number of bytes in the current GDAL my guess is that
would cause an error condition. In an asynchronous read there is no
completion expectation so a PENDING could be returned. That type of behavior
was not captured in the RFC.


> I appreciate your feedback though I am concerned that you are raising
> pretty fundamental changes in the implementation.  I'd want pretty
> compelling reasoning to start over.


I don't think a rewrite is at all required to implement some of these
suggestions. If it would be helpful for me to demonstrate how to derive the
same GetNextUpdateRegion interface from a callback style interface.

Aron

-- 
Aron Rubin
Handy Husband & Daddy Jungle Gym & Senior Engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100317/0c6fbcde/attachment.html