[gdal-dev] RE: progressive rendering

[Norman Barker]

Hi Frank,

-----Original Message-----
From: Frank Warmerdam [mailto:warmerdam at pobox.com] 
Sent: Tuesday, August 19, 2008 3:45 PM
To: Norman Barker
Cc: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] RE: progressive rendering

Norman Barker wrote:
> Mainly it is either end of response, image done, or window done to
> indicate that there is a buffer update and the display should be
> updated, anything else can pretty much be ignored.  
> 
> Ideally the callback should indicate the region that has changed,
since
> the server is entitled to change the region it streams if the client
has
> requested an invalid ROI.

Norman,

If we load this capability onto RasterIO() call I don't imagine we
would want to allow the client to change the area being loaded. If
an invalid ROI is requested the driver should either error out, or
only update the valid part of the buffer.

<ncb>
We have 

User -> format driver -> server

The server is allowed (by the spec) to change the ROI it serves, the
returned stream is self-describing and the format driver interprets this
stream. 

It is unlikely that the format driver will have a complete blank buffer
of the whole image and only update sections of this buffer, it will
probably just keep a tiny buffer and the attributes (x,y, width, height,
res etc.) of this buffer and publish this to the user.
</ncb>

> The callback isn't per scan line, but a whole window (at a particular
> resolution level).
> </ncb>

Ah, gotcha.  I hadn't even considered this aspect!

> The simpliest case I can imagine would be passing (or installing) a
> callback that drivers *could* call periodically to indicate that some
> unspecified area of the RasterIO() buffer has been updated.
> 
> <ncb>
> The callback would need to include the x,y offsets, width, height and
> resx, resy since otherwise there needs to be too much state
> synchronization between the format driver and the client.  The
simplest
> would be to specify x,y, width, height at the base resolution of the
> image and to use resx, resy to sample for the display implying the
> buffer size.  The format driver would keep the cache.
> </ncb>

OK, now you have lost me.  The RasterIO() call includes a buffer,
and defines the size of that buffer (in x and y) and the size of
file window to load into that buffer.  I don't get what you mean
about there being too much state synchronization between the driver
and the client.

<ncb>
The user (client display) is changing the request window constantly to
the format driver as they pan and zoom the image, there is a lag between
this change and the server response - however the response stream is
self describing, and the format driver will have knowledge of the
currently processed request window (which may not have been updated with
the new user request).  It is simpler for the format driver to return
the window parameters it used to fill the buffer rather than the user
(client display) to think it has received a different buffer. 

The reason for the mismatch is that the call to the server and format
driver call back are likely to be in different threads.

Sorry for not having made that clearer!
</ncb>

> Is the main objective to be able to update stuff on the screen while a
> buffer is being downloaded by JPIP or other slow access mechanisms?
> 
> <ncb>
> Definitely, this is the main objective.  A secondary objective is to
use
> the mechanism to download complete sections of a streaming image (so
to
> stream until a window done message) for use with gdal_translate for
> example.
> </ncb>

I don't follow the secondary objective.   How does having the callback
add value to just reading the data into the buffer in chunks and
then writing it to the output file?

<ncb>
There is no advantage with the secondary objective, it is just that a
single request window to a jpip server has been disabled in GDAL for a
while, and if we are getting objective (1) to work, then it would be
good to get (2) to work as well!
</ncb>

Perhaps it would make sense to mock up a proposed RasterIO() variation
with the callback in RFC form so we can more easily talk about the same
thing?  This can be done in the trac wiki.

<ncb>
I agree, watch this space - I will get something up on the wiki for
discussion.  Thanks again for discussing this.

Norman
</ncb>

Best regards,
-- 
---------------------------------------+--------------------------------
------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent