[PROJ] Advice for debugging network callback implementation

[Even Rouault]

Hi,

> I’m currently implementing PROJ's network callback functionality for the
> proj-rs Rust crate. I’ve got a skeleton of the functionality, but the docs
> are a little vague on some points, which is making debugging tricky.
> Specifically, I’ve implemented the four callbacks (open_cbk, close_cbk,
> get_header_value_cbk, read_range_cbk), and I’ve set up my test as follows:
> 
> - chosen EPSG codes that will trigger a download: from EPSG4326, to
> EPSG:4326+3855 - create new proj context with proj_context_create
> - disabled the grid cache
> - enabled the network callbacks using set_network_callbacks
> - used the context to create a transform between known EPSG codes using
> proj_create_crs_to_crs - A call to open_cbk is triggered:
> 
> The url is https://cdn.proj.org/us_nga_egm08_25.tif, with a “Range:
> bytes=0-16384” header.

If offset == 0 and size_to_read == 16384, you should issue a 
"Range: bytes=0-16383" header, since this is the HTTP convention is to use inclusive start and 
end offset. See
https://github.com/OSGeo/PROJ/blob/master/src/networkfilemanager.cpp#L1693

> I can confirm that I get 1639 bytes of content back,
> along with a response header: “content-range: bytes 0-1638/93581”

The values you report are not consistent with your above range

> I cast
> the response bytes to a void pointer, and assign it to the buffer that is
> passed into open_cbk by PROJ. I assign the headers and a client instance
> (this is an implementation detail for now) to a struct which I cast to a
> void pointer and then to a *PROJ_NETWORK_HANDLE, which I return.

Sounds fine

> I’m now not sure where to proceed next: Is there any special format or
> structure I should be assigning to the bytes from the GET call before I
> cast them to a void pointer and put them into the buffer?

No. Up to you to create your own structure.

> Is there any way
> I can verify that I’m getting back what I should? I can certainly manually
> read the first 16385 bytes from the tif, and compare them to what I get
> back over the wire. What is PROJ trying to do as a next step after it
> checks the header value, as this is the step which is currently failing?

You could look at the default curl implementation. See entry points at
https://github.com/OSGeo/PROJ/blob/master/src/networkfilemanager.cpp#L1907

When using that default implementation, you can set the PROJ_CURL_VERBOSE 
environment variable (to any value), and you'll get curl verbose traces for requests issued.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/proj/attachments/20200713/f31d9d04/attachment.html>