[gdal-dev] Logging network requests/replies?

[Sean Gillies]

Hi Even,

On Fri, Feb 8, 2019 at 4:59 AM Even Rouault <even.rouault at spatialys.com>
wrote:

> > Actually, what I had in mind (thinking out loud) would sit solely
> > within GDAL, e.g. a method to globally specify a callback function to
> > be called whenever GDAL creates a network request, e.g. within
> > CPLHTTPFetchEx (and similarly for replies).
>
> CPLHTTPFetchEx() is just one of the site that does network access.
> cpl_vsil_curl.cpp, cpl_vsil_s3.cpp, cpl_vsil_az.cpp, cpl_vsil_swift.cpp,
> cpl_vsil_webhdfs.cpp also do network access
>
> But at the end of all them must go through CPLHTTPSetOptions() in
> cpl_http.cpp
>
> > (With clients being
> > responsible for ensuring that their callback functions are
> > appropriately thread safe).
> >
> > Does this sound desirable? Any concerns/blockers before I start hacking
> on
> > it?
>
> I had several thoughts about this
>
> 1) yes a dedicated callback mechanism could work of course
>
> 2) As Sean mentionned, and provided we redirect curl logging to our own
> callbacks instead of stderr, we could potentially just use the CPLDebug()
> infrastructure with the the callback being installed with
> CPLSetErrorHandler()
> + CPLSetCurrentErrorHandlerCatchDebug(true). We probably wouldn't want so
> much
> debugging information to be emitted by default (both because of verbosity
> and
> performance), so this should still be conditionalized by the
> CPL_CURL_VERBOSE=YES config option/env. variable
>
> 3) An alternative would be to have a dedicated generic logging API
>
> void CPLLogEvent(const char* topic, const char* fmt, ...);
>
> int CPLHasLoggerFor(const char* topic);
>
> typedef void (*CPLLoggerCallback)(const char* topic,
>
>        const char* msg, void* user_data);
>
> void* CPLInstallLogger(const char* topic,
>                                                         CPLLoggerCallback
> cbk, void* user_data);
> void CPLUninstallLogger(void*);
>
> but I'm wondering if there wouldn't be too much confusion/intersection
> with
> the CPLDebug() mechanism.
>
>
> My current slight preference would go to 2) to reuse existing
> infrastructure.
> Maybe others, including Robert CC'ed who has hacked in the error/debug
> area
> recently, have thoughts on this.
>
>
> Even
>

I'd prefer to see the existing infrastructure reused because then some
logging of network activity would automatically appear for users of
Rasterio (which already subscribes to CPLDebug messages).

I think the right amount of default logging would be to call CPLDebug for
new connections and report the URL, but not necessarily call CPLDebug for
every request made using that connection.

-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20190208/886df784/attachment-0001.html>