[gdal-dev] Call for review and discussion on RFC96: Deferred in-tree C++ plugin loading

[Even Rouault]

Hi,

The OSGeo code sprint last week has been the opportunity to advance the 
candidate implementation of the RFC. It is now available at 
https://github.com/OSGeo/gdal/pull/8695 :

  * All drivers that can be built as plugins and depend on external
    libraries have been converted to use the deferred plugin loading
    capability. Plus a few other drivers that only depend on core
  * The PNG, JPEG, GIF, MRF, NITF and planetary (PDS, PDS4, ISIS2,
    ISIS3, VICAR) drivers can also now be built as plugins (when not
    using the in-tree vendored libraries such as libpng, libjpeg, etc.),
    and use deferred plugin loading

On my full configuration with all drivers that can be built as plugins 
effectively built as plugins - that is those that only depend on 
libgdal, with -DGDAL_ENABLE_PLUGINS_NO_DEPS=ON and those that depend on 
third-party libraries, with -DGDAL_ENABLE_PLUGINS=ON -, the 
initialization time has dropped from ~ 300 ms to 70 ms. If building only 
as plugins drivers that depend on third-party libraries with 
-DGDAL_ENABLE_PLUGINS=ON (which is the config that makes most sense), it 
drops to 26 ms, with all 55 plugins in deferred loading mode. One of the 
biggest culprit identified during the conversion process was the DGNv8 / 
DWG driver wich links to gazillion shared libraries of the Teigha SDK.

I've also added a bit of extra user friendliness:

    When a plugin driver is known of core libgdal, but not available as 
a plugin at
     runtime, GDAL will inform the user that the plugin is not 
available, but could
     be installed. It is possible to give more hints on how to install a 
plugin
     by setting the following option:

     .. option:: 
GDAL_DRIVER_<driver_name>_PLUGIN_INSTALLATION_MESSAGE:STRING

     .. option:: OGR_DRIVER_<driver_name>_PLUGIN_INSTALLATION_MESSAGE:STRING

         Custom message to give a hint to the user how to install a 
missing plugin


     For example, if doing a build with::

         cmake .. -DOGR_DRIVER_PARQUET_PLUGIN_INSTALLATION_MESSAGE="You 
may install it with with 'conda install -c conda-forge 
libgdal-arrow-parquet'"

     and opening a Parquet file while the plugin is not installed will 
display the
     following error::

         $ ogrinfo poly.parquet
         ERROR 4: `poly.parquet' not recognized as a supported file 
format. It could have been recognized by driver Parquet, but plugin 
ogr_Parquet.so is not available in your installation. You may install it 
with with 'conda install -c conda-forge libgdal-arrow-parquet'

I'll submit soon the RFC to vote if there are no further remarks.

Even

Le 06/11/2023 à 14:50, Even Rouault via gdal-dev a écrit :
> Hi,
>
> I've revised a bit the RFC 
> (https://github.com/OSGeo/gdal/pull/8648/commits/b7053db2f433275edf16286596e71f3ceb9738a5) 
> to unify the way the plugin driver proxy and the actual driver declare 
> their metadata. This avoids the new GDALPluginDriverFeatures class, 
> and will limit the risk of omissions or inconsistencies between the 
> proxy and actual drivers.
>
> Even
>
> Le 02/11/2023 à 12:59, Even Rouault via gdal-dev a écrit :
>> Hi,
>>
>> I'm seeking for feedback and review on a new RFC (RFC 96: Deferred 
>> in-tree C++ plugin loading),
>> detailed in https://github.com/OSGeo/gdal/pull/8648, whose summary is:
>>
>> This RFC adds a mechanism to defer the loading of in-tree C++ plugin 
>> drivers to
>> the point where their executable code is actually needed, and 
>> converts a number
>> of relevant drivers to use that mechanism. The aim is to allow for 
>> more modular
>> GDAL builds, while improving the performance of plugin loading.
>>
>> (This is material only for GDAL 3.9 of course)
>>
>> Even
>>
-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20231113/c56af9a9/attachment.htm>