[gdal-dev] VRT OGRVRTDataSource filter & python function

[Seth G]

Hi,

>From the MapServer side, did you try a runtime substitution parameter in the VRT file? There is a chance it loads the file and does the replacement afterwards, although it probably just sends on the filename to GDAL. 

The FILTER is indeed processed in MapServer. Does the PROCESSING 'NATIVE_FILTER=id=234' approach work?

Depending on how many unique values you have, you could generate multiple VRTs and use runtime substitution to load the correct one. 

Seth

--
mastodon: @geographika at mastodon.social

On Tue, Jul 29, 2025, at 12:18 AM, Peter Schmitt via gdal-dev wrote:
> Hi,
> 
> I have two GDAL questions that are Mapserver adjacent (or perhaps vice-versa?).   
> 
> 1.  Does OGRVRTDataSource allow Python code similar to raster VRT Python Pixel Functions?
> 2.  Is there a reasonable way to pass custom arguments to a VRT?
> 
> 
> I have a VRT that executes some SQL:
> 
>     <!-- file.vrt -->
>     <OGRVRTDataSource>
>       <OGRVRTLayer name="0">
>         <SrcDataSource>PG:host=db user=me dbname=datastore password=abc</SrcDataSource>
>         <SrcSQL>
>           SELECT some_table.location, some_table.id, some_table.geom FROM some_table
>         </SrcSQL>
>       </OGRVRTLayer>
>     </OGRVRTDataSource>
> 
> Is there a reasonable way to pass some argument to filter the VRT?  For example, a condition like `WHERE some_table.id = 123`.  Imagine the query has a very large number of records. I dont want to process the filter "client side":  Ideally the condition gets executed in SQL before OGR gets any results.  I am aware of ogr2ogr's -sql statement https://gdal.org/en/stable/programs/ogr2ogr.html#cmdoption-ogr2ogr-sql
> 
> Ultimately this will happen via MapServer, which is referencing the VRT layer.  I would like the client to pass a QUERYSTRING argument `foo_id=123` to perform a filter on the data in the VRT.  Usually I do this via MapServer Runtime substitution.  This works if I embed the contents of file.vrt directly in my Mapfile 
> 
>     LAYER
>         NAME my_layer
>         TYPE POLYGON
>         EXTENT -180 -90 180 90
>         PROJECTION
>             "init=epsg:4326"
>         END
>         CONNECTIONTYPE OGR
>         TILEITEM 'location'
>         TILEINDEX "<OGRVRTDataSource>...WHERE some_table.id = %foo_id%</SrcSQL></OGRVRTLayer></OGRVRTDataSource>"
> 
>         VALIDATION
>             'foo_id' "^[0-9]+$"
>             'default_foo_id' "-1"
>         END
>     END
> 
> However, my actual OGRVRTDataSource file is large.  If I inline a large VRT Mapserver complains "msBuildPath(): Unable to access file. Path is too long.  Check server logs.".  I could use a Mapserver FILTER https://mapserver.org/mapfile/layer.html#filter, but I think that occurs AFTER the VRT is evaluated. . . so if there are millions of rows, that data gets filtered by the client (Mapserver) and not by SQL, so there's a lot of data moving around and very slow performance.
> 
> Thanks!
> Pete
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250729/cc4805f2/attachment.htm>