<div dir="ltr">I'd agree this is a bug - just not sure what the fix should be. </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Sep 6, 2021 at 4:21 AM Sommer, Ashley (L&W, Dutton Park) <Ashley.Sommer@csiro.au> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">




<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div>Hi Everyone,</div>
<div>I'm trying to get WCS service with TIME parameter support working on latest Mapserver.</div>
<div>My data is a timeseries of large geotiff files, in a directory, with the date encoded in the filename.</div>
<div>Eg:</div>
<div> - /datasets/mydataset/v1/layer1/raster_2020-08-01.tif</div>
<div> - /datasets/mydataset/v1/layer1/raster_2020-08-02.tif</div>
<div> - etc.</div>
<div>I'm had it working on the WMS side of things, simply using Runtime Substitution, and embedding the TIME parameter into the DATA directive.</div>
<div>eg:</div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  DATA "/datasets/mydataset/v1/layer1/raster_%time%.tif"</span></div>
<div>That worked for WMS, but not for WCS. When Mapserver's WCS service encounters a TIME parameter, it assumes I'm using a tile index. If there is no tile index, it errors out.</div>
<div><br>
</div>
<div>So I create a tile index for this dataset. Using gdaltindex for a record of "location" for each raster, then adding a temporal attribute to the dbf file, and assigning dates to each of the records.</div>
<div>I then tried to use that in place of DATA:</div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">LAYER</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  NAME "mylayer"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEINDEX "/datasets/mydataset/v1/layer1/index.shp"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEITEM "location"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">END</span></div>
<div>Unfortunately, that doesn't work, because Mapserver now requires the use of new-style Layer-Tile-Index directives in order for WCS to work properly.</div>
<div>So I changed it to:</div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">LAYER</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  NAME "MyIndex"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEINDEX "/datasets/mydataset/v1/layer1/index.shp"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEITEM "location"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">END</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">LAYER</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEINDEX "MyIndex"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">  TILEITEM "location"</span></div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">END</span></div>
<div><br>
</div>
<div>This now works to some extent, but then I get an error:</div>
<div><span style="font-size:11pt;font-family:"Courier New",monospace">msTiledSHPTryOpen(): Unable to access file. Unable to open shapefile 'raster_2020-08-02.tif' for layer 'MyIndex' ... fatal error. msShapefileOpen(): Unable to access file. (/tmp/raster_2020-08-02.tif)
 msShapefileOpen(): Unable to access file. (/datasets/mydataset/v1/layer1/raster_2020-08-01.tif)</span>
</div>
<div><br>
</div>
<div>I believe I have tracked down why this is happening, and I believe it is a bug.</div>
<div>1) Mapserver _can_ open the .shp and .dbf file at the location. It must be able to to get the file location from the dbf index.</div>
<div>2) msTiledSHPTryOpen() is a helper fn that runs _after_ opening a tile index, to test if we can open the first-indexed file (to template some parts of the layer).</div>
<div>See comment here:<a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L2028" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L2028" target="_blank"> https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L2028</a></div>
<div>See also msTileSHPTryOpen fn here: <a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1868" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1868" target="_blank">
https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1868</a></div>
<div>3) So msTiledSHPTryOpen() tries to open the first-indexed file: "/datasets/mydataset/v1/layer1/raster_2020-08-02.tif" but returns an error, even though it _can_ open that file.</div>
<div>The problem is, it is using msShapefileOpen() to do this, which in-turn uses msSHPOpen() to open the tif file:</div>
<div>See msShapefileOpen fn: <a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1665" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1665" target="_blank">
https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1665</a></div>
<div>See opener: <a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1686" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1686" target="_blank">
https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L1686</a></div>
<div>4) msSHPOpen() assumes whatever file its opening will have a .shp file and a .shx file.</div>
<div>In this case, it's trying to open a .tif file. The function tries to open a non-existent /datasets/mydataset/v1/layer1/raster_2020-08-02.shp file.</div>
<div>See the implementation here: <a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L194" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L194" target="_blank">
https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L194</a></div>
<div>And where it opens a file here: <a href="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L252" title="https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L252" target="_blank">
https://github.com/MapServer/MapServer/blob/0d156a72893b1df712cb49c1b3ddaeed5365a68b/mapshape.c#L252</a></div>
<div><br>
</div>
<div>So I believe it is a bug to use msShapefileOpen() to test if a tileindex can open a file at its first-indexed location, because that file may not be a .shp file.</div>
<div><br>
</div>
<div>I might be able to put in a PR to fix the bug, but I don't know the codebase well enough to not potentially introduce regressions.</div>
Let me know what you think?<br>
<br>
- Ashley Sommer</div>
</div>

_______________________________________________<br>
MapServer-users mailing list<br>
<a href="mailto:MapServer-users@lists.osgeo.org" target="_blank">MapServer-users@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/mapserver-users" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a><br>
</blockquote></div>