[mapserver-users] Mapserver vsis3 perfomance

[Peter Schmitt]

Hi Elari,

I'm the author of that repo.  The format of the input raster data is
critical to achieve optimal /vsis3/ performance. Much more so than
multithreading/multiprocessing configuration.  I have had the best
performance serving raster data formatted as a Cloud Optimized GeoTIFF
(COG).  Newer versions of GDAL have a COG driver
https://gdal.org/drivers/raster/cog.html . You can convert your raster data
to a COG like so:

> gdal_translate input.jp2 cog.tif -of COG

A couple of GDAL config options can substantially improve Mapserver
performance reading data from /vsicurl/ filesystems. Here are some settings
you can put in your Mapserver mapfile:

CONFIG "CPL_VSIL_CURL_ALLOWED_EXTENSIONS" "tif vrt gpkg zip shp shx dbf qix
cpg prj"
CONFIG "GDAL_DISABLE_READDIR_ON_OPEN"     "YES"

There are a lot of other GDAL config options to improve performance.  Here
is a good writeup that closely matches my experience serving imagery stored
by a cloud computing provider like S3:
https://developmentseed.org/titiler/advanced/performance_tuning/

Cheers,
Pete


On Fri, Jul 22, 2022 at 1:12 AM Elari Roop <elari.roop at gmail.com> wrote:

> Hi all,
>
> I am rather new to mapserver and I am trying to read mosaic from S3, but I
> have run into perfomance issues.
>
> https://github.com/pedros007/mapserver-docker
>
>
> I folowed the instructions above, but it seems that increasing number of
> threads below does not have any impact on perfomance.
>
>> CONFIG "GDAL_NUM_THREADS" "50"
>
>
> Could it be that the mapserver is reading data with only one thread or
> could this be related to something else like overviews configuration,
> reading from tileindex shp file etc?
>
> Best regards,
> Elari
> _______________________________________________
> MapServer-users mailing list
> MapServer-users at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>


-- 
Pete
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20220722/77d5a797/attachment.htm>