[MapServer-users] Can MapServer cache directly to AWS S3?

[Richard Greenwood]

On Fri, Mar 7, 2025 at 7:29 AM Rob Dennett <Rob.Dennett at twdb.texas.gov>
wrote:

> So, can you have both the MapCache module and the MapServer CGI script
> running on the same Apache server?
>

Sure. There's no conflict.


> Is there an example showing how to configure this?
>

Look at the MapCache documentation. MapCache is configured with an xml
file. If MapServer is on the same server then you'd have a statement in the
the "source" section like:
     <source name="something" type="wms">
          . . .
          <url>http://localhost/cgi-bin/mapserv.fcgi</url>
          . . .
     </source>


> We have a bunch of front ends which use our MapServer (it's
> customer-facing, too) and I am not clear if all those URLs would have to
> change.
>

Your app makes requests to https://youserver/mapcache for tiles or
https://yourserver/cgi-bin/mapserv.fcgi for non-tiles requests. MapCache is
reading and writing to and from S3, your app is not directly access S3.

Rich



> Thanks,
> Rob
> ------------------------------
> *From:* Richard Greenwood <richard.greenwood at gmail.com>
> *Sent:* Friday, March 7, 2025 7:44 AM
> *To:* Rob Dennett <Rob.Dennett at twdb.texas.gov>
> *Cc:* mapserver <mapserver-users at lists.osgeo.org>
> *Subject:* Re: [MapServer-users] Can MapServer cache directly to AWS S3?
>
>
> External: Beware of links/attachments.
>
>
>
> On Thu, Mar 6, 2025, 3:27 PM Rob Dennett via MapServer-users <
> mapserver-users at lists.osgeo.org> wrote:
>
> Does MapCache have to be built from source?
>
>
> Probably not. If you're on Linux it will be in your distribution's repo.
> Not sure about Windows or Mac.
>
> Also, does it have to be it's own server?
>
>
> MapCache is a module that the Apache web server runs.
>
>   It's not clear to me how it connects with MapServer.  Is it the case
> that MapServer renders tiles as necessary based on requests and then
> MapCache serves those files if requested again?
>
>
> No. Apache passes tile requests to MapCache. If MapCache has the tile it
> gets returned. If MapCache doesn't have the tile it makes a WMS request to
> MapServer, saves the tile in its cache and returns the tile to the web
> server. MapCache can save the tile on the local file system, S3 or
> elsewhere. You configure where it saves the cached tiles.
>
> As mentioned, MapServer is currently storing tiles in a directory mapped
> to S3 which is specified by IMAGEPATH.
>
>
> Using MapCache you don't need IMAGEPATH.
>
>
> These tiles are not pre-seeded, so I can't specify a particular raster or
> vector file to serve from S3.  I don't think I understand how caching is
> supposed to work with S3 in the way we're doing it.  Can you simply specify
> a folder in S3 where cached data lives and have MapServer render anything
> that doesn't exist yet?
>
>
> If you are trying to serve tiles directly from S3 by making tile requests
> directly to the S3 bucket that is a whole different thing and not supported
> by MapCache.
>
> HTH,
> Rich
>
>
>
> Thanks,
> Rob
> ------------------------------
> *From:* MapServer-users <mapserver-users-bounces at lists.osgeo.org> on
> behalf of Scott via MapServer-users <mapserver-users at lists.osgeo.org>
> *Sent:* Wednesday, March 5, 2025 11:50 AM
> *To:* mapserver-users at lists.osgeo.org <mapserver-users at lists.osgeo.org>
> *Subject:* Re: [MapServer-users] Can MapServer cache directly to AWS S3?
>
> External: Beware of links/attachments.
>
>
>
> You can store any raster/vector file format supported by GDAL on S3 and
> use it via MapServer. Note the /vsis3 virtual connection:
>
> Raster:
> DATA "/vsis3/myS3Bucket/raster.tif"
>
> Vector:
> CONNECTION "/vsis3/myS3Bucket/vector.fgb"
>
> MapServer is definitely a WMS server. Any OGC compliant WMS url can be
> parsed and processed. Using MapCache along MapServer is highly desirable
> as all subsequent WMS requests are cached. Further, you can seed any
> cache directly with MapCache.
>
> Scott
>
>
> On 3/5/25 07:57, Rob Dennett via MapServer-users wrote:
> > Is it possible to cache tiles  to and serve them from S3 buckets?  We
> > currently mount S3 folders in EC2 instances one of which is the
> > directory we specify in MAP/IMAGEPATH, but it requires us to use s3fs-
> > fuse and that means we have to use EC2 as opposed to Fargate, which we
> > would prefer.  Can MapServer act as a client to S3?
> >
> > I have been looking at MapCache, and as far as I can tell, it's a
> > separate server that works alongside a WMS server, although I am not
> > sure that MapServer counts as a WMS server.  We use it to serve geoJSON,
> > UTF grid, etc.  Is it what we want to use for this?
> >
> > Thanks,
> > Rob
> >
> > _______________________________________________
> > MapServer-users mailing list
> > MapServer-users at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
> _______________________________________________
> MapServer-users mailing list
> MapServer-users at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
> _______________________________________________
> MapServer-users mailing list
> MapServer-users at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>

-- 
Richard W. Greenwood
www.greenwoodmap.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20250307/a878c8f9/attachment.htm>