From mniemyjski at cloudferro.com Mon Jul 3 06:04:02 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Mon, 3 Jul 2023 13:04:02 +0000 Subject: [MapServer-users] Problems using mask and time dimension Message-ID: Hello, I have a problem: I have created the following mapfile:https://paste.ofcode.org/ianeSsdL45hJtDgh7SxMEm it is based on the tileindex and time dimension. Along with Sentinel-2, cloud mask is attached in the form of a raster. As you can see, one of the attributes of my tile index is the path to the mask raster, from which I extract clouds and their shadows to use the MASK and mark clouds in the images. And it works. The problem is that when using the time dimension, the cloud mask is also applied to images from other dates where there are no clouds (there are only clouds for one specific date) :( The question is: How can I apply the mask only to the raster of each individual date for which it was created, so that I can effectively use masking and the time dimension? Best, Marcin [cid:2b3cfdde-291a-4d0a-9225-7199f357600a] Book time to meet with me -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Outlook-oz0pftj1.png Type: image/png Size: 528 bytes Desc: Outlook-oz0pftj1.png URL: From james.f.farmer at nasa.gov Mon Jul 3 08:59:46 2023 From: james.f.farmer at nasa.gov (Farmer, James F. (LARC-D207)[RSES]) Date: Mon, 3 Jul 2023 15:59:46 +0000 Subject: [MapServer-users] HTTPS Broken Message-ID: Hello all, I support an application that sources tiles from vmap0.tiles.osgeo.org. Recently we noticed our requests failing. They return: This site can't provide a secure connection vmap0.tiles.osgeo.org uses an unsupported protocol. ERR_SSL_VERSION_OR_CIPHER_MISMATCH I can access requested tiles over http manually, but our app requires https and at some point the tile server lost its ability to do that. Is it possible for this functionality to be returned? * Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From kathleen.hirst at bom.gov.au Wed Jul 5 22:29:18 2023 From: kathleen.hirst at bom.gov.au (Kathleen Hirst) Date: Thu, 6 Jul 2023 05:29:18 +0000 Subject: [MapServer-users] Suppress queryable=1 for wms_layer_group [SEC=OFFICIAL] Message-ID: Hi, In the previous version of Mapserver when using wms_layer_group, the groups were not queryable in the GetCapabilities. In following examples, the configuration is LAYER>METADATA> OWS_LAYER_GROUP "/Radar/Australia" Version 6.4.2 Radar Australia IDR00009 Radar Offline | Australia Radar Offline | Australia EPSG:4283 In the latest version 8.0.0, all the groups are queryable Radar Radar Australia Australia IDR00009 Radar Offline | Australia Radar Offline | Australia EPSG:4283 Is there a way to not have the group queryable, as it is causing issues with some web applications eg picking up the wrong layer within a group or listing the group as well as the layer Thanks Kathleen -------------- next part -------------- An HTML attachment was scrubbed... URL: From traviskirstine at gmail.com Thu Jul 6 05:14:30 2023 From: traviskirstine at gmail.com (Travis Kirstine) Date: Thu, 6 Jul 2023 08:14:30 -0400 Subject: [MapServer-users] Suppress queryable=1 for wms_layer_group [SEC=OFFICIAL] In-Reply-To: References: Message-ID: Hi Kathleen, I haven't experimented with GROUPed layers but have you looked at *wms_enable_request* (or *ows_enable_request*) defined in the METADATA section of the layer "wms_enable_request" "* !GetFeatureInfo" See docs: https://mapserver.org/ogc/wms_server.html#web-object-metadata Regards On Thu, 6 Jul 2023 at 01:36, Kathleen Hirst wrote: > Hi, > > > > In the previous version of Mapserver when using wms_layer_group, the > groups were not queryable in the GetCapabilities. In following examples, > the configuration is > > > > LAYER>METADATA> OWS_LAYER_GROUP "/Radar/Australia" > > > > > > Version 6.4.2 > > > > > > Radar > > > > Australia > > > > IDR00009 > > Radar Offline | Australia > > Radar Offline | Australia > > EPSG:4283 > > > > > > In the latest version 8.0.0, all the groups are queryable > > > > > > Radar > > Radar > > > > Australia > > Australia > > > > IDR00009 > > Radar Offline | Australia > > Radar Offline | Australia > > EPSG:4283 > > > > Is there a way to not have the group queryable, as it is causing issues > with some web applications eg picking up the wrong layer within a group or > listing the group as well as the layer > > > > Thanks > > > > Kathleen > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Thu Jul 6 06:39:29 2023 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Thu, 6 Jul 2023 13:39:29 +0000 Subject: [MapServer-users] Suppress queryable=1 for wms_layer_group [SEC=OFFICIAL] In-Reply-To: References: Message-ID: Hi, Do you use the same mapfile? There is a difference in GetCapabilities Radar Radar If layer has just a title but no name it is not queryable, nor does it support GetMaps. This applies to group layers as well. But it may be that since version 7.2 it is not possible to configure a WMS group without a name https://mapserver.org/ogc/wms_server.html. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Kathleen Hirst L?hetetty: torstai 6. hein?kuuta 2023 8.29 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: [MapServer-users] Suppress queryable=1 for wms_layer_group [SEC=OFFICIAL] Hi, In the previous version of Mapserver when using wms_layer_group, the groups were not queryable in the GetCapabilities. In following examples, the configuration is LAYER>METADATA> OWS_LAYER_GROUP "/Radar/Australia" Version 6.4.2 Radar Australia IDR00009 Radar Offline | Australia Radar Offline | Australia EPSG:4283 In the latest version 8.0.0, all the groups are queryable Radar Radar Australia Australia IDR00009 Radar Offline | Australia Radar Offline | Australia EPSG:4283 Is there a way to not have the group queryable, as it is causing issues with some web applications eg picking up the wrong layer within a group or listing the group as well as the layer Thanks Kathleen -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.basques at ci.stpaul.mn.us Fri Jul 7 07:19:00 2023 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Fri, 7 Jul 2023 14:19:00 +0000 Subject: [MapServer-users] [ANNOUNCE] OSGeo Local Chapter (Twin Cities) Meeting, July 12th, 2023 Message-ID: <50C47B85-EDB4-451E-B1BF-DEB7AB93F982@ci.stpaul.mn.us> All, There were some interesting topics and discussion at the last meeting. Don?t miss out. Face to face meeting this month will be at the Lift Bridge Brewing Co. As always, any presentations on geo related projects, tips and tricks, or just plain old showing stuff off are welcome. I can set up a virtual presentation for those in attendance and handle the recording aspects. Place: Lift Bridge Brewing Co. https://goo.gl/maps/jcJ4n7pfMNTAj3dx5 Date: July 12th, 2023 Time: 4:30 pm ? 6:00 Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mh at mapcentia.com Tue Jul 11 09:59:29 2023 From: mh at mapcentia.com (=?UTF-8?Q?Martin_H=c3=b8gh?=) Date: Tue, 11 Jul 2023 18:59:29 +0200 Subject: [MapServer-users] Some remote SVG files doesn't in 8.1 Message-ID: <91fc9317-6a77-26b0-4f39-460c611e6c35@mapcentia.com> I've a MapFile with a point layer using a class style with a SVG symbol: CLASS ??? NAME 'point' ??? STYLE ??? ??? SYMBOL 'https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg' ??? END END In 7.6 this works fine, but in 8.1 the SVG symbol is not showing (no errors but the image is blank). But if change the SVG file to e.g.: https://dev.w3.org/SVG/tools/svgweb/samples/svg-files/AJ_Digital_Camera.svg it works. So I guess it's something with the SVG file? MapServer version 8.0.1 PROJ version 7.2 GDAL version 3.4 OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=OGCAPI_SERVER SUPPORTS=FASTCGI SUPPORTS=GEOS SUPPORTS=PBF INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE INPUT=FLATGEOBUF From bob.basques at ci.stpaul.mn.us Tue Jul 11 20:26:13 2023 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Wed, 12 Jul 2023 03:26:13 +0000 Subject: [MapServer-users] [REMINDER] OSGeo Local Chapter (Twin Cities) Meeting, July 12th, 2023 In-Reply-To: <50C47B85-EDB4-451E-B1BF-DEB7AB93F982@ci.stpaul.mn.us> References: <50C47B85-EDB4-451E-B1BF-DEB7AB93F982@ci.stpaul.mn.us> Message-ID: All, There were some interesting topics and discussion at the last meeting. Don?t miss out. Face to face meeting this month will be at the Lift Bridge Brewing Co. As always, any presentations on geo related projects, tips and tricks, or just plain old showing stuff off are welcome. I can set up a virtual presentation for those in attendance and handle the recording aspects. Place: Lift Bridge Brewing Co. https://goo.gl/maps/jcJ4n7pfMNTAj3dx5 Date: July 12th, 2023 Time: 4:30 pm ? 6:00 Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Fri Jul 14 05:55:03 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Fri, 14 Jul 2023 12:55:03 +0000 Subject: [MapServer-users] Problems using masking and time dimension part 2 Message-ID: Hi, I have an idea, but I don't know how to implement it: https://marc.info/?l=mapserver-users&m=168838940628381&w=2 In my previous email, I asked about the possibility of combining WMS with a time dimension with masking. I would like the mask of a particular day not to cut out the data in rasters from other days, but unfortunately it is cutting them out, and I have something like this: https://pastecode.io/s/vwfwdtqe I came up with the idea of creating a VRT in which the RGB raster would be marked and the indicated raster with the cloud mask as an alpha channel, but I would like to keep all of this only as paths or attributes in PostGIS. And here is my question, whether I could somehow (so that I don't have to create a file, I know that it is possible to define VRT in the mapfile) store the content of VRT (with the defined alpha channel) in tileindex in POstgis? I also have a request to send other ideas on how to use masking in WMS-Time. Thank you for your ideas. Best regards, Marcin -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Mon Jul 17 03:22:27 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Mon, 17 Jul 2023 10:22:27 +0000 Subject: [MapServer-users] Combination of MASK, TIME and TILEINDEX Message-ID: Hello, Is it possible to apply a MASK on a layer but only on selected objects? I'm thinking of something like this: PROCESSING "MASK WHERE ["layer_attribute"] = ["mask_attribute"] Both layers are stored in the same tile index. In short, it looks like this: date | path_to_layer | path_to_mask What I'm trying to do is to mask rasters using their corresponding masks. Specifically, I'm trying to avoid a situation where a raster from 2023-01-01 has burnt-out holes when using the mask from 2023-01-07. Please help and provide some guidance. My previous messages can be found in the thread "Problems using masking and time dimension part 2." Best, Marcin [cid:0f22f998-8944-4b2c-9a54-0d1c0cad504e] Book time to meet with me -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Outlook-luq2hkvn.png Type: image/png Size: 528 bytes Desc: Outlook-luq2hkvn.png URL: From sdlime at gmail.com Mon Jul 17 16:59:19 2023 From: sdlime at gmail.com (Steve Lime) Date: Mon, 17 Jul 2023 18:59:19 -0500 Subject: [MapServer-users] Some remote SVG files doesn't in 8.1 In-Reply-To: <91fc9317-6a77-26b0-4f39-460c611e6c35@mapcentia.com> References: <91fc9317-6a77-26b0-4f39-460c611e6c35@mapcentia.com> Message-ID: There seems to be a regression with svg- I have another sample file that doesn?t render correctly (or at all). Just need to find a little time to resolve it. On Tue, Jul 11, 2023 at 12:10 PM Martin H?gh wrote: > I've a MapFile with a point layer using a class style with a SVG symbol: > > CLASS > NAME 'point' > STYLE > SYMBOL > ' > https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg > ' > END > END > > In 7.6 this works fine, but in 8.1 the SVG symbol is not showing (no > errors but the image is blank). But if change the SVG file to e.g.: > > https://dev.w3.org/SVG/tools/svgweb/samples/svg-files/AJ_Digital_Camera.svg > > it works. So I guess it's something with the SVG file? > > MapServer version 8.0.1 PROJ version 7.2 GDAL version 3.4 OUTPUT=PNG > OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE > SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV > SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT > SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER > SUPPORTS=SOS_SERVER SUPPORTS=OGCAPI_SERVER SUPPORTS=FASTCGI > SUPPORTS=GEOS SUPPORTS=PBF INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL > INPUT=SHAPEFILE INPUT=FLATGEOBUF > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Wed Jul 19 14:54:39 2023 From: sethg at geographika.co.uk (Seth G) Date: Wed, 19 Jul 2023 23:54:39 +0200 Subject: [MapServer-users] Some remote SVG files doesn't in 8.1 In-Reply-To: References: <91fc9317-6a77-26b0-4f39-460c611e6c35@mapcentia.com> Message-ID: Hi Martin, The issue seems to be related to the viewBox property in the SVG file. If you save the SVG source from view-source:https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg and remove the following text in a text editor the symbols display correctly: viewBox="0 0 396.85 396.85" Maybe it could be an upstream issue, or something has changed upstream that breaks symbol size calculations. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Tue, Jul 18, 2023, at 1:59 AM, Steve Lime wrote: > There seems to be a regression with svg- I have another sample file that doesn?t render correctly (or at all). Just need to find a little time to resolve it. > > On Tue, Jul 11, 2023 at 12:10 PM Martin H?gh wrote: >> I've a MapFile with a point layer using a class style with a SVG symbol: >> >> CLASS >> NAME 'point' >> STYLE >> SYMBOL >> 'https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg' >> END >> END >> >> In 7.6 this works fine, but in 8.1 the SVG symbol is not showing (no >> errors but the image is blank). But if change the SVG file to e.g.: >> >> https://dev.w3.org/SVG/tools/svgweb/samples/svg-files/AJ_Digital_Camera.svg >> >> it works. So I guess it's something with the SVG file? >> >> MapServer version 8.0.1 PROJ version 7.2 GDAL version 3.4 OUTPUT=PNG >> OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE >> SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV >> SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT >> SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER >> SUPPORTS=SOS_SERVER SUPPORTS=OGCAPI_SERVER SUPPORTS=FASTCGI >> SUPPORTS=GEOS SUPPORTS=PBF INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL >> INPUT=SHAPEFILE INPUT=FLATGEOBUF >> >> >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/mapserver-users > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Thu Jul 20 00:52:48 2023 From: sethg at geographika.co.uk (Seth G) Date: Thu, 20 Jul 2023 09:52:48 +0200 Subject: [MapServer-users] Some remote SVG files doesn't in 8.1 In-Reply-To: References: <91fc9317-6a77-26b0-4f39-460c611e6c35@mapcentia.com> Message-ID: Adding a width and height to the SVG also works, without having to remove the viewBox: Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 19, 2023, at 11:54 PM, Seth G wrote: > Hi Martin, > > The issue seems to be related to the viewBox property in the SVG file. > If you save the SVG source from view-source:https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg and remove the following text in a text editor the symbols display correctly: > > viewBox="0 0 396.85 396.85" > > Maybe it could be an upstream issue, or something has changed upstream that breaks symbol size calculations. > > Seth > > > -- > web:https://geographika.net & https://mapserverstudio.net > twitter: @geographika > > On Tue, Jul 18, 2023, at 1:59 AM, Steve Lime wrote: >> There seems to be a regression with svg- I have another sample file that doesn?t render correctly (or at all). Just need to find a little time to resolve it. >> >> On Tue, Jul 11, 2023 at 12:10 PM Martin H?gh wrote: >>> I've a MapFile with a point layer using a class style with a SVG symbol: >>> >>> CLASS >>> NAME 'point' >>> STYLE >>> SYMBOL >>> 'https://raw.githubusercontent.com/NordjyllandsTrafikselskab/gisimages/ca64d7a849c7cf0290ec41ec263a97b9195d0ff8/01.01.01.Bus.svg' >>> END >>> END >>> >>> In 7.6 this works fine, but in 8.1 the SVG symbol is not showing (no >>> errors but the image is blank). But if change the SVG file to e.g.: >>> >>> https://dev.w3.org/SVG/tools/svgweb/samples/svg-files/AJ_Digital_Camera.svg >>> >>> it works. So I guess it's something with the SVG file? >>> >>> MapServer version 8.0.1 PROJ version 7.2 GDAL version 3.4 OUTPUT=PNG >>> OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE >>> SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV >>> SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT >>> SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER >>> SUPPORTS=SOS_SERVER SUPPORTS=OGCAPI_SERVER SUPPORTS=FASTCGI >>> SUPPORTS=GEOS SUPPORTS=PBF INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL >>> INPUT=SHAPEFILE INPUT=FLATGEOBUF >>> >>> >>> _______________________________________________ >>> MapServer-users mailing list >>> MapServer-users at lists.osgeo.org >>> https://lists.osgeo.org/mailman/listinfo/mapserver-users >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/mapserver-users >> > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Wed Jul 26 02:53:46 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Wed, 26 Jul 2023 09:53:46 +0000 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Message-ID: Hello, I have a problem, Namely, I have created a WCS and here is its mapfile: https://pastecode.io/s/6v3qfssb For a better understanding, here is a view of the tileindex through which the data is served: https://pasteboard.co/ydgvDiqxz4lf.png I am connecting to files located in a private bucket, so I am using vsis3. The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: https://pasteboard.co/8G8uF41jqgHc.png It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). Additionally, I am sending you URLs to sample data: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 The case is - how to speed it up? Am I making any goofy mistakes? I would be grateful for any recommendations or if you notice any errors, I might be making :) Best, Marcin -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Wed Jul 26 03:58:59 2023 From: sethg at geographika.co.uk (Seth G) Date: Wed, 26 Jul 2023 12:58:59 +0200 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? In-Reply-To: References: Message-ID: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> Hi Marcin, As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct): EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088 If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote: > Hello, > > I have a problem, Namely, I have created a WCS and here is its mapfile: > https://pastecode.io/s/6v3qfssb > > For a better understanding, here is a view of the tileindex through which the data is served: > https://pasteboard.co/ydgvDiqxz4lf.png > > I am connecting to files located in a private bucket, so I am using vsis3. > > The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: > https://pasteboard.co/8G8uF41jqgHc.png > > It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). > > Additionally, I am sending you URLs to sample data: > https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 > https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 > > The case is - how to speed it up? Am I making any goofy mistakes? > I would be grateful for any recommendations or if you notice any errors, I might be making :) > > Best, > Marcin > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Wed Jul 26 05:05:57 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Wed, 26 Jul 2023 12:05:57 +0000 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? In-Reply-To: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> References: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> Message-ID: Hi Seth, as you suggested, I've added the Extent parameter to all of my layers, and it resulted in decreasing time to 5m33.534s. But still, it's quite a lot of time for generating 1.5 MiB file. Any other suggestions? Maybe there is something wrong with projection so beyond just serving data mapserver have to reproject it multiple times? (more than one time by using tilesrs option)? Thanks, Marcin ________________________________ From: Seth G Sent: Wednesday, July 26, 2023 12:58 PM To: Marcin Niemyjski ; MapServer Users Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Marcin, As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct): EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088 If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote: Hello, I have a problem, Namely, I have created a WCS and here is its mapfile: https://pastecode.io/s/6v3qfssb For a better understanding, here is a view of the tileindex through which the data is served: https://pasteboard.co/ydgvDiqxz4lf.png I am connecting to files located in a private bucket, so I am using vsis3. The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: https://pasteboard.co/8G8uF41jqgHc.png It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). Additionally, I am sending you URLs to sample data: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 The case is - how to speed it up? Am I making any goofy mistakes? I would be grateful for any recommendations or if you notice any errors, I might be making :) Best, Marcin _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Wed Jul 26 15:05:40 2023 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Wed, 26 Jul 2023 22:05:40 +0000 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? In-Reply-To: References: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> Message-ID: Hi, Does the tileindex query " DATA 'geometry from (select * from plua order by "maxCC" desc) " return several images from the requested BBOX? I know that MapServer WCS can be fast and we have also a demo at https://beta-karttakuva.maanmittauslaitos.fi/demo/ortokuvat_ja_korkeusmallit/#15.05/67.01553/27.16873 but unfortunately the user interface is only in Finnish. The source images (maybe 10 TB or so) are GeoTIFFs and they are not in the cloud. The time for downloading a small GetCoverage response as GeoTIFF seems to take around 400 milliseconds. We had to write a small service that makes our WCS service to support time as a dimension because MapServer does not support it with WCS https://www.mail-archive.com/mapserver-users at lists.osgeo.org/msg26265.html. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Marcin Niemyjski via MapServer-users L?hetetty: keskiviikko 26. hein?kuuta 2023 15.06 Vastaanottaja: MapServer Users Aihe: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Seth, as you suggested, I've added the Extent parameter to all of my layers, and it resulted in decreasing time to 5m33.534s. But still, it's quite a lot of time for generating 1.5 MiB file. Any other suggestions? Maybe there is something wrong with projection so beyond just serving data mapserver have to reproject it multiple times? (more than one time by using tilesrs option)? Thanks, Marcin ________________________________ From: Seth G > Sent: Wednesday, July 26, 2023 12:58 PM To: Marcin Niemyjski >; MapServer Users > Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Marcin, As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct): EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088 If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote: Hello, I have a problem, Namely, I have created a WCS and here is its mapfile: https://pastecode.io/s/6v3qfssb For a better understanding, here is a view of the tileindex through which the data is served: https://pasteboard.co/ydgvDiqxz4lf.png I am connecting to files located in a private bucket, so I am using vsis3. The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: https://pasteboard.co/8G8uF41jqgHc.png It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). Additionally, I am sending you URLs to sample data: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 The case is - how to speed it up? Am I making any goofy mistakes? I would be grateful for any recommendations or if you notice any errors, I might be making :) Best, Marcin _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Thu Jul 27 02:26:25 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Thu, 27 Jul 2023 09:26:25 +0000 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? In-Reply-To: References: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> Message-ID: Hello Jukka, Yup I guess that's the case, I assumed that WCS has the same capabilities as WMS when it comes to handling time dimension and it would use the default time from mapfile while requesting the data. This is my mapserv log file from one getCoverage request (very small, that's why It took only 30 sec to process it). It would be great if you check it ?: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/log2.txt as you can see mapserver lists all of tile item objects (or at least I guess it does haha) which intersect with requested BBOX. So, this is the problem, that's why it takes so long to generate file. I've got to pass somehow time parameter to request, or at least to query(?) to reduce number of returning shapes. You mentioned your own supporting time WCS service, is it open source? If not, can you give me advice on how to start writing my own? Just some basics ^^ Thanks for your complex answer it helped a lot, Marcin ________________________________ From: Rahkonen Jukka Sent: Thursday, July 27, 2023 12:05 AM To: Marcin Niemyjski ; MapServer Users Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi, Does the tileindex query ? DATA 'geometry from (select * from plua order by "maxCC" desc) ? return several images from the requested BBOX? I know that MapServer WCS can be fast and we have also a demo at https://beta-karttakuva.maanmittauslaitos.fi/demo/ortokuvat_ja_korkeusmallit/#15.05/67.01553/27.16873 but unfortunately the user interface is only in Finnish. The source images (maybe 10 TB or so) are GeoTIFFs and they are not in the cloud. The time for downloading a small GetCoverage response as GeoTIFF seems to take around 400 milliseconds. We had to write a small service that makes our WCS service to support time as a dimension because MapServer does not support it with WCS https://www.mail-archive.com/mapserver-users at lists.osgeo.org/msg26265.html. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Marcin Niemyjski via MapServer-users L?hetetty: keskiviikko 26. hein?kuuta 2023 15.06 Vastaanottaja: MapServer Users Aihe: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Seth, as you suggested, I've added the Extent parameter to all of my layers, and it resulted in decreasing time to 5m33.534s. But still, it's quite a lot of time for generating 1.5 MiB file. Any other suggestions? Maybe there is something wrong with projection so beyond just serving data mapserver have to reproject it multiple times? (more than one time by using tilesrs option)? Thanks, Marcin ________________________________ From: Seth G > Sent: Wednesday, July 26, 2023 12:58 PM To: Marcin Niemyjski >; MapServer Users > Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Marcin, As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct): EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088 If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote: Hello, I have a problem, Namely, I have created a WCS and here is its mapfile: https://pastecode.io/s/6v3qfssb For a better understanding, here is a view of the tileindex through which the data is served: https://pasteboard.co/ydgvDiqxz4lf.png I am connecting to files located in a private bucket, so I am using vsis3. The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: https://pasteboard.co/8G8uF41jqgHc.png It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). Additionally, I am sending you URLs to sample data: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 The case is - how to speed it up? Am I making any goofy mistakes? I would be grateful for any recommendations or if you notice any errors, I might be making :) Best, Marcin _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Thu Jul 27 14:20:38 2023 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Thu, 27 Jul 2023 21:20:38 +0000 Subject: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? In-Reply-To: References: <0936668e-0eff-439c-93d3-db900d57a6a2@app.fastmail.com> Message-ID: Hi, Mapserver is not clever at all when it finds many overlapping images. It renders them all one by one. If the list of images is sorted correctly then the end result is what was desired, but with all too much work. See https://github.com/MapServer/MapServer/issues/5899. We use two tricks: 1) We have computed the time range of validity for each image and we save the start data and end date into two attributes in the tileindex table in a database. Start date is the acquisition date of the image, and end date is the date when the last pixel of the image gets covered by some newer image. If the requested time is not within the range of validity then the image is skipped. Skipping is based on the SQL query that is used as DATA for the tileindex layer. 2) Because time is not supported as a dimension we have written a small utility that is manipulating the incoming http GetCoverage requests before they are sent to Mapserver. The utility converts the time subset, for example SUBSET=time("2012-12-31T02:00:00Z") into something like "&start_date<2012-12-31T02:00:00Z&end_date>2012-12-31T02:00:00Z. The start_date and end_date variables are defined and validated in the mapfile. As a result Mapserver usually does not get overlapping images from the tileindex so it does not need to spend time for multiple rendering either. We have typically about 10 aerial photos from different times at each place and without the above mentioned tweaks the service was all too slow. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Marcin Niemyjski via MapServer-users L?hetetty: torstai 27. hein?kuuta 2023 12.26 Vastaanottaja: MapServer Users Aihe: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hello Jukka, Yup I guess that's the case, I assumed that WCS has the same capabilities as WMS when it comes to handling time dimension and it would use the default time from mapfile while requesting the data. This is my mapserv log file from one getCoverage request (very small, that's why It took only 30 sec to process it). It would be great if you check it ?: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/log2.txt as you can see mapserver lists all of tile item objects (or at least I guess it does haha) which intersect with requested BBOX. So, this is the problem, that's why it takes so long to generate file. I've got to pass somehow time parameter to request, or at least to query(?) to reduce number of returning shapes. You mentioned your own supporting time WCS service, is it open source? If not, can you give me advice on how to start writing my own? Just some basics ^^ Thanks for your complex answer it helped a lot, Marcin ________________________________ From: Rahkonen Jukka > Sent: Thursday, July 27, 2023 12:05 AM To: Marcin Niemyjski >; MapServer Users > Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi, Does the tileindex query ? DATA 'geometry from (select * from plua order by "maxCC" desc) ? return several images from the requested BBOX? I know that MapServer WCS can be fast and we have also a demo at https://beta-karttakuva.maanmittauslaitos.fi/demo/ortokuvat_ja_korkeusmallit/#15.05/67.01553/27.16873 but unfortunately the user interface is only in Finnish. The source images (maybe 10 TB or so) are GeoTIFFs and they are not in the cloud. The time for downloading a small GetCoverage response as GeoTIFF seems to take around 400 milliseconds. We had to write a small service that makes our WCS service to support time as a dimension because MapServer does not support it with WCS https://www.mail-archive.com/mapserver-users at lists.osgeo.org/msg26265.html. -Jukka Rahkonen- L?hett?j?: MapServer-users > Puolesta Marcin Niemyjski via MapServer-users L?hetetty: keskiviikko 26. hein?kuuta 2023 15.06 Vastaanottaja: MapServer Users > Aihe: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Seth, as you suggested, I've added the Extent parameter to all of my layers, and it resulted in decreasing time to 5m33.534s. But still, it's quite a lot of time for generating 1.5 MiB file. Any other suggestions? Maybe there is something wrong with projection so beyond just serving data mapserver have to reproject it multiple times? (more than one time by using tilesrs option)? Thanks, Marcin ________________________________ From: Seth G > Sent: Wednesday, July 26, 2023 12:58 PM To: Marcin Niemyjski >; MapServer Users > Subject: Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service? Hi Marcin, As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct): EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088 If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote: Hello, I have a problem, Namely, I have created a WCS and here is its mapfile: https://pastecode.io/s/6v3qfssb For a better understanding, here is a view of the tileindex through which the data is served: https://pasteboard.co/ydgvDiqxz4lf.png I am connecting to files located in a private bucket, so I am using vsis3. The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile: https://pasteboard.co/8G8uF41jqgHc.png It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size). Additionally, I am sending you URLs to sample data: https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2 https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2 The case is - how to speed it up? Am I making any goofy mistakes? I would be grateful for any recommendations or if you notice any errors, I might be making :) Best, Marcin _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From mniemyjski at cloudferro.com Mon Jul 31 06:24:25 2023 From: mniemyjski at cloudferro.com (Marcin Niemyjski) Date: Mon, 31 Jul 2023 13:24:25 +0000 Subject: [MapServer-users] TileIndex stored in POSTGIS: text VRT instead of text path to file in location column? Message-ID: Hello, Because now most of my work with Mapserver consists of the usage of tileindex and VRT (GDAL virtual raster), I was wondering if it is possible to combine these two aspects, namely: store raw VRT in the location column instead of (local or S3) path to the VRT file. So, instead of this: Table screenshot from PgAdmin4: https://pasteboard.co/akhv1c17F6LY.png I would have this in the overview column (example value of one record): https://pastecode.io/s/d4ueuijg In this approach, Mapserver, after checking out the intersection of the requested BBOX with table features' geometries, would directly open the VRT stored in the column overview instead of opening the path to the .vrt file. I hope this is possible. It would solve the problem of generating a big number of files. Best, Marcin -------------- next part -------------- An HTML attachment was scrubbed... URL: