From Rob.Dennett at twdb.texas.gov Fri Feb 2 15:37:49 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Fri, 2 Feb 2024 23:37:49 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? Message-ID: I recently upgraded our db for our Mapserver from Postgres 11 to Postgres 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the postgis 2.4 raster queries and add the new postgis_raster extension. Now I have web request which takes over 30 seconds where it used to take about 2. I made copies of the .map file and question and pointed one at the old db and set the debug level for the map to 2 for both. These .map files were created by someone who no longer works here and I am not a GIS professional, just a software developer, so I am having trouble understanding the output. After hitting the web page for both, I see the debug file is twice as large for the new db as it was for the old one. The .map file has 4 layers, three of which have a status of on, one for polygons, one for lines and one for point and one called "AllSources" (which is off). For the first 3, they contain DATA "geom from the_table_in_question" which is a field of type geometry. In the logs I see lines like select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') There are 8 of these in each output file, but their contents don't match. I am not sure what's generating them. I do note that they, along with SELECT geom from ...; statements take roughly equal time whether executed against the old db or the new one. What is generating these log statements? Why does what is for all intents and purposes the same .map file produce different output? I know it's because I changed the db engine and postgis, but if you could be more specific, that would be very helpful. Thanks, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From traviskirstine at gmail.com Sun Feb 4 08:18:36 2024 From: traviskirstine at gmail.com (Travis Kirstine) Date: Sun, 4 Feb 2024 11:18:36 -0500 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: Message-ID: First thought is to check if the table has a spatial index. https://postgis.net/workshops/postgis-intro/indexing.html On Fri, 2 Feb 2024 at 18:37, Rob Dennett via MapServer-users < mapserver-users at lists.osgeo.org> wrote: > I recently upgraded our db for our Mapserver from Postgres 11 to Postgres > 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the > postgis 2.4 raster queries and add the new postgis_raster extension. Now I > have web request which takes over 30 seconds where it used to take about > 2. I made copies of the .map file and question and pointed one at the old > db and set the debug level for the map to 2 for both. > > These .map files were created by someone who no longer works here and I am > not a GIS professional, just a software developer, so I am having trouble > understanding the output. > > After hitting the web page for both, I see the debug file is twice as > large for the new db as it was for the old one. The .map file has 4 > layers, three of which have a status of on, one for polygons, one for lines > and one for point and one called "AllSources" (which is off). For the > first 3, they contain > > DATA "geom from the_table_in_question" > > which is a field of type geometry. In the logs I see lines like > > select > "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') > as geom,"cartodb_id"::text from the_table_in_question where "geom" && > ST_GeomFromText('POLYGON((-12520996.7293382 > 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 > 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 > 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and > ("featuretyp"::text = 'polygon') > > There are 8 of these in each output file, but their contents don't match. > I am not sure what's generating them. I do note that they, along with > SELECT geom from ...; statements take roughly equal time whether executed > against the old db or the new one. > > What is generating these log statements? > > Why does what is for all intents and purposes the same .map file produce > different output? I know it's because I changed the db engine and postgis, > but if you could be more specific, that would be very helpful. > > Thanks, > Rob > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Rob.Dennett at twdb.texas.gov Sun Feb 4 13:39:37 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Sun, 4 Feb 2024 21:39:37 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: Message-ID: Running select * from pg_indexes where tablename like '%the_table_in_question%' yields the same results for both the old and new dbs. As mentioned, the mapserver debug output is very different from the old and new db are very different, even though the .map files are identical except for the connection string. There are 8 select statements in each, but the statements aren't the same and the output from the new db is twice as large, about 1.9MB, as the old one. I compiled the queries each one runs and ran them as a group against both dbs. The old db ran both sets of queries in about 5 seconds each, whereas the new db ran the new set in about 15 seconds and the old set in about 10 seconds. I haven't yet run VACUUM on the new db, and will be doing so I Monday, but I am just as confused as to why the debug output would be so different. Can anyone shed some light on that? Thanks, Rob ________________________________ From: Travis Kirstine Sent: Sunday, February 4, 2024 10:18 AM To: Rob Dennett Cc: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. First thought is to check if the table has a spatial index. https://postgis.net/workshops/postgis-intro/indexing.html On Fri, 2 Feb 2024 at 18:37, Rob Dennett via MapServer-users > wrote: I recently upgraded our db for our Mapserver from Postgres 11 to Postgres 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the postgis 2.4 raster queries and add the new postgis_raster extension. Now I have web request which takes over 30 seconds where it used to take about 2. I made copies of the .map file and question and pointed one at the old db and set the debug level for the map to 2 for both. These .map files were created by someone who no longer works here and I am not a GIS professional, just a software developer, so I am having trouble understanding the output. After hitting the web page for both, I see the debug file is twice as large for the new db as it was for the old one. The .map file has 4 layers, three of which have a status of on, one for polygons, one for lines and one for point and one called "AllSources" (which is off). For the first 3, they contain DATA "geom from the_table_in_question" which is a field of type geometry. In the logs I see lines like select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') There are 8 of these in each output file, but their contents don't match. I am not sure what's generating them. I do note that they, along with SELECT geom from ...; statements take roughly equal time whether executed against the old db or the new one. What is generating these log statements? Why does what is for all intents and purposes the same .map file produce different output? I know it's because I changed the db engine and postgis, but if you could be more specific, that would be very helpful. Thanks, Rob _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From Rob.Dennett at twdb.texas.gov Sun Feb 4 16:45:44 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Mon, 5 Feb 2024 00:45:44 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: Message-ID: So, after further analysis of the queries, when pointed at the old db, the mapserver output contains 4 queries which map to the 4 layers in the .map file, one for polygon, then line, then point, then centroid. These 4 are then repeated with different polygons. The queries all look similar, e.g. select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','the_table_in_question','geom')) and ("featuretyp"::text = 'polygon'); When run against the new db (with the same data, mind you, just a different version of Postgres and Postgis), the queries are polygon, polygon, line, line, point, point and centroid, centroid. While the contents of the POLYGON() function are the same for each set of 4 queries, they don't completely match what's seen in the other debug output file. That is, only every other number matches: old -12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378 new -11268652.4579138 2507134.52775378,-11268652.4579138 3754586.82936786,-10021200.1562997 3754586.82936786,-10021200.1562997 2507134.52775378,-11268652.4579138 2507134.52775378 What is POLYGON in the above query? Is it the same as ST_Polygon? What do the numbers mean? Thanks, Rob ________________________________ From: MapServer-users on behalf of Rob Dennett via MapServer-users Sent: Sunday, February 4, 2024 3:39 PM To: Travis Kirstine Cc: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Running select * from pg_indexes where tablename like '%the_table_in_question%' yields the same results for both the old and new dbs. As mentioned, the mapserver debug output is very different from the old and new db are very different, even though the .map files are identical except for the connection string. There are 8 select statements in each, but the statements aren't the same and the output from the new db is twice as large, about 1.9MB, as the old one. I compiled the queries each one runs and ran them as a group against both dbs. The old db ran both sets of queries in about 5 seconds each, whereas the new db ran the new set in about 15 seconds and the old set in about 10 seconds. I haven't yet run VACUUM on the new db, and will be doing so I Monday, but I am just as confused as to why the debug output would be so different. Can anyone shed some light on that? Thanks, Rob ________________________________ From: Travis Kirstine Sent: Sunday, February 4, 2024 10:18 AM To: Rob Dennett Cc: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. First thought is to check if the table has a spatial index. https://postgis.net/workshops/postgis-intro/indexing.html On Fri, 2 Feb 2024 at 18:37, Rob Dennett via MapServer-users > wrote: I recently upgraded our db for our Mapserver from Postgres 11 to Postgres 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the postgis 2.4 raster queries and add the new postgis_raster extension. Now I have web request which takes over 30 seconds where it used to take about 2. I made copies of the .map file and question and pointed one at the old db and set the debug level for the map to 2 for both. These .map files were created by someone who no longer works here and I am not a GIS professional, just a software developer, so I am having trouble understanding the output. After hitting the web page for both, I see the debug file is twice as large for the new db as it was for the old one. The .map file has 4 layers, three of which have a status of on, one for polygons, one for lines and one for point and one called "AllSources" (which is off). For the first 3, they contain DATA "geom from the_table_in_question" which is a field of type geometry. In the logs I see lines like select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') There are 8 of these in each output file, but their contents don't match. I am not sure what's generating them. I do note that they, along with SELECT geom from ...; statements take roughly equal time whether executed against the old db or the new one. What is generating these log statements? Why does what is for all intents and purposes the same .map file produce different output? I know it's because I changed the db engine and postgis, but if you could be more specific, that would be very helpful. Thanks, Rob _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsg at internode.on.net Sun Feb 4 16:44:06 2024 From: jsg at internode.on.net (James Gardner) Date: Mon, 05 Feb 2024 08:44:06 +0800 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: Message-ID: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> An HTML attachment was scrubbed... URL: From Rob.Dennett at twdb.texas.gov Sun Feb 4 16:53:43 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Mon, 5 Feb 2024 00:53:43 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: I don't think so. These queries are generated somehow. Here's what the PolygonSources layer looks like, and as you can see, the query I am specifying is just "geom from the_table_in_question": LAYER NAME "PolygonSources" CONNECTIONTYPE POSTGIS CONNECTION "xxxxxxxxxx" TYPE POLYGON STATUS ON DATA "geom FROM the_table_in_question" UTFITEM "cartodb_id" UTFDATA "{\"cartodb_id\":\"[cartodb_id]\",\"sourceid\":\"[sourceid]\",\"name\":\"[name]\",\"sourcetype\":\"[sourcetype]\",\"drawingord\":\"[drawingord]\",\"featuretyp\":\"[featuretyp]\",\"isnew\":\"[isnew]\"}" TEMPLATE WMSGetFeatureInfo FILTERITEM "featuretyp" CLASSITEM "sourceid" FILTER "polygon" CLASS NAME "polygon" EXPRESSION ([sourceid] != 169 AND [sourceid] != 412 AND [sourceid] != 820 AND [sourceid] != 1067) STYLE COLOR "#0B3A71B3" OUTLINECOLOR "#AFBFD0B3" OUTLINEWIDTH????1.5 END END METADATA "wms_title" "Polygon Source Features" "wms_include_items" "all" "wms_abstract" "Layer of all polygon geometry sources." "wfs_title" "Polygon Source Features" "wfs_srs" "EPSG:3857 EPSG:4326" "wfs_enable_request" "*" "wfs_abstract" "Layer of all polygon geometry sources." "gml_include_items" "all" "gml_featureid" "cartodb_id" END PROJECTION "init=epsg:3857" END PROCESSING "CLOSE_CONNECTION=DEFER" END ________________________________ From: James Gardner Sent: Sunday, February 4, 2024 6:44 PM To: Rob Dennett Cc: Travis Kirstine ; mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Could you try replacing find_srid with a hard coded srid... I found it had to run find_srid on every tuple... -James Gardner On 5 Feb 2024 5:39 am, Rob Dennett via MapServer-users wrote: Running select * from pg_indexes where tablename like '%the_table_in_question%' yields the same results for both the old and new dbs. As mentioned, the mapserver debug output is very different from the old and new db are very different, even though the .map files are identical except for the connection string. There are 8 select statements in each, but the statements aren't the same and the output from the new db is twice as large, about 1.9MB, as the old one. I compiled the queries each one runs and ran them as a group against both dbs. The old db ran both sets of queries in about 5 seconds each, whereas the new db ran the new set in about 15 seconds and the old set in about 10 seconds. I haven't yet run VACUUM on the new db, and will be doing so I Monday, but I am just as confused as to why the debug output would be so different. Can anyone shed some light on that? Thanks, Rob ________________________________ From: Travis Kirstine Sent: Sunday, February 4, 2024 10:18 AM To: Rob Dennett Cc: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. First thought is to check if the table has a spatial index. https://postgis.net/workshops/postgis-intro/indexing.html On Fri, 2 Feb 2024 at 18:37, Rob Dennett via MapServer-users > wrote: I recently upgraded our db for our Mapserver from Postgres 11 to Postgres 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the postgis 2.4 raster queries and add the new postgis_raster extension. Now I have web request which takes over 30 seconds where it used to take about 2. I made copies of the .map file and question and pointed one at the old db and set the debug level for the map to 2 for both. These .map files were created by someone who no longer works here and I am not a GIS professional, just a software developer, so I am having trouble understanding the output. After hitting the web page for both, I see the debug file is twice as large for the new db as it was for the old one. The .map file has 4 layers, three of which have a status of on, one for polygons, one for lines and one for point and one called "AllSources" (which is off). For the first 3, they contain DATA "geom from the_table_in_question" which is a field of type geometry. In the logs I see lines like select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') There are 8 of these in each output file, but their contents don't match. I am not sure what's generating them. I do note that they, along with SELECT geom from ...; statements take roughly equal time whether executed against the old db or the new one. What is generating these log statements? Why does what is for all intents and purposes the same .map file produce different output? I know it's because I changed the db engine and postgis, but if you could be more specific, that would be very helpful. Thanks, Rob _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Sun Feb 4 23:25:55 2024 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Mon, 5 Feb 2024 07:25:55 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Hi, See https://mapserver.org/input/vector/postgis.html, there is an example about how to define the SRID on the DATA line: DATA "the_geom from the_database using unique gid using srid=4326" The SRID value to use is the native SRID of the PostGIS table. What Mapserver does here is that with ?&&? operator it selects those features from the table which intersect with the reference geometry and because of that it needs to know the SRID of the table. The reference geometry (POLYGON) is either the BBOX of the request or the EXTENT used in the mapfile, I cannot say for sure with this information. I believe that find_srid is rather fast but it is good to include ?using srid=? anyway. Adding ?unique? for defining the primary key of the table cannot make any harm either. You wrote ?There are 8 select statements in each, but the statements aren't the same? and then you showed one of the statements. It would help to see also the other statement. But if the tables have the same data and the output from the new db is much larger, and there is no other filter in the SQL query than the &&, then I quess that the reference polygon of the latter case is bigger for some reason. -Jukka Rahkonen- L?hett?j?: MapServer-users > Puolesta Rob Dennett via MapServer-users L?hetetty: maanantai 5. helmikuuta 2024 2.54 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? I don't think so. These queries are generated somehow. Here's what the PolygonSources layer looks like, and as you can see, the query I am specifying is just "geom from the_table_in_question": LAYER NAME "PolygonSources" CONNECTIONTYPE POSTGIS CONNECTION "xxxxxxxxxx" TYPE POLYGON STATUS ON DATA "geom FROM the_table_in_question" UTFITEM "cartodb_id" UTFDATA "{\"cartodb_id\":\"[cartodb_id]\",\"sourceid\":\"[sourceid]\",\"name\":\"[name]\",\"sourcetype\":\"[sourcetype]\",\"drawingord\":\"[drawingord]\",\"featuretyp\":\"[featuretyp]\",\"isnew\":\"[isnew]\"}" TEMPLATE WMSGetFeatureInfo FILTERITEM "featuretyp" CLASSITEM "sourceid" FILTER "polygon" CLASS NAME "polygon" EXPRESSION ([sourceid] != 169 AND [sourceid] != 412 AND [sourceid] != 820 AND [sourceid] != 1067) STYLE COLOR "#0B3A71B3" OUTLINECOLOR "#AFBFD0B3" OUTLINEWIDTH????1.5 END END METADATA "wms_title" "Polygon Source Features" "wms_include_items" "all" "wms_abstract" "Layer of all polygon geometry sources." "wfs_title" "Polygon Source Features" "wfs_srs" "EPSG:3857 EPSG:4326" "wfs_enable_request" "*" "wfs_abstract" "Layer of all polygon geometry sources." "gml_include_items" "all" "gml_featureid" "cartodb_id" END PROJECTION "init=epsg:3857" END PROCESSING "CLOSE_CONNECTION=DEFER" END ________________________________ From: James Gardner > Sent: Sunday, February 4, 2024 6:44 PM To: Rob Dennett > Cc: Travis Kirstine >; mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Could you try replacing find_srid with a hard coded srid... I found it had to run find_srid on every tuple... -James Gardner -------------- next part -------------- An HTML attachment was scrubbed... URL: From oldbay at bk.ru Mon Feb 5 01:24:10 2024 From: oldbay at bk.ru (=?UTF-8?B?QmFldiBBbGV4YW5kcg==?=) Date: Mon, 05 Feb 2024 12:24:10 +0300 Subject: [MapServer-users] =?utf-8?q?Geos_segfault_error_under_certain_co?= =?utf-8?q?nditions?= Message-ID: <1707125050.869858802@f713.i.mail.ru> Hello All! ? Under certain conditions I get a strange error: ?? ? The combination of spatil geometry, style of class in map and bbox in WMS request - leads to the libgeos segmentation error. ? All files of Example adding to email: * bug_json.map * geodata_bug.geojson * mapfile_web.py * mapstyle_for_7_0_4.zip ? Run this: python3 mapfile_web.py bug_json.map ? Passed WMS request: http://localhost:3007/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=3032662.425975159276,5637758.341410524212,3340530.939469947945,5864962.737246668898&CRS=EPSG:3857&WIDTH=1103&HEIGHT=814&LAYERS=good&FORMAT=image/png ? Failed WMS request: http://localhost:3007/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=3032662.425975159276,5637758.341410524212,3340530.939469947945,5864962.737246668898&CRS=EPSG:3857&WIDTH=1103&HEIGHT=814&LAYERS=bad&FORMAT=image/png ? Reproduction conditions of error: * mapserver & python3-mapscript v7.6.4, v7.6.5 (manual package), geos v3.5.1 (repository package) - OS linux debian 9 * mapserver & python3-mapscript v7.6.2 (repository package), geos v3.9.0 (repository package) - OS linux debian 11 ? Error message before segafoult: python3: geos_ts_c.cpp:3742: int GEOSCoordSeq_getOrdinate_r(GEOSContextHandle_t, const geos::geom::CoordinateSequence*, unsigned int, unsigned int, double*): Assertion `0 != cs' failed. ? ? The error does not cause segfault in mapserver v7.0.4,(repository package)? geos v3.5.1 (repository package) - OS linux debian 9: ? Run this (mapstyle & shp files in mapstyle_for_7_0_4.zip): python3 mapfile_web.py bug_shp.map ? All WMS requests from the example above. ? Error message don't segafoult - all WMS query (rendering passed): OWSDispatch Error: msGEOSError(): GEOS library error. (message repeated 1 times) ? ? P.S: Are there any recommendations or patches for mapserver 7.6.*(Preferably) - what to do with this problem? ? ? Baev Alexandr oldbay at bk.ru -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bug_json.map Type: application/x-navimap Size: 865 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: geodata_bug.geojson Type: application/geo+json Size: 1591 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mapfile_web.py Type: application/x-python Size: 1538 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mapstyle_for_7_0_4.zip Type: application/zip Size: 2135 bytes Desc: not available URL: From Rob.Dennett at twdb.texas.gov Mon Feb 5 07:34:34 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Mon, 5 Feb 2024 15:34:34 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: So, looking at this further, this is the bounding box for what's happening is that when pointed at the old db, the .map file is making the calls using these coordinates -12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378 and then again with these -12518550.744433 2509580.51265891,-12518550.744433 5004485.11588706,-10023646.1412049 5004485.11588706,-10023646.1412049 2509580.51265891,-12518550.744433 2509580.51265891 When pointed at the new db (which does indeed have identical data and structure), the polygons are -11268652.4579138 2507134.52775378,-11268652.4579138 3754586.82936786,-10021200.1562997 3754586.82936786,-10021200.1562997 2507134.52775378,-11268652.4579138 2507134.52775378 and -12068252.5062205 2978893.85839647,-12068252.5062205 4369793.96473184,-10212686.4400848 4369793.96473184,-10212686.4400848 2978893.85839647,-12068252.5062205 2978893.85839647 I have no idea where these polygons are coming from, nor do I understand why they are different when pointed at the old vs. new db nor why the queries are called in a different order. The .map file has a line defining the extent: EXTENT -11871597.4858696 2978893.85839647 -10409341.4604357 4369793.96473184 but that doesn't match up with these polygons. The new polygons appear to be a bit smaller than the old ones, if I understand correctly. Does anyone know why I am seeing what I am seeing? Thanks, Rob ________________________________ From: Rahkonen Jukka Sent: Monday, February 5, 2024 1:25 AM To: Rob Dennett ; mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Hi, See https://mapserver.org/input/vector/postgis.html, there is an example about how to define the SRID on the DATA line: DATA "the_geom from the_database using unique gid using srid=4326" The SRID value to use is the native SRID of the PostGIS table. What Mapserver does here is that with ?&&? operator it selects those features from the table which intersect with the reference geometry and because of that it needs to know the SRID of the table. The reference geometry (POLYGON) is either the BBOX of the request or the EXTENT used in the mapfile, I cannot say for sure with this information. I believe that find_srid is rather fast but it is good to include ?using srid=? anyway. Adding ?unique? for defining the primary key of the table cannot make any harm either. You wrote ?There are 8 select statements in each, but the statements aren't the same? and then you showed one of the statements. It would help to see also the other statement. But if the tables have the same data and the output from the new db is much larger, and there is no other filter in the SQL query than the &&, then I quess that the reference polygon of the latter case is bigger for some reason. -Jukka Rahkonen- L?hett?j?: MapServer-users > Puolesta Rob Dennett via MapServer-users L?hetetty: maanantai 5. helmikuuta 2024 2.54 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? I don't think so. These queries are generated somehow. Here's what the PolygonSources layer looks like, and as you can see, the query I am specifying is just "geom from the_table_in_question": LAYER NAME "PolygonSources" CONNECTIONTYPE POSTGIS CONNECTION "xxxxxxxxxx" TYPE POLYGON STATUS ON DATA "geom FROM the_table_in_question" UTFITEM "cartodb_id" UTFDATA "{\"cartodb_id\":\"[cartodb_id]\",\"sourceid\":\"[sourceid]\",\"name\":\"[name]\",\"sourcetype\":\"[sourcetype]\",\"drawingord\":\"[drawingord]\",\"featuretyp\":\"[featuretyp]\",\"isnew\":\"[isnew]\"}" TEMPLATE WMSGetFeatureInfo FILTERITEM "featuretyp" CLASSITEM "sourceid" FILTER "polygon" CLASS NAME "polygon" EXPRESSION ([sourceid] != 169 AND [sourceid] != 412 AND [sourceid] != 820 AND [sourceid] != 1067) STYLE COLOR "#0B3A71B3" OUTLINECOLOR "#AFBFD0B3" OUTLINEWIDTH????1.5 END END METADATA "wms_title" "Polygon Source Features" "wms_include_items" "all" "wms_abstract" "Layer of all polygon geometry sources." "wfs_title" "Polygon Source Features" "wfs_srs" "EPSG:3857 EPSG:4326" "wfs_enable_request" "*" "wfs_abstract" "Layer of all polygon geometry sources." "gml_include_items" "all" "gml_featureid" "cartodb_id" END PROJECTION "init=epsg:3857" END PROCESSING "CLOSE_CONNECTION=DEFER" END ________________________________ From: James Gardner > Sent: Sunday, February 4, 2024 6:44 PM To: Rob Dennett > Cc: Travis Kirstine >; mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Could you try replacing find_srid with a hard coded srid... I found it had to run find_srid on every tuple... -James Gardner -------------- next part -------------- An HTML attachment was scrubbed... URL: From traviskirstine at gmail.com Mon Feb 5 08:50:01 2024 From: traviskirstine at gmail.com (Travis Kirstine) Date: Mon, 5 Feb 2024 11:50:01 -0500 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Not sure I understand the issue with the polygons. The polygon value in the select statement will change based on the client's view extent, for example if the users moves the map a new request with a different polygon / bbox would be issued to mapserver and through to postgres. The EXTENT value defined in the mapfile defines the extent of the map or layer coverage. I would try running EXPLAIN ANALYZE on the same query on both versions of Postgres and see what the differences are. explain analyze select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') On Mon, 5 Feb 2024 at 10:35, Rob Dennett via MapServer-users < mapserver-users at lists.osgeo.org> wrote: > So, looking at this further, this is the bounding box for what's happening > is that when pointed at the old db, the .map file is making the calls using > these coordinates > > -12520996.7293382 2507134.52775378,-12520996.7293382 > 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 > 2507134.52775378,-12520996.7293382 2507134.52775378 > > and then again with these > > -12518550.744433 2509580.51265891,-12518550.744433 > 5004485.11588706,-10023646.1412049 5004485.11588706,-10023646.1412049 > 2509580.51265891,-12518550.744433 2509580.51265891 > > When pointed at the new db (which does indeed have identical data and > structure), the polygons are > > -11268652.4579138 2507134.52775378,-11268652.4579138 > 3754586.82936786,-10021200.1562997 3754586.82936786,-10021200.1562997 > 2507134.52775378,-11268652.4579138 2507134.52775378 > > and > > -12068252.5062205 2978893.85839647,-12068252.5062205 > 4369793.96473184,-10212686.4400848 4369793.96473184,-10212686.4400848 > 2978893.85839647,-12068252.5062205 2978893.85839647 > > I have no idea where these polygons are coming from, nor do I understand > why they are different when pointed at the old vs. new db nor why the > queries are called in a different order. The .map file has a line defining > the extent: > > EXTENT -11871597.4858696 2978893.85839647 -10409341.4604357 > 4369793.96473184 > > but that doesn't match up with these polygons. The new polygons appear to > be a bit smaller than the old ones, if I understand correctly. Does anyone > know why I am seeing what I am seeing? > > Thanks, > Rob > ------------------------------ > *From:* Rahkonen Jukka > *Sent:* Monday, February 5, 2024 1:25 AM > *To:* Rob Dennett ; > mapserver-users at lists.osgeo.org > *Subject:* Re: [MapServer-users] Why is Mapserver slower in Postgres 13 > than in Postgres 11? > > > External: Beware of links/attachments. > > > Hi, > > > > See https://mapserver.org/input/vector/postgis.html, there is an example > about how to define the SRID on the DATA line: > DATA "the_geom from the_database using unique gid using srid=4326" > > > > The SRID value to use is the native SRID of the PostGIS table. > > What Mapserver does here is that with ?&&? operator it selects those > features from the table which intersect with the reference geometry and > because of that it needs to know the SRID of the table. The reference > geometry (POLYGON) is either the BBOX of the request or the EXTENT used in > the mapfile, I cannot say for sure with this information. I believe that > find_srid is rather fast but it is good to include ?using srid=? anyway. > Adding ?unique? for defining the primary key of the table cannot make any > harm either. > > You wrote ?There are 8 select statements in each, but the statements > aren't the same? and then you showed one of the statements. It would help > to see also the other statement. But if the tables have the same data and > the output from the new db is much larger, and there is no other filter in > the SQL query than the &&, then I quess that the reference polygon of the > latter case is bigger for some reason. > > > > -Jukka Rahkonen- > > > > *L?hett?j?:* MapServer-users *Puolesta > *Rob Dennett via MapServer-users > *L?hetetty:* maanantai 5. helmikuuta 2024 2.54 > *Vastaanottaja:* mapserver-users at lists.osgeo.org > *Aihe:* Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than > in Postgres 11? > > > > I don't think so. These queries are generated somehow. Here's what the > PolygonSources layer looks like, and as you can see, the query I am > specifying is just "geom from the_table_in_question": > > LAYER > > NAME "PolygonSources" > > CONNECTIONTYPE POSTGIS > > CONNECTION "xxxxxxxxxx" > > TYPE POLYGON > > STATUS ON > > DATA "geom FROM the_table_in_question" > > UTFITEM "cartodb_id" > > UTFDATA > "{\"cartodb_id\":\"[cartodb_id]\",\"sourceid\":\"[sourceid]\",\"name\":\"[name]\",\"sourcetype\":\"[sourcetype]\",\"drawingord\":\"[drawingord]\",\"featuretyp\":\"[featuretyp]\",\"isnew\":\"[isnew]\"}" > > TEMPLATE WMSGetFeatureInfo > > FILTERITEM "featuretyp" > > CLASSITEM "sourceid" > > FILTER "polygon" > > CLASS > > NAME "polygon" > > EXPRESSION ([sourceid] != 169 AND [sourceid] != 412 AND [sourceid] > != 820 AND [sourceid] != 1067) > > STYLE > > COLOR "#0B3A71B3" > > OUTLINECOLOR "#AFBFD0B3" > > OUTLINEWIDTH 1.5 > > END > > END > > METADATA > > "wms_title" "Polygon Source Features" > > "wms_include_items" "all" > > "wms_abstract" "Layer of all polygon geometry sources." > > "wfs_title" "Polygon Source Features" > > "wfs_srs" "EPSG:3857 EPSG:4326" > > "wfs_enable_request" "*" > > "wfs_abstract" "Layer of all polygon geometry sources." > > "gml_include_items" "all" > > "gml_featureid" "cartodb_id" > > END > > PROJECTION > > "init=epsg:3857" > > END > > PROCESSING "CLOSE_CONNECTION=DEFER" > > END > ------------------------------ > > *From:* James Gardner > *Sent:* Sunday, February 4, 2024 6:44 PM > *To:* Rob Dennett > *Cc:* Travis Kirstine ; > mapserver-users at lists.osgeo.org > *Subject:* Re: [MapServer-users] Why is Mapserver slower in Postgres 13 > than in Postgres 11? > > > > *External: Beware of links/attachments. * > > > > Could you try replacing find_srid with a hard coded srid... I found it had > to run find_srid on every tuple... > > -James Gardner > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkvoelkl at nelson-games.de Tue Feb 6 05:08:10 2024 From: gkvoelkl at nelson-games.de (=?utf-8?Q? G._V=C3=B6lkl ?=) Date: Tue, 6 Feb 2024 14:08:10 +0100 Subject: [MapServer-users] Security and LDAP with mapservet Message-ID: Hello, I want to create some WMS Services which can be only accessed by some Special Users. Is it possible to Secure wms Services? Is it possible to use LDAP. Best regards Gerhard Von meinem iPhone gesendet From bjorn.ove at grotan.com Tue Feb 6 08:35:42 2024 From: bjorn.ove at grotan.com (=?utf-8?B?QmrDuHJuIE92ZSBHcsO4dGFu?=) Date: Tue, 6 Feb 2024 17:35:42 +0100 Subject: [MapServer-users] Security and LDAP with mapservet In-Reply-To: References: Message-ID: G. V?lkl via MapServer-users: > Hello, > > I want to create some WMS Services which can be only accessed by some Special Users. > > Is it possible to Secure wms Services? Is it possible to use LDAP. > > Best regards Gerhard > Von meinem iPhone gesendet You can secure Mapserver WMS using whatever webserver you are exposing them through, be it NGINX og Apache (0). I know Apache supports ldap (1) and I would guess NGINX does to. If you expose Mapserver WMS through mapscript, you can handle authn/authz inside your mapscript-code. (0) https://httpd.apache.org/docs/2.4/howto/auth.html (1) https://httpd.apache.org/docs/2.4/mod/mod_ldap.html -- BEst regards Bj?rn Ove Gr?tan From Rob.Dennett at twdb.texas.gov Tue Feb 6 11:02:24 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Tue, 6 Feb 2024 19:02:24 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: So, I ran explain analyze, and got different results when run against the old and new db. I ran VACUUM on the table (which ran in less than ? a second) and checked the query plans again. They're now the same except for the time estimates and against the new db they're still much slower. We ran across a StackExchange article about performance degradation after upgrading to postgres 13 and postgis 3.4 (https://dba.stackexchange.com/questions/300292/why-is-my-spatial-query-slower-in-postgres-13-than-in-postgres-11) and they mention a change in the way postgis works, but since these queries are generated by mapserver, I am not sure what I can do on my end. Is there a version of MapServer that's recommended for Postgres 13/PostGIS 3? [https://cdn.sstatic.net/Sites/dba/Img/apple-touch-icon at 2.png?v=246e2cb2439c] Why is my spatial query slower in Postgres 13 than in Postgres 11? Postgres versions PostgreSQL 13.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit PostgreSQL 11.11 (Debian 11.11-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled ... dba.stackexchange.com ________________________________ From: Travis Kirstine Sent: Monday, February 5, 2024 10:50 AM To: Rob Dennett Cc: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Not sure I understand the issue with the polygons. The polygon value in the select statement will change based on the client's view extent, for example if the users moves the map a new request with a different polygon / bbox would be issued to mapserver and through to postgres. The EXTENT value defined in the mapfile defines the extent of the map or layer coverage. I would try running EXPLAIN ANALYZE on the same query on both versions of Postgres and see what the differences are. explain analyze select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matt.Lernout at stantec.com Tue Feb 6 11:12:58 2024 From: Matt.Lernout at stantec.com (Lernout, Matthew) Date: Tue, 6 Feb 2024 19:12:58 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate Message-ID: Hi, Having trouble making requests since updating to 8.0.1 from 7.6.2. Mapserver is running in IIS using FastCGI and the configuration steps from: https://mapserver.org/installation/iis.html A new config file was added to the filesystem and server Environment variables as per documented requirements added since 7.6.3, but having trouble validating any map arguments. All requests return the error "CGI variable "map" fails to validate" even though regexp should pass based on expression tests. Here is the full config file in use: CONFIG ENV MS_MAP_PATTERN "\.map$" PROJ_LIB "F:/MapServer/bin/proj7/SHARE" CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" END PLUGINS "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" END END The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 working before I update to the real regexp, and my understanding is it should map anything that ends in .map? But every map argument I throw in (including verified working arguments for 7.6.2) result in a failure to validate. Are there other crucial configurations missing that are necessary to pass validation? Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Tue Feb 6 11:42:02 2024 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Tue, 6 Feb 2024 19:42:02 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Hi, I made a test with a PostGIS on my own laptop. My versions are: "POSTGIS=""3.4.1 3.4.1"" [EXTENSION] PGSQL=""160"" GEOS=""3.12.1-CAPI-1.18.1"" SFCGAL=""SFCGAL 1.5.0, CGAL 5.6, BOOST 1.78.0"" PROJ=""8.2.1 The polygon layer has 215000 lake polygons. My BBOX select 73 polygons. It takes 66 milliseconds. My query and the execution plan are as follows: select * from jarvinemo where "geom" && ST_GeomFromText( 'POLYGON (( 314728.6874003611 6936494.124854623, 314728.6874003611 6946067.332484153, 327290.0638853506 6946067.332484153, 327290.0638853506 6936494.124854623, 314728.6874003611 6936494.124854623 ))'); "Bitmap Heap Scan on jarvinemo (cost=4.58..155.40 rows=39 width=588) (actual time=0.048..0.076 rows=73 loops=1)" " Recheck Cond: (geom && '0103000000010000000500000059E1E5BFA23513413F9EFD87EB755A4159E1E5BFA23513419D6B47D5447F5A414E296B41E8F913419D6B47D5447F5A414E296B41E8F913413F9EFD87EB755A4159E1E5BFA23513413F9EFD87EB755A41'::geometry)" " Heap Blocks: exact=64" " -> Bitmap Index Scan on jarvinemo_geom_geom_idx (cost=0.00..4.58 rows=39 width=0) (actual time=0.039..0.039 rows=73 loops=1)" " Index Cond: (geom && '0103000000010000000500000059E1E5BFA23513413F9EFD87EB755A4159E1E5BFA23513419D6B47D5447F5A414E296B41E8F913419D6B47D5447F5A414E296B41E8F913413F9EFD87EB755A4159E1E5BFA23513413F9EFD87EB755A41'::geometry)" "Planning Time: 0.142 ms" "Execution Time: 0.106 ms" I can see that the bbox (POLYGON) in your query is big, something like 1500-2500 kilometres wide, and covers almost the whole EXTENT of your data. That means that the spatial filter is ineffective because it does not filter out anything. Have a try with a small polygon as a filter and compare the execution plan with the one that my database makes. That helps you to find out if the spatial index kicks in at some time. Of course, that does not help when the map covers the whole area. Then all the data must be read. But spending 5 seconds (the old db) or 15 seconds (the new db) for that as you have written before feels very slow. I wonder if the slowness comes from the connection to the database rather than from the database itself. It is some hosted database, cartodb perhaps by some logs that you have showed. Have you made any tests with a local db? -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Rob Dennett via MapServer-users L?hetetty: tiistai 6. helmikuuta 2024 21.02 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? So, I ran explain analyze, and got different results when run against the old and new db. I ran VACUUM on the table (which ran in less than ? a second) and checked the query plans again. They're now the same except for the time estimates and against the new db they're still much slower. We ran across a StackExchange article about performance degradation after upgrading to postgres 13 and postgis 3.4 (https://dba.stackexchange.com/questions/300292/why-is-my-spatial-query-slower-in-postgres-13-than-in-postgres-11) and they mention a change in the way postgis works, but since these queries are generated by mapserver, I am not sure what I can do on my end. Is there a version of MapServer that's recommended for Postgres 13/PostGIS 3? [https://cdn.sstatic.net/Sites/dba/Img/apple-touch-icon at 2.png?v=246e2cb2439c] Why is my spatial query slower in Postgres 13 than in Postgres 11? Postgres versions PostgreSQL 13.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit PostgreSQL 11.11 (Debian 11.11-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled ... dba.stackexchange.com ________________________________ From: Travis Kirstine > Sent: Monday, February 5, 2024 10:50 AM To: Rob Dennett > Cc: mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? External: Beware of links/attachments. Not sure I understand the issue with the polygons. The polygon value in the select statement will change based on the client's view extent, for example if the users moves the map a new request with a different polygon / bbox would be issued to mapserver and through to postgres. The EXTENT value defined in the mapfile defines the extent of the map or layer coverage. I would try running EXPLAIN ANALYZE on the same query on both versions of Postgres and see what the differences are. explain analyze select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon') -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Tue Feb 6 11:50:50 2024 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Tue, 6 Feb 2024 19:50:50 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: Message-ID: Hi, This one that is slightly edited from the MS4W config file works for me MS_MAP_PATTERN "^(C:)?\/ms4w_data\/((?!\.{2})[_A-Za-z0-9\-\.]+\/{1})*([_A-Za-z0-9\-\.]+\.(map))$" I am not a regexp expert but it seems like a slash must be escaped with a backslash. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Lernout, Matthew via MapServer-users L?hetetty: tiistai 6. helmikuuta 2024 21.13 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: [MapServer-users] CGI variable "map" fails to validate Hi, Having trouble making requests since updating to 8.0.1 from 7.6.2. Mapserver is running in IIS using FastCGI and the configuration steps from: https://mapserver.org/installation/iis.html A new config file was added to the filesystem and server Environment variables as per documented requirements added since 7.6.3, but having trouble validating any map arguments. All requests return the error "CGI variable "map" fails to validate" even though regexp should pass based on expression tests. Here is the full config file in use: CONFIG ENV MS_MAP_PATTERN "\.map$" PROJ_LIB "F:/MapServer/bin/proj7/SHARE" CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" END PLUGINS "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" END END The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 working before I update to the real regexp, and my understanding is it should map anything that ends in .map? But every map argument I throw in (including verified working arguments for 7.6.2) result in a failure to validate. Are there other crucial configurations missing that are necessary to pass validation? Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From Rob.Dennett at twdb.texas.gov Tue Feb 6 11:53:03 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Tue, 6 Feb 2024 19:53:03 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Both DBs are hosted on AWS Aurora. The old one is serverless v1 and the new one is serverless v2. We were thinking that perhaps the geom column for my table needed a spatial index even though the old db doesn't have one. Is that possible? Also, would it make sense that the same query run on both should take the same amount of time (roughly)? Thanks, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From jukka.rahkonen at maanmittauslaitos.fi Tue Feb 6 12:05:59 2024 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Tue, 6 Feb 2024 20:05:59 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Hi, If spatial index is needed depends on the data. It is certainly needed when the table is large (million features or so) and requests typically cover a small area. If for example 1000 features gets selected it means that the spatial index can filter out 999 thousand features. If the table is small like one thousand features, then it is not so expensive to read everything without spatial filter. You are giving us little by little more information. I do not believe that the PostgreSQL 13 vs version 11 is the main thing at all. However, I do not have databases hosted on AWS and I can not make tests in the similar environment that you have. -Jukka Rahkonen- L?hett?j?: MapServer-users Puolesta Rob Dennett via MapServer-users L?hetetty: tiistai 6. helmikuuta 2024 21.53 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? Both DBs are hosted on AWS Aurora. The old one is serverless v1 and the new one is serverless v2. We were thinking that perhaps the geom column for my table needed a spatial index even though the old db doesn't have one. Is that possible? Also, would it make sense that the same query run on both should take the same amount of time (roughly)? Thanks, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Tue Feb 6 14:15:57 2024 From: sethg at geographika.co.uk (Seth G) Date: Tue, 06 Feb 2024 23:15:57 +0100 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: <6674e120-5504-44b9-b837-5e2d702e800c@app.fastmail.com> Hi Rob, Are you using the same MapServer machine to make the queries? If the MapServer version, and dependencies are all identical then the generated SQL queries should also be identical. You wrote about "hitting the web page" - but if the requested extents are different then it is likely the client requests are different. Are you using WMS services? What do these requests look like? Also you mention 4 different layers, so I'd try testing them one by one - are they all generating different SQL and response times? Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Tue, Feb 6, 2024, at 8:53 PM, Rob Dennett via MapServer-users wrote: > Both DBs are hosted on AWS Aurora. The old one is serverless v1 and the new one is serverless v2. We were thinking that perhaps the geom column for my table needed a spatial index even though the old db doesn't have one. Is that possible? Also, would it make sense that the same query run on both should take the same amount of time (roughly)? > > Thanks, > Rob > > > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joerg.thomsen at wheregroup.com Wed Feb 7 00:23:32 2024 From: joerg.thomsen at wheregroup.com (=?UTF-8?Q?J=C3=B6rg_Thomsen_=28WhereGroup=29?=) Date: Wed, 7 Feb 2024 09:23:32 +0100 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: Message-ID: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> Hi, I'm not sure whether the pattern is correct. Shouldn't ist start with ^? Have yout tried MS_MAP_PATTERN '.'? (not for production, just as a test) J?rg Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: > Hi, > > Having trouble making requests since updating to 8.0.1 from 7.6.2. > Mapserver?is running in IIS using FastCGI and the configuration steps > from: https://mapserver.org/installation/iis.html > > > A new config file was added to the filesystem and server?Environment > variables?as per documented requirements added since 7.6.3, but having > trouble validating any map?arguments. All requests return the error "CGI > variable "map" fails to validate" even though regexp should pass based > on expression tests. > > Here is the full config file in use: > > CONFIG > ? ENV > ? ? MS_MAP_PATTERN "\.map$" > ? ? PROJ_LIB "F:/MapServer/bin/proj7/SHARE" > ? ? CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" > ? END > > > ? PLUGINS > ? ? "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" > ? ? "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" > ? END > END > > The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 > working before I update to the real regexp, and my understanding is it > should map?anything that ends in .map? But every map?argument I throw in > (including verified?working arguments for 7.6.2) result in a failure to > validate. Are there other crucial configurations?missing that are > necessary to pass validation? > > Thanks, > > Matt > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users -- Viele Gr??e, J?rg Thomsen ********************************** Aufwind durch Wissen! Web-Seminare und Online-Schulungen bei der www.foss-academy.com/ ********************************** J?rg Thomsen WhereGroup GmbH Bundesallee 23 10717 Berlin Germany Tel: +49 (0)30 / 5130 278 74 Fax: +49 (0)30 / 5130 278 11 joerg.thomsen at wheregroup.com www.wheregroup.com Gesch?ftsf?hrer: Olaf Knopp, Peter Stamm Amtsgericht Bonn, HRB 9885 ------------------------------- Folgen Sie der WhereGroup auf twitter: http://twitter.com/WhereGroup_com From Matt.Lernout at stantec.com Wed Feb 7 08:33:16 2024 From: Matt.Lernout at stantec.com (Lernout, Matthew) Date: Wed, 7 Feb 2024 16:33:16 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> Message-ID: MS_MAP_PATTERN '.' also results in the same failed validation error, which is what makes me think the issue may be with another config setting. '.' should match any input, right? Just to make sure my config file was being loaded, I commented out the MS_MAP_PATTERN entry entirely, and got the error "msCGILoadMap(): Web application error. Required configuration value MS_MAP_PATTERN not set." Matt -----Original Message----- From: MapServer-users On Behalf Of J?rg Thomsen (WhereGroup) via MapServer-users Sent: Wednesday, February 7, 2024 3:24 AM To: mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] CGI variable "map" fails to validate [You don't often get email from mapserver-users at lists.osgeo.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi, I'm not sure whether the pattern is correct. Shouldn't ist start with ^? Have yout tried MS_MAP_PATTERN '.'? (not for production, just as a test) J?rg Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: > Hi, > > Having trouble making requests since updating to 8.0.1 from 7.6.2. > Mapserver is running in IIS using FastCGI and the configuration steps > from: > https://maps/ > erver.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40stan > tec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4d > 80281e7%7C0%7C0%7C638428910243090792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM > C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C > &sdata=dh%2BpgGwk5996f0GEMeX7O8rcv5J81TnPzeFNZ9XoVLs%3D&reserved=0 > server.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40sta > ntec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4 > d80281e7%7C0%7C0%7C638428910243098246%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi > MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7 > C&sdata=OIoWHSlKZJrXWspqwVfY5jhX4lzBcQuI%2Bw5IeOVtXBA%3D&reserved=0> > > A new config file was added to the filesystem and server Environment > variables as per documented requirements added since 7.6.3, but having > trouble validating any map arguments. All requests return the error > "CGI variable "map" fails to validate" even though regexp should pass > based on expression tests. > > Here is the full config file in use: > > CONFIG > ENV > MS_MAP_PATTERN "\.map$" > PROJ_LIB "F:/MapServer/bin/proj7/SHARE" > CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" > END > > > PLUGINS > "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" > "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" > END > END > > The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 > working before I update to the real regexp, and my understanding is it > should map anything that ends in .map? But every map argument I throw > in (including verified working arguments for 7.6.2) result in a > failure to validate. Are there other crucial configurations missing > that are necessary to pass validation? > > Thanks, > > Matt > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7Cmatt > .lernout%40stantec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638428910243105153%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=c%2BjBl2NTy%2BIZxCYZmp0CLwa2ujlD%2BC%2BfzzIQgXKw > lyI%3D&reserved=0 -- Viele Gr??e, J?rg Thomsen ********************************** Aufwind durch Wissen! Web-Seminare und Online-Schulungen bei der http://www.foss-academy.com/ ********************************** J?rg Thomsen WhereGroup GmbH Bundesallee 23 10717 Berlin Germany Tel: +49 (0)30 / 5130 278 74 Fax: +49 (0)30 / 5130 278 11 joerg.thomsen at wheregroup.com http://www.wheregroup.com/ Gesch?ftsf?hrer: Olaf Knopp, Peter Stamm Amtsgericht Bonn, HRB 9885 ------------------------------- Folgen Sie der WhereGroup auf twitter: http://twitter.com/WhereGroup_com _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users Caution: This email originated from outside of Stantec. Please take extra precaution. Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez prendre des pr?cautions suppl?mentaires. Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por favor, tome precauciones adicionales. From sethg at geographika.co.uk Wed Feb 7 08:48:27 2024 From: sethg at geographika.co.uk (Seth G) Date: Wed, 07 Feb 2024 17:48:27 +0100 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> Message-ID: <4569fee3-77e4-4362-96bc-9fa1bb87a4b3@app.fastmail.com> Hi Matthew, The CONFIG file I think is only reloaded when IIS or the Application Pool is restarted, so some changes to the regex may have been ignored. Do you have a sample request with the &map= parameter? Is there anything in a web.config file that could be stripping these out or modifying them? Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Wed, Feb 7, 2024, at 5:33 PM, Lernout, Matthew via MapServer-users wrote: > MS_MAP_PATTERN '.' also results in the same failed validation error, > which is what makes me think the issue may be with another config > setting. '.' should match any input, right? > > Just to make sure my config file was being loaded, I commented out the > MS_MAP_PATTERN entry entirely, and got the error "msCGILoadMap(): Web > application error. Required configuration value MS_MAP_PATTERN not set." > > Matt > > -----Original Message----- > From: MapServer-users On > Behalf Of J?rg Thomsen (WhereGroup) via MapServer-users > Sent: Wednesday, February 7, 2024 3:24 AM > To: mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] CGI variable "map" fails to validate > > [You don't often get email from mapserver-users at lists.osgeo.org. Learn > why this is important at https://aka.ms/LearnAboutSenderIdentification ] > > Hi, > > I'm not sure whether the pattern is correct. Shouldn't ist start with ^? > > Have yout tried > MS_MAP_PATTERN '.'? (not for production, just as a test) > > J?rg > > Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: >> Hi, >> >> Having trouble making requests since updating to 8.0.1 from 7.6.2. >> Mapserver is running in IIS using FastCGI and the configuration steps >> from: >> https://maps/ >> erver.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40stan >> tec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4d >> 80281e7%7C0%7C0%7C638428910243090792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM >> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C >> &sdata=dh%2BpgGwk5996f0GEMeX7O8rcv5J81TnPzeFNZ9XoVLs%3D&reserved=0 >> > server.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40sta >> ntec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4 >> d80281e7%7C0%7C0%7C638428910243098246%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi >> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7 >> C&sdata=OIoWHSlKZJrXWspqwVfY5jhX4lzBcQuI%2Bw5IeOVtXBA%3D&reserved=0> >> >> A new config file was added to the filesystem and server Environment >> variables as per documented requirements added since 7.6.3, but having >> trouble validating any map arguments. All requests return the error >> "CGI variable "map" fails to validate" even though regexp should pass >> based on expression tests. >> >> Here is the full config file in use: >> >> CONFIG >> ENV >> MS_MAP_PATTERN "\.map$" >> PROJ_LIB "F:/MapServer/bin/proj7/SHARE" >> CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" >> END >> >> >> PLUGINS >> "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" >> "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" >> END >> END >> >> The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 >> working before I update to the real regexp, and my understanding is it >> should map anything that ends in .map? But every map argument I throw >> in (including verified working arguments for 7.6.2) result in a >> failure to validate. Are there other crucial configurations missing >> that are necessary to pass validation? >> >> Thanks, >> >> Matt >> >> >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://list/ >> s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7Cmatt >> .lernout%40stantec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c21 >> 9a469297d3f2b4d80281e7%7C0%7C0%7C638428910243105153%7CUnknown%7CTWFpbG >> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% >> 3D%7C0%7C%7C%7C&sdata=c%2BjBl2NTy%2BIZxCYZmp0CLwa2ujlD%2BC%2BfzzIQgXKw >> lyI%3D&reserved=0 > > -- > Viele Gr??e, > J?rg Thomsen > > ********************************** > Aufwind durch Wissen! > Web-Seminare und Online-Schulungen > bei der http://www.foss-academy.com/ > ********************************** > > > J?rg Thomsen > WhereGroup GmbH > Bundesallee 23 > 10717 Berlin > Germany > > Tel: +49 (0)30 / 5130 278 74 > Fax: +49 (0)30 / 5130 278 11 > > joerg.thomsen at wheregroup.com > http://www.wheregroup.com/ > > Gesch?ftsf?hrer: > Olaf Knopp, Peter Stamm > Amtsgericht Bonn, HRB 9885 > > ------------------------------- > Folgen Sie der WhereGroup auf twitter: http://twitter.com/WhereGroup_com > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > Caution: This email originated from outside of Stantec. Please take > extra precaution. > > Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez > prendre des pr?cautions suppl?mentaires. > > Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por > favor, tome precauciones adicionales. > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users From Rob.Dennett at twdb.texas.gov Wed Feb 7 08:53:41 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Wed, 7 Feb 2024 16:53:41 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Resending this since it was too large the first time Thanks to everyone for their help so far. There are only 3 things that changed when we upgraded. One, the Postgres version, two, the PostGIS version and three, we went from Aurora serverless v1 to serverless v2. MapServer itself runs on Amazon ECS which in this case has 8 Fargate instances all running an identical version of MapServer in a Docker container. These instances are load balanced. The table in question has 3479 rows. I am not clear how to determine how many features it has. Is it one per row? My URLs look like https://mapserver.mydomain.org/?map=/path/to/my/mapfile.map I actually made two copies of the original map file and changed the config for where the MS_ERRORFILE should be written and added DEBUG 2. One of those copies had the CONNECTION string set to point at the old db. The extent listed in the map section of the .map file is EXTENT -11871597.4858696 2978893.85839647 -10409341.4604357 4369793.96473184 It does seem to zoom in a little as it loads, but I am not sure where it's getting the polygon information in the queries it's generating. I hit the two URLs again, and this time got a debug file from the .mapfile pointed at the old db that is 226 lines long and one from the one pointed at the new db that is almost 24,000 lines long. The map file for the new db has thousands of lines like this (as did the .map file pointed at the old db the 1st time I ran it): [Tue Feb 6 20:37:34 2024].128829 msPostGISLayerNextShape called. [Tue Feb 6 20:37:34 2024].128833 msPostGISReadShape called. [Tue Feb 6 20:37:34 2024].128838 msPostGISLayerNextShape called. [Tue Feb 6 20:37:34 2024].128842 msPostGISReadShape called. [Tue Feb 6 20:37:34 2024].129474 msPostGISLayerNextShape called. [Tue Feb 6 20:37:34 2024].129480 msPostGISReadShape called. The queries are the same for the new db as the were the last time I loaded the page although they occur in a different order for each. However, the .map file pointed at the old db had different values for the polygons and all the queries returned 0 rows, even though the map rendered was fine. To be clear, running either set in pgAdmin is much slower against the new db compared to the old. Additionally, the set of old queries which all return 0 results took less than 1 sec on the old db and about 15 on the new. Additinonally, I tried adding an index in the new db, but while it seems slightly faster, it still takes twice as long as the old db does without a spatial index. Here's what I ran: CREATE INDEX idx_the_table_in_question_geom ON the_table_in_question USING gist (geom); Thanks, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matt.Lernout at stantec.com Wed Feb 7 09:00:42 2024 From: Matt.Lernout at stantec.com (Lernout, Matthew) Date: Wed, 7 Feb 2024 17:00:42 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: <4569fee3-77e4-4362-96bc-9fa1bb87a4b3@app.fastmail.com> References: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> <4569fee3-77e4-4362-96bc-9fa1bb87a4b3@app.fastmail.com> Message-ID: I've ensured I am restarting the mapserv process with each config change - a quick test of commenting out the MS_MAP_PATTERN variable confirmed that is enough to refresh the config via error. Here's a snippet of a sample request. It works with my running instance of Mapserver 7.6.2 but doesn't pass validation (pattern '.') in MapServer 8.0.1. /cgi-bin/mapserv.exe?map=f:\IMS_Maps\Client\projectarea.map Matt -----Original Message----- From: Seth G Sent: Wednesday, February 7, 2024 11:48 AM To: Lernout, Matthew ; MapServer Users Subject: Re: [MapServer-users] CGI variable "map" fails to validate [You don't often get email from sethg at geographika.co.uk. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Matthew, The CONFIG file I think is only reloaded when IIS or the Application Pool is restarted, so some changes to the regex may have been ignored. Do you have a sample request with the &map= parameter? Is there anything in a web.config file that could be stripping these out or modifying them? Seth -- web:https://geographika.net/ & https://mapserverstudio.net/ twitter: @geographika On Wed, Feb 7, 2024, at 5:33 PM, Lernout, Matthew via MapServer-users wrote: > MS_MAP_PATTERN '.' also results in the same failed validation error, > which is what makes me think the issue may be with another config > setting. '.' should match any input, right? > > Just to make sure my config file was being loaded, I commented out the > MS_MAP_PATTERN entry entirely, and got the error "msCGILoadMap(): Web > application error. Required configuration value MS_MAP_PATTERN not set." > > Matt > > -----Original Message----- > From: MapServer-users On > Behalf Of J?rg Thomsen (WhereGroup) via MapServer-users > Sent: Wednesday, February 7, 2024 3:24 AM > To: mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] CGI variable "map" fails to validate > > [You don't often get email from mapserver-users at lists.osgeo.org. Learn > why this is important at https://aka.ms/LearnAboutSenderIdentification > ] > > Hi, > > I'm not sure whether the pattern is correct. Shouldn't ist start with ^? > > Have yout tried > MS_MAP_PATTERN '.'? (not for production, just as a test) > > J?rg > > Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: >> Hi, >> >> Having trouble making requests since updating to 8.0.1 from 7.6.2. >> Mapserver is running in IIS using FastCGI and the configuration steps >> from: >> https://maps/ >> erver.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40sta >> n >> tec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4 >> d >> 80281e7%7C0%7C0%7C638428910243090792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi >> M >> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7 >> C >> &sdata=dh%2BpgGwk5996f0GEMeX7O8rcv5J81TnPzeFNZ9XoVLs%3D&reserved=0 >> > server.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40st >> a >> ntec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b >> 4 >> d80281e7%7C0%7C0%7C638428910243098246%7CUnknown%7CTWFpbGZsb3d8eyJWIjo >> i >> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C% >> 7 >> C&sdata=OIoWHSlKZJrXWspqwVfY5jhX4lzBcQuI%2Bw5IeOVtXBA%3D&reserved=0> >> >> A new config file was added to the filesystem and server Environment >> variables as per documented requirements added since 7.6.3, but >> having trouble validating any map arguments. All requests return the >> error "CGI variable "map" fails to validate" even though regexp >> should pass based on expression tests. >> >> Here is the full config file in use: >> >> CONFIG >> ENV >> MS_MAP_PATTERN "\.map$" >> PROJ_LIB "F:/MapServer/bin/proj7/SHARE" >> CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" >> END >> >> >> PLUGINS >> "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" >> "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" >> END >> END >> >> The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 >> working before I update to the real regexp, and my understanding is >> it should map anything that ends in .map? But every map argument I >> throw in (including verified working arguments for 7.6.2) result in a >> failure to validate. Are there other crucial configurations missing >> that are necessary to pass validation? >> >> Thanks, >> >> Matt >> >> >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://list/ >> s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7Cmat >> t >> .lernout%40stantec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c2 >> 1 >> 9a469297d3f2b4d80281e7%7C0%7C0%7C638428910243105153%7CUnknown%7CTWFpb >> G >> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 >> % >> 3D%7C0%7C%7C%7C&sdata=c%2BjBl2NTy%2BIZxCYZmp0CLwa2ujlD%2BC%2BfzzIQgXK >> w >> lyI%3D&reserved=0 > > -- > Viele Gr??e, > J?rg Thomsen > > ********************************** > Aufwind durch Wissen! > Web-Seminare und Online-Schulungen > bei der > http://www.f/ > oss-academy.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc803 > 339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7 > C638429213306207068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj > oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=N3W9lnwc3z > oNjXmUW993zzswV1YBXAED99N2OoI615g%3D&reserved=0 > ********************************** > > > J?rg Thomsen > WhereGroup GmbH > Bundesallee 23 > 10717 Berlin > Germany > > Tel: +49 (0)30 / 5130 278 74 > Fax: +49 (0)30 / 5130 278 11 > > joerg.thomsen at wheregroup.com > http://www.w/ > heregroup.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc80333 > 9354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7C6 > 38429213306211969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=2eHjEWZMX%2B > r5Ciu10Erm5j6rVD%2BOWyIbwBtePd2gDJ8%3D&reserved=0 > > Gesch?ftsf?hrer: > Olaf Knopp, Peter Stamm > Amtsgericht Bonn, HRB 9885 > > ------------------------------- > Folgen Sie der WhereGroup auf twitter: > http://twitt/ > er.com%2FWhereGroup_com&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6b > fc803339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0% > 7C0%7C638429213306217142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=nIup% > 2BinkBHUNOhGb4s5sM8IrV58Ar5t%2B5NRsAkZDzkc%3D&reserved=0 > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306222189%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=T9CIsI3SiBLp%2BDt9jsu9Q9%2FNVmUd1hCtEgoOgYtzdt8% > 3D&reserved=0 > Caution: This email originated from outside of Stantec. Please take > extra precaution. > > Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez > prendre des pr?cautions suppl?mentaires. > > Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por > favor, tome precauciones adicionales. > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306226943%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=fxiaaf1f3%2F2cUrMl%2BBjIRBi8BDPaL8Tzyd3L9qPQMdA% > 3D&reserved=0 Caution: This email originated from outside of Stantec. Please take extra precaution. Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez prendre des pr?cautions suppl?mentaires. Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por favor, tome precauciones adicionales. From jukka.rahkonen at maanmittauslaitos.fi Wed Feb 7 10:05:18 2024 From: jukka.rahkonen at maanmittauslaitos.fi (Rahkonen Jukka) Date: Wed, 7 Feb 2024 18:05:18 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> <4569fee3-77e4-4362-96bc-9fa1bb87a4b3@app.fastmail.com> Message-ID: Hi, "." works for me. I have a Windows env set first as set MAPSERVER_CONFIG_FILE=\ms4w\ms4w.conf These are the meaningful lines in the "ms4w.conf" file. CONFIG ENV MS_MAP_PATTERN "." END END Test proves that validation is successful http://localhost:8060/cgi-bin/mapserv.exe?map=f:\IMS_Maps\Client\projectarea.map msLoadMap(): Unable to access file. (f:\IMS_Maps\Client\projectarea.map) -Jukka Rahkonen- -----Alkuper?inen viesti----- L?hett?j?: MapServer-users Puolesta Lernout, Matthew via MapServer-users L?hetetty: keskiviikko 7. helmikuuta 2024 19.01 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] CGI variable "map" fails to validate I've ensured I am restarting the mapserv process with each config change - a quick test of commenting out the MS_MAP_PATTERN variable confirmed that is enough to refresh the config via error. Here's a snippet of a sample request. It works with my running instance of Mapserver 7.6.2 but doesn't pass validation (pattern '.') in MapServer 8.0.1. /cgi-bin/mapserv.exe?map=f:\IMS_Maps\Client\projectarea.map Matt -----Original Message----- From: Seth G Sent: Wednesday, February 7, 2024 11:48 AM To: Lernout, Matthew ; MapServer Users Subject: Re: [MapServer-users] CGI variable "map" fails to validate [You don't often get email from sethg at geographika.co.uk. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Matthew, The CONFIG file I think is only reloaded when IIS or the Application Pool is restarted, so some changes to the regex may have been ignored. Do you have a sample request with the &map= parameter? Is there anything in a web.config file that could be stripping these out or modifying them? Seth -- web:https://geographika.net/ & https://mapserverstudio.net/ twitter: @geographika On Wed, Feb 7, 2024, at 5:33 PM, Lernout, Matthew via MapServer-users wrote: > MS_MAP_PATTERN '.' also results in the same failed validation error, > which is what makes me think the issue may be with another config > setting. '.' should match any input, right? > > Just to make sure my config file was being loaded, I commented out the > MS_MAP_PATTERN entry entirely, and got the error "msCGILoadMap(): Web > application error. Required configuration value MS_MAP_PATTERN not set." > > Matt > > -----Original Message----- > From: MapServer-users On > Behalf Of J?rg Thomsen (WhereGroup) via MapServer-users > Sent: Wednesday, February 7, 2024 3:24 AM > To: mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] CGI variable "map" fails to validate > > [You don't often get email from mapserver-users at lists.osgeo.org. Learn > why this is important at https://aka.ms/LearnAboutSenderIdentification > ] > > Hi, > > I'm not sure whether the pattern is correct. Shouldn't ist start with ^? > > Have yout tried > MS_MAP_PATTERN '.'? (not for production, just as a test) > > J?rg > > Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: >> Hi, >> >> Having trouble making requests since updating to 8.0.1 from 7.6.2. >> Mapserver is running in IIS using FastCGI and the configuration steps >> from: >> https://maps/ >> erver.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40sta >> n >> tec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4 >> d >> 80281e7%7C0%7C0%7C638428910243090792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi >> M >> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7 >> C >> &sdata=dh%2BpgGwk5996f0GEMeX7O8rcv5J81TnPzeFNZ9XoVLs%3D&reserved=0 >> > server.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40st >> a >> ntec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b >> 4 >> d80281e7%7C0%7C0%7C638428910243098246%7CUnknown%7CTWFpbGZsb3d8eyJWIjo >> i >> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C% >> 7 >> C&sdata=OIoWHSlKZJrXWspqwVfY5jhX4lzBcQuI%2Bw5IeOVtXBA%3D&reserved=0> >> >> A new config file was added to the filesystem and server Environment >> variables as per documented requirements added since 7.6.3, but >> having trouble validating any map arguments. All requests return the >> error "CGI variable "map" fails to validate" even though regexp >> should pass based on expression tests. >> >> Here is the full config file in use: >> >> CONFIG >> ENV >> MS_MAP_PATTERN "\.map$" >> PROJ_LIB "F:/MapServer/bin/proj7/SHARE" >> CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" >> END >> >> >> PLUGINS >> "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" >> "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" >> END >> END >> >> The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 >> working before I update to the real regexp, and my understanding is >> it should map anything that ends in .map? But every map argument I >> throw in (including verified working arguments for 7.6.2) result in a >> failure to validate. Are there other crucial configurations missing >> that are necessary to pass validation? >> >> Thanks, >> >> Matt >> >> >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://list/ >> s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7Cmat >> t >> .lernout%40stantec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c2 >> 1 >> 9a469297d3f2b4d80281e7%7C0%7C0%7C638428910243105153%7CUnknown%7CTWFpb >> G >> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 >> % >> 3D%7C0%7C%7C%7C&sdata=c%2BjBl2NTy%2BIZxCYZmp0CLwa2ujlD%2BC%2BfzzIQgXK >> w >> lyI%3D&reserved=0 > > -- > Viele Gr??e, > J?rg Thomsen > > ********************************** > Aufwind durch Wissen! > Web-Seminare und Online-Schulungen > bei der > http://www.f/ > %2F&data=05%7C02%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb1ac588f3ea > 94378069c08dc27fe5c36%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638 > 429220639531250%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2 > luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=6DJDUpY4WQcsCQ > c95tGoKBRTx9Ff8naSFezd%2FDbuM7s%3D&reserved=0 > oss-academy.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc803 > 339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7 > C638429213306207068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj > oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=N3W9lnwc3z > oNjXmUW993zzswV1YBXAED99N2OoI615g%3D&reserved=0 > ********************************** > > > J?rg Thomsen > WhereGroup GmbH > Bundesallee 23 > 10717 Berlin > Germany > > Tel: +49 (0)30 / 5130 278 74 > Fax: +49 (0)30 / 5130 278 11 > > joerg.thomsen at wheregroup.com > http://www.w/ > %2F&data=05%7C02%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb1ac588f3ea > 94378069c08dc27fe5c36%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638 > 429220639536742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2 > luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=GHmLgYpKwyBx4J > BxoERE3BKI6Oll7uYA%2BJoV5Fp3VFU%3D&reserved=0 > heregroup.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc80333 > 9354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7C6 > 38429213306211969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=2eHjEWZMX%2B > r5Ciu10Erm5j6rVD%2BOWyIbwBtePd2gDJ8%3D&reserved=0 > > Gesch?ftsf?hrer: > Olaf Knopp, Peter Stamm > Amtsgericht Bonn, HRB 9885 > > ------------------------------- > Folgen Sie der WhereGroup auf twitter: > http://twitt/ > er.com%2FWhereGroup_com&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6b > fc803339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0% > 7C0%7C638429213306217142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=nIup% > 2BinkBHUNOhGb4s5sM8IrV58Ar5t%2B5NRsAkZDzkc%3D&reserved=0 > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306222189%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=T9CIsI3SiBLp%2BDt9jsu9Q9%2FNVmUd1hCtEgoOgYtzdt8% > 3D&reserved=0 > Caution: This email originated from outside of Stantec. Please take > extra precaution. > > Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez > prendre des pr?cautions suppl?mentaires. > > Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por > favor, tome precauciones adicionales. > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306226943%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=fxiaaf1f3%2F2cUrMl%2BBjIRBi8BDPaL8Tzyd3L9qPQMdA% > 3D&reserved=0 Caution: This email originated from outside of Stantec. Please take extra precaution. Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez prendre des pr?cautions suppl?mentaires. Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por favor, tome precauciones adicionales. _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users From Matt.Lernout at stantec.com Wed Feb 7 12:32:06 2024 From: Matt.Lernout at stantec.com (Lernout, Matthew) Date: Wed, 7 Feb 2024 20:32:06 +0000 Subject: [MapServer-users] CGI variable "map" fails to validate In-Reply-To: References: <09aaf146-fd34-49fa-91b5-45b85da37656@wheregroup.com> <4569fee3-77e4-4362-96bc-9fa1bb87a4b3@app.fastmail.com> Message-ID: Hi, After scratching my head for a while, realized all my testing was over HTTPS. Made a localhost connection over HTTP and the map variable was then valid. I was looking in the wrong place - the error reports a failed validation, but that's due to failed certificate trust from what I can see. Thanks for helping me steer into the real issue - I now have a direction to move in! Matt -----Original Message----- From: Rahkonen Jukka Sent: Wednesday, February 7, 2024 1:05 PM To: Lernout, Matthew ; mapserver-users at lists.osgeo.org Subject: Re: [MapServer-users] CGI variable "map" fails to validate [You don't often get email from jukka.rahkonen at maanmittauslaitos.fi. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi, "." works for me. I have a Windows env set first as set MAPSERVER_CONFIG_FILE=\ms4w\ms4w.conf These are the meaningful lines in the "ms4w.conf" file. CONFIG ENV MS_MAP_PATTERN "." END END Test proves that validation is successful http://localhost:8060/cgi-bin/mapserv.exe?map=f:\IMS_Maps\Client\projectarea.map msLoadMap(): Unable to access file. (f:\IMS_Maps\Client\projectarea.map) -Jukka Rahkonen- -----Alkuper?inen viesti----- L?hett?j?: MapServer-users Puolesta Lernout, Matthew via MapServer-users L?hetetty: keskiviikko 7. helmikuuta 2024 19.01 Vastaanottaja: mapserver-users at lists.osgeo.org Aihe: Re: [MapServer-users] CGI variable "map" fails to validate I've ensured I am restarting the mapserv process with each config change - a quick test of commenting out the MS_MAP_PATTERN variable confirmed that is enough to refresh the config via error. Here's a snippet of a sample request. It works with my running instance of Mapserver 7.6.2 but doesn't pass validation (pattern '.') in MapServer 8.0.1. /cgi-bin/mapserv.exe?map=f:\IMS_Maps\Client\projectarea.map Matt -----Original Message----- From: Seth G Sent: Wednesday, February 7, 2024 11:48 AM To: Lernout, Matthew ; MapServer Users Subject: Re: [MapServer-users] CGI variable "map" fails to validate [You don't often get email from sethg at geographika.co.uk. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Matthew, The CONFIG file I think is only reloaded when IIS or the Application Pool is restarted, so some changes to the regex may have been ignored. Do you have a sample request with the &map= parameter? Is there anything in a web.config file that could be stripping these out or modifying them? Seth -- web:https://geographika.net/ & https://mapserverstudio.net/ twitter: @geographika On Wed, Feb 7, 2024, at 5:33 PM, Lernout, Matthew via MapServer-users wrote: > MS_MAP_PATTERN '.' also results in the same failed validation error, > which is what makes me think the issue may be with another config > setting. '.' should match any input, right? > > Just to make sure my config file was being loaded, I commented out the > MS_MAP_PATTERN entry entirely, and got the error "msCGILoadMap(): Web > application error. Required configuration value MS_MAP_PATTERN not set." > > Matt > > -----Original Message----- > From: MapServer-users On > Behalf Of J?rg Thomsen (WhereGroup) via MapServer-users > Sent: Wednesday, February 7, 2024 3:24 AM > To: mapserver-users at lists.osgeo.org > Subject: Re: [MapServer-users] CGI variable "map" fails to validate > > [You don't often get email from mapserver-users at lists.osgeo.org. Learn > why this is important at https://aka.ms/LearnAboutSenderIdentification > ] > > Hi, > > I'm not sure whether the pattern is correct. Shouldn't ist start with ^? > > Have yout tried > MS_MAP_PATTERN '.'? (not for production, just as a test) > > J?rg > > Am 06.02.24 um 20:12 schrieb Lernout, Matthew via MapServer-users: >> Hi, >> >> Having trouble making requests since updating to 8.0.1 from 7.6.2. >> Mapserver is running in IIS using FastCGI and the configuration steps >> from: >> https://maps/ >> erver.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40sta >> n >> tec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b4 >> d >> 80281e7%7C0%7C0%7C638428910243090792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi >> M >> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7 >> C >> &sdata=dh%2BpgGwk5996f0GEMeX7O8rcv5J81TnPzeFNZ9XoVLs%3D&reserved=0 >> > server.org%2Finstallation%2Fiis.html&data=05%7C02%7Cmatt.lernout%40st >> a >> ntec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c219a469297d3f2b >> 4 >> d80281e7%7C0%7C0%7C638428910243098246%7CUnknown%7CTWFpbGZsb3d8eyJWIjo >> i >> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C% >> 7 >> C&sdata=OIoWHSlKZJrXWspqwVfY5jhX4lzBcQuI%2Bw5IeOVtXBA%3D&reserved=0> >> >> A new config file was added to the filesystem and server Environment >> variables as per documented requirements added since 7.6.3, but >> having trouble validating any map arguments. All requests return the >> error "CGI variable "map" fails to validate" even though regexp >> should pass based on expression tests. >> >> Here is the full config file in use: >> >> CONFIG >> ENV >> MS_MAP_PATTERN "\.map$" >> PROJ_LIB "F:/MapServer/bin/proj7/SHARE" >> CURL_CA_BUNDLE "F:\MapServer\bin\curl\curl-ca-bundle.crt" >> END >> >> >> PLUGINS >> "mssql" "F:/MapServer/bin/ms/plugins/mssql2008/msplugin_mssql2008.dll" >> "oci" "F:/MapServer/bin/ms/plugins/oci/msplugin_oracle.dll" >> END >> END >> >> The current MS_MAP_PATTERN is just a simplified test to confirm 8.0.1 >> working before I update to the real regexp, and my understanding is >> it should map anything that ends in .map? But every map argument I >> throw in (including verified working arguments for 7.6.2) result in a >> failure to validate. Are there other crucial configurations missing >> that are necessary to pass validation? >> >> Thanks, >> >> Matt >> >> >> _______________________________________________ >> MapServer-users mailing list >> MapServer-users at lists.osgeo.org >> https://list/ >> s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7Cmat >> t >> .lernout%40stantec.com%7C34322cf56ae745f61ae908dc27b617c5%7C413c6f2c2 >> 1 >> 9a469297d3f2b4d80281e7%7C0%7C0%7C638428910243105153%7CUnknown%7CTWFpb >> G >> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 >> % >> 3D%7C0%7C%7C%7C&sdata=c%2BjBl2NTy%2BIZxCYZmp0CLwa2ujlD%2BC%2BfzzIQgXK >> w >> lyI%3D&reserved=0 > > -- > Viele Gr??e, > J?rg Thomsen > > ********************************** > Aufwind durch Wissen! > Web-Seminare und Online-Schulungen > bei der > http://www.f/ > %2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C10165ecd716b4149010108 > dc28075d52%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7C63842925930776 > 7073%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT > iI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=c%2F6yf3KoD736puorBam%2BF > h0CPIvAZHeM9JE%2BaLLh40A%3D&reserved=0 > %2F&data=05%7C02%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb1ac588f3ea > 94378069c08dc27fe5c36%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638 > 429220639531250%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2 > luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=6DJDUpY4WQcsCQ > c95tGoKBRTx9Ff8naSFezd%2FDbuM7s%3D&reserved=0 > oss-academy.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc803 > 339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7 > C638429213306207068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj > oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=N3W9lnwc3z > oNjXmUW993zzswV1YBXAED99N2OoI615g%3D&reserved=0 > ********************************** > > > J?rg Thomsen > WhereGroup GmbH > Bundesallee 23 > 10717 Berlin > Germany > > Tel: +49 (0)30 / 5130 278 74 > Fax: +49 (0)30 / 5130 278 11 > > joerg.thomsen at wheregroup.com > http://www.w/ > %2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C10165ecd716b4149010108 > dc28075d52%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7C63842925930777 > 1350%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT > iI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=%2Bsjgvauyi7OelgrSWhkxILz > 2UenzLzP4UCdinOXxVKs%3D&reserved=0 > %2F&data=05%7C02%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb1ac588f3ea > 94378069c08dc27fe5c36%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638 > 429220639536742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2 > luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=GHmLgYpKwyBx4J > BxoERE3BKI6Oll7uYA%2BJoV5Fp3VFU%3D&reserved=0 > heregroup.com%2F&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6bfc80333 > 9354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0%7C0%7C6 > 38429213306211969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=2eHjEWZMX%2B > r5Ciu10Erm5j6rVD%2BOWyIbwBtePd2gDJ8%3D&reserved=0 > > Gesch?ftsf?hrer: > Olaf Knopp, Peter Stamm > Amtsgericht Bonn, HRB 9885 > > ------------------------------- > Folgen Sie der WhereGroup auf twitter: > http://twitt/ > er.com%2FWhereGroup_com&data=05%7C02%7CMatt.Lernout%40stantec.com%7C6b > fc803339354908ddc508dc27fca7b1%7C413c6f2c219a469297d3f2b4d80281e7%7C0% > 7C0%7C638429213306217142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=nIup% > 2BinkBHUNOhGb4s5sM8IrV58Ar5t%2B5NRsAkZDzkc%3D&reserved=0 > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306222189%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=T9CIsI3SiBLp%2BDt9jsu9Q9%2FNVmUd1hCtEgoOgYtzdt8% > 3D&reserved=0 > Caution: This email originated from outside of Stantec. Please take > extra precaution. > > Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez > prendre des pr?cautions suppl?mentaires. > > Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por > favor, tome precauciones adicionales. > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://list/ > s.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-users&data=05%7C02%7CMatt > .Lernout%40stantec.com%7C6bfc803339354908ddc508dc27fca7b1%7C413c6f2c21 > 9a469297d3f2b4d80281e7%7C0%7C0%7C638429213306226943%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C0%7C%7C%7C&sdata=fxiaaf1f3%2F2cUrMl%2BBjIRBi8BDPaL8Tzyd3L9qPQMdA% > 3D&reserved=0 Caution: This email originated from outside of Stantec. Please take extra precaution. Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez prendre des pr?cautions suppl?mentaires. Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por favor, tome precauciones adicionales. _______________________________________________ MapServer-users mailing list MapServer-users at lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users Caution: This email originated from outside of Stantec. Please take extra precaution. Attention: Ce courriel provient de l'ext?rieur de Stantec. Veuillez prendre des pr?cautions suppl?mentaires. Atenci?n: Este correo electr?nico proviene de fuera de Stantec. Por favor, tome precauciones adicionales. From Rob.Dennett at twdb.texas.gov Thu Feb 8 07:31:08 2024 From: Rob.Dennett at twdb.texas.gov (Rob Dennett) Date: Thu, 8 Feb 2024 15:31:08 +0000 Subject: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11? In-Reply-To: References: <39b79f36-0a12-46db-9ae9-ce0c6c32f48b@email.android.com> Message-ID: Something that's curious is that while the old db takes about 5 seconds to run the queries it generates, the actual web page loads in about 2 seconds. Another mystery to add to the pile. Thanks, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.basques at ci.stpaul.mn.us Fri Feb 9 07:37:24 2024 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Fri, 9 Feb 2024 15:37:24 +0000 Subject: [MapServer-users] OSGeo Twin Cities Local Chapter Monthly Meeting (aka TCMUG) Message-ID: All, This month?s meeting will be at the https://www.lakemonsterbrewing.com/ Date: Feb 14thth, 4:30 ? 6:00 Place: Lake Monster (google map link) in Inver Grove Heights, Mn. See you there. Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.basques at ci.stpaul.mn.us Fri Feb 9 07:44:23 2024 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Fri, 9 Feb 2024 15:44:23 +0000 Subject: [MapServer-users] [CORRECTION] OSGeo Twin Cities Local Chapter Monthly Meeting (aka TCMUG) Message-ID: Sorry everyone, Friday and not thinking straight . . . (Thanks Pete) All, This month?s meeting will be at the https://www.lakemonsterbrewing.com/ Date: Feb 14thth, 4:30 ? 6:00 Place: Lake Monster (google map link) in Saint Paul, Mn. (Midway area). See you there. Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.basques at ci.stpaul.mn.us Mon Feb 12 04:33:06 2024 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Mon, 12 Feb 2024 12:33:06 +0000 Subject: [MapServer-users] [RESCHEDULE] OSGeo Twin Cities Local Chapter Monthly Meeting (aka TCMUG) Message-ID: It was pointed out to me that the meeting was on Valentine?s day this week, so moving to the following week. New DATE ? Feb 21st, see below . . . All, This month?s meeting will be at the https://www.lakemonsterbrewing.com/ Date: Feb 14thth Feb 21st , 4:30 ? 6:00 Place: Lake Monster (google map link) in Saint Paul, Mn. (Midway area). See you there. Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlos.padron at gigaclear.com Mon Feb 12 04:33:06 2024 From: carlos.padron at gigaclear.com (Carlos Padron Florez) Date: Mon, 12 Feb 2024 12:33:06 +0000 Subject: [MapServer-users] filter issues with WFS Message-ID: Hi, I have a WFS version 2.0.0 and I want to request from it features using a filter but I get this error: msWFSGetFeature(): WFS server error. Invalid or Unsupported FILTER in GetFeature : [signature_261337870] Gigaclear ltd, Building 1, Wyndyke Furlong, Abingdon, Oxfordshire, OX14 1UQ Gigaclear plc is registered in England and Wales with company number 07476617 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Outlook-signature_.png Type: image/png Size: 10923 bytes Desc: Outlook-signature_.png URL: From gkvoelkl at nelson-games.de Wed Feb 14 01:20:04 2024 From: gkvoelkl at nelson-games.de (=?utf-8?Q? G._V=C3=B6lkl ?=) Date: Wed, 14 Feb 2024 10:20:04 +0100 Subject: [MapServer-users] Mapserver and keycloak Message-ID: <81C05A0A-5386-4F89-B509-EF76D1851DB7@nelson-games.de> Hello Any information about using mapserver with keycloak? Thanks Best regards Gerhard Von meinem iPhone gesendet From mbojko at cloudferro.com Fri Feb 16 07:31:03 2024 From: mbojko at cloudferro.com (=?utf-8?B?TWljaGHFgiBCb2prbw==?=) Date: Fri, 16 Feb 2024 15:31:03 +0000 Subject: [MapServer-users] Pass time range in WCS 1.0.0 request GetCoverage Message-ID: Hello, I am trying to build WCS service with usage of postgis database, but now I am unable to pass timerange in request to server. In my postgis database I have specified column for geometry and timestamp and request that use BBOX and TIME (single value) work fine. The problem is I cannot pass multiple dates or date range. When providing TIME in such manner, I receive this error message: "WCS server error. Temporal ranges are not supported, only individual values". In official documentation it is stated that it's possible to pass at least two TIME parameters at once in version 1.0.0, but it doesn't work for me (https://mapserver.org/pl/ogc/wcs_server.html#test-with-a-getcoverage-request). Is there specified way to pass timeranges/timelist (I didn't find any information about it)? Or maybe it isn't possible to pass multiple values? But when I look into mapserver logs I see that postgis query searches for records that are between selected one day, so why wouldn't it be possible to pass two days and search for records between them? Below I will paste some request and queries: Request with single TIME value that's working for me: http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10 Request with two TIME values that isn't working for me and error message: http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10,2022-03-15 WCS server error. Temporal lists are not supported, only individual values. Request with TIME range that also isn't working for me and error message: http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10/2022-03-15 WCS server error. Temporal ranges are not supported, only individual values. Postgis query from mapserver logs when request contains only one TIME value: select "timestamp"::text,"cloudless_10"::text,"epsg"::text,ST_AsBinary(("geometry"),'NDR') as geom,"unique_id"::text from (select * from mrc where maxcc <= 100 AND (pid = '-1' OR '-1' = '-1') AND (tile = '-1' OR '-1' = '-1') order by maxcc desc) as subquerry where ST_Intersects("geometry", ST_GeomFromText('POLYGON((3572710 6277224,3572710 6334334,3637738 6334334,3637738 6277224,3572710 6277224))',3857)) and ("timestamp" between date_trunc('day',date '2022-03-10') and date_trunc('day',date '2022-03-10') + interval '1 day' - interval '1 second') So, is there any possibility to use multiple TIME values in one GetCoverage request? Best regards, Micha? Micha? Bojko Trainee CloudFerro S.A. office: Fabryczna 5 ????? m: 502 705 120 00-446 Warszawa, Poland ???e: mbojko at cloudferro.com [https://i.imgur.com/XRnmHDu.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Sat Feb 17 03:23:18 2024 From: sethg at geographika.co.uk (Seth G) Date: Sat, 17 Feb 2024 12:23:18 +0100 Subject: [MapServer-users] Pass time range in WCS 1.0.0 request GetCoverage In-Reply-To: References: Message-ID: <3fb1ec67-e7f8-4e24-92d9-82548e5d7dc2@app.fastmail.com> Hi Micha?, Looking at the code, I think this is an error in the documentation - time ranges in WCS 1.0 do not seem to be supported. There are no tests that cover time ranges either. You could use runtime substitution to create your own queries with custom parameters e.g. START_DATE=2022-03-10&END_DATE=2022-04-10 The section at https://mapserver.org/ogc/wcs_server.html#spatio-temporal-indexes recommends using a TILEINDEX for the layer. Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Fri, Feb 16, 2024, at 4:31 PM, Micha? Bojko via MapServer-users wrote: > Hello, > I am trying to build WCS service with usage of postgis database, but now I am unable to pass timerange in request to server. In my postgis database I have specified column for geometry and timestamp and request that use BBOX and TIME (single value) work fine. The problem is I cannot pass multiple dates or date range. When providing TIME in such manner, I receive this error message: "WCS server error. Temporal ranges are not supported, only individual values". In official documentation it is stated that it's possible to pass at least two TIME parameters at once in version 1.0.0, but it doesn't work for me (https://mapserver.org/pl/ogc/wcs_server.html#test-with-a-getcoverage-request). Is there specified way to pass timeranges/timelist (I didn't find any information about it)? Or maybe it isn't possible to pass multiple values? But when I look into mapserver logs I see that postgis query searches for records that are between selected one day, so why wouldn't it be possible to pass two days and search for records between them? Below I will paste some request and queries: > Request with single TIME value that's working for me: > http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10 > > Request with two TIME values that isn't working for me and error message: > http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10,2022-03-15 > WCS server error. Temporal lists are not supported, only individual values. > > Request with TIME range that also isn't working for me and error message: > http://localhost/cgi-bin/mapserv?map=/var/www/my_wcs.map&SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&coverage=T36SVD&CRS=EPSG:3857&RESX=10&RESY=10&FORMAT=GTiff&BBOX=3572705,6277219,3637743,6334339&TIME=2022-03-10/2022-03-15 > WCS server error. Temporal ranges are not supported, only individual values. > > Postgis query from mapserver logs when request contains only one TIME value: > select "timestamp"::text,"cloudless_10"::text,"epsg"::text,ST_AsBinary(("geometry"),'NDR') as geom,"unique_id"::text from (select * from mrc where maxcc <= 100 AND (pid = '-1' OR '-1' = '-1') AND (tile = '-1' OR '-1' = '-1') order by maxcc desc) as subquerry where ST_Intersects("geometry", ST_GeomFromText('POLYGON((3572710 6277224,3572710 6334334,3637738 6334334,3637738 6277224,3572710 6277224))',3857)) and ("timestamp" between date_trunc('day',date '2022-03-10') and date_trunc('day',date '2022-03-10') + interval '1 day' - interval '1 second') > > So, is there any possibility to use multiple TIME values in one GetCoverage request? > Best regards, > Micha? > > *Micha? Bojko* > Trainee > CloudFerro S.A. > office: Fabryczna 5 ????? *m:* 502 705 120 > 00-446 Warszawa, Poland ???*e:* mbojko at cloudferro.com > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sethg at geographika.co.uk Sat Feb 17 03:36:12 2024 From: sethg at geographika.co.uk (Seth G) Date: Sat, 17 Feb 2024 12:36:12 +0100 Subject: [MapServer-users] filter issues with WFS In-Reply-To: References: Message-ID: <75203c61-334c-4cd8-8b5d-a154792a5bdd@app.fastmail.com> Hi Carlos, If you add DEBUG 5 to the LAYER, and set the CONFIG MS_ERRORFILE do you get more details in the logs for why the filter is failing? The error indicates MapServer cannot parse the filter. The unencoded filter is shown below. wkb_geometry 448535 197714 448849 197791 wkb_geometry POLYGON((447915.65299999999115244 195673.18600000001606531, 449272.48200000001816079 195673.18600000001606531, 449272.48200000001816079 200345.65200000000186265, 447915.65299999999115244 200345.65200000000186265, 447915.65299999999115244 195673.18600000001606531)) I'm guessing the geomFromWKT is not supported, so the filter should pass a GML Polygon object to filter by using an intersecting geometry. See https://mapserver.org/ogc/filter_encoding.html#tests for an example Geometry 135.5329,34.6624 135.4921,34.8153 135.3673,34.7815 135.3800,34.6216 135.5361,34.6210 135.5329,34.6624 Seth -- web:https://geographika.net & https://mapserverstudio.net twitter: @geographika On Mon, Feb 12, 2024, at 1:33 PM, Carlos Padron Florez via MapServer-users wrote: > Hi, > > I have a WFS version 2.0.0 and I want to request from it features using a filter but I get this error: > msWFSGetFeature(): WFS server error. Invalid or Unsupported FILTER in GetFeature : > The request is the following: > > https://SERVER/wfs?SERVICE=WFS&REQUEST=GetFeature&VERSION=2.0.0&TYPENAMES=ms:layername&STARTINDEX=0&COUNT=1000000&SRSNAME=urn:ogc:def:crs:EPSG::27700&FILTER=%3Cfes:Filter xmlns:fes%3D%22http://www.opengis.net/fes/2.0%22 xmlns:gml%3D%22http://www.opengis.net/gml/3.2%22%3E%0A %3Cfes:And%3E%0A %3Cfes:BBOX%3E%0A %3Cfes:ValueReference%3Ewkb_geometry%3C/fes:ValueReference%3E%0A %3Cgml:Envelope srsName%3D%22urn:ogc:def:crs:EPSG::27700%22%3E%0A %3Cgml:lowerCorner%3E448535 197714%3C/gml:lowerCorner%3E%0A %3Cgml:upperCorner%3E448849 197791%3C/gml:upperCorner%3E%0A %3C/gml:Envelope%3E%0A %3C/fes:BBOX%3E%0A %3Cfes:Intersects xmlns:fes%3D%22http://www.opengis.net/fes/2.0%22%3E%0A %3Cfes:ValueReference%3Ewkb_geometry%3C/fes:ValueReference%3E%0A %3Cfes:Function xmlns:fes%3D%22http://www.opengis.net/fes/2.0%22 name%3D%22geomFromWKT%22%3E%0A %3Cfes:Literal xmlns:fes%3D%22http://www.opengis.net/fes/2.0%22%3EPOLYGON((447915.65299999999115244 195673.18600000001606531, 449272.48200000001816079 195673.18600000001606531, 449272.48200000001816079 200345.65200000000186265, 447915.65299999999115244 200345.65200000000186265, 447915.65299999999115244 195673.18600000001606531))%3C/fes:Literal%3E%0A %3C/fes:Function%3E%0A %3C/fes:Intersects%3E%0A %3C/fes:And%3E%0A%3C/fes:Filter%3E%0A > > The same request, with obvious changes of server name, works on Geoserver. This request is generated by Qgis so I don't if I can fix it via Mapserver or via QGIS. > Any ideas? > > Kind regards > > *Carlos Padr?n, MRes *| Geomatics Team Lead > > > > D: 01865 664120 W: www.gigaclear.com > > > > > > signature_261337870 > > Gigaclear ltd, Building 1, Wyndyke Furlong, Abingdon, Oxfordshire, OX14 1UQ > > Gigaclear plc is registered in England and Wales with company number 07476617 > > > > _______________________________________________ > MapServer-users mailing list > MapServer-users at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Outlook-signature_.png Type: image/png Size: 10923 bytes Desc: not available URL: From bob.basques at ci.stpaul.mn.us Tue Feb 20 08:15:21 2024 From: bob.basques at ci.stpaul.mn.us (Bob Basques) Date: Tue, 20 Feb 2024 16:15:21 +0000 Subject: [MapServer-users] [REMINDER - RESCHEDULE] OSGeo Twin Cities Local Chapter Monthly Meeting (aka TCMUG) Message-ID: It was pointed out to me that the meeting was on Valentine?s day this week, so moving to the following week. New DATE ? Feb 21st, see below . . . All, This month?s meeting will be at the https://www.lakemonsterbrewing.com/ Date: Feb 14thth Feb 21st , 4:30 ? 6:00 Place: Lake Monster (google map link) in Saint Paul, Mn. (Midway area). See you there. Bobb Get me in Teams PW19-S295-C024 -------------- next part -------------- An HTML attachment was scrubbed... URL: