[MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11?

[Rob Dennett]

So, after further analysis of the queries, when pointed at the old db, the mapserver output contains 4 queries which map to the 4 layers in the .map file, one for polygon, then line, then point, then centroid.  These 4 are then repeated with different polygons.  The queries all look similar, e.g.

select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','the_table_in_question','geom')) and ("featuretyp"::text = 'polygon');

When run against the new db (with the same data, mind you, just a different version of Postgres and Postgis), the queries are polygon, polygon, line, line, point, point and centroid, centroid.  While the contents of the POLYGON() function are the same for each set of 4 queries, they don't completely match what's seen in the other debug output file.  That is, only every other number matches:

old
-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378
new
-11268652.4579138 2507134.52775378,-11268652.4579138 3754586.82936786,-10021200.1562997 3754586.82936786,-10021200.1562997 2507134.52775378,-11268652.4579138 2507134.52775378

What is POLYGON in the above query?  Is it the same as ST_Polygon?  What do the numbers mean?

Thanks,
Rob
________________________________
From: MapServer-users <mapserver-users-bounces at lists.osgeo.org> on behalf of Rob Dennett via MapServer-users <mapserver-users at lists.osgeo.org>
Sent: Sunday, February 4, 2024 3:39 PM
To: Travis Kirstine <traviskirstine at gmail.com>
Cc: mapserver-users at lists.osgeo.org <mapserver-users at lists.osgeo.org>
Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11?


External: Beware of links/attachments.


Running

select * from pg_indexes where tablename like '%the_table_in_question%'

yields the same results for both the old and new dbs.

As mentioned, the mapserver debug output is very different from the old and new db are very different, even though the .map files are identical except for the connection string.

There are 8 select statements in each, but the statements aren't the same and the output from the new db is twice as large, about 1.9MB, as the old one.  I compiled the queries each one runs and ran them as a group against both dbs.  The old db ran both sets of queries in about 5 seconds each, whereas the new db ran the new set in about 15 seconds and the old set in about 10 seconds.

I haven't yet run VACUUM on the new db, and will be doing so I Monday, but I am just as confused as to why the debug output would be so different.  Can anyone shed some light on that?

Thanks,
Rob
________________________________
From: Travis Kirstine <traviskirstine at gmail.com>
Sent: Sunday, February 4, 2024 10:18 AM
To: Rob Dennett <Rob.Dennett at twdb.texas.gov>
Cc: mapserver-users at lists.osgeo.org <mapserver-users at lists.osgeo.org>
Subject: Re: [MapServer-users] Why is Mapserver slower in Postgres 13 than in Postgres 11?


External: Beware of links/attachments.


First thought is to check if the table has a spatial index.

https://postgis.net/workshops/postgis-intro/indexing.html

On Fri, 2 Feb 2024 at 18:37, Rob Dennett via MapServer-users <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>> wrote:
I recently upgraded our db for our Mapserver from Postgres 11 to Postgres 13 and also had to update Postgis from 2.4 to 3.4, as well as remove the postgis 2.4 raster queries and add the new postgis_raster extension.  Now I have web request which takes over 30 seconds where it used to take about 2.  I made copies of the .map file and question and pointed one at the old db and set the debug level for the map to 2 for both.

These .map files were created by someone who no longer works here and I am not a GIS professional, just a software developer, so I am having trouble understanding the output.

After hitting the web page for both, I see the debug file is twice as large for the new db as it was for the old one.  The .map file has 4 layers, three of which have a status of on, one for polygons, one for lines and one for point and one called "AllSources" (which is off).  For the first 3, they contain

DATA "geom from the_table_in_question"

which is a field of type geometry.  In the logs I see lines like

select "sourceid"::text,"featuretyp"::text,"cartodb_id"::text,"name"::text,"sourcetype"::text,"drawingord"::text,"isnew"::text,ST_AsBinary(("geom"),'NDR') as geom,"cartodb_id"::text from the_table_in_question where "geom" && ST_GeomFromText('POLYGON((-12520996.7293382 2507134.52775378,-12520996.7293382 3754586.82936786,-11273544.4277241 3754586.82936786,-11273544.4277241 2507134.52775378,-12520996.7293382 2507134.52775378))',find_srid('','iswp_sourcefeatures2022','geom')) and ("featuretyp"::text = 'polygon')

There are 8 of these in each output file, but their contents don't match.  I am not sure what's generating them.  I do note that they, along with SELECT geom from ...; statements take roughly equal time whether executed against the old db or the new one.

What is generating these log statements?

Why does what is for all intents and purposes the same .map file produce different output?  I know it's because I changed the db engine and postgis, but if you could be more specific, that would be very helpful.

Thanks,
Rob
_______________________________________________
MapServer-users mailing list
MapServer-users at lists.osgeo.org<mailto:MapServer-users at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/mapserver-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20240205/b95a9cee/attachment-0001.htm>