[mapserver-users] Corrupted PNG issue

Fri Nov 19 11:01:32 PST 2021

Thanks for the feedback Jukka, Steve, and Seth.

Using mapserv.exe –nh at the command line produces the same corrupt PNGs as I get through a web server.

Using MapServer 7.7.0dev from MS4W as Jukka suggested produces good PNGs, so that’s a viable alternative for me.

I’ve done some playing around with different versions available from GISInternals and it appears that PNG images generate fine up through their GDAL 3.2.1 and MapServer 7.6.2 version (release-1900-x64-gdal-3-2-1-mapserver-7-6-2), but started producing corrupt PNGs in their GDAL 3.2.2 and MapServer 7.6.2 version (release-1928-x64-gdal-3-2-2-mapserver-7-6-2).

If anyone with Windows wants to attempt to reproduce, the older GISInternals versions are available here: https://www.gisinternals.com/archive.php

And the .map file I’m using for testing this is just hitting a publicly-available WMS.

MAP
  NAME "MAP"
  CONFIG "MS_ERRORFILE" "c:/temp/ms_error.txt"
  CONFIG "PROJ_LIB" "C:/ms/projlib/"
  EXTENT -20037508.342789244 -20037508.342789244 20037508.342789244 20037508.342789244
  SIZE 256 256
  SYMBOLSET "c:\ms\symbols.txt"
  FONTSET "c:\ms\fonts.txt"
  IMAGECOLOR 255 255 255
  TRANSPARENT ON
  DEFRESOLUTION 72
  RESOLUTION 72
  UNITS meters
  PROJECTION "init=epsg:3857" END
  WEB
    METADATA
      "wms_enable_request" "*"
    END
  END
  LAYER
    STATUS DEFAULT TYPE RASTER NAME "WMS_DRG"
    CONNECTIONTYPE WMS CONNECTION "https://basemap.nationalmap.gov/arcgis/services/USGSTopo/MapServer/WmsServer?"
    PROJECTION "init=epsg:3857" END
    METADATA
      "wms_srs"  "EPSG:3857"
      "wms_name"  "0"
      "wms_server_version"  "1.1.1"
      "wms_format"  "image/png"
    END
  END
END

From: Steve Lime <sdlime at gmail.com>
Sent: Thursday, November 18, 2021 8:53 AM
To: John Huotari <John.Huotari at RCIS.com>
Cc: mapserver-users at lists.osgeo.org
Subject: [EXTERNAL] Re: [mapserver-users] Corrupted PNG issue

Hmmm... I've not run into or heard of this before although I'm not a windows user. I did a quick sanity check with a mapfile here and the latest 7.4 and 7.6 versions. While not exactly the same setup you have in terms of versions, they produce the exact same png image.

What do you get for output if you use mapserv.exe at the command line? So something like:

  mapserv.exe -nh "QUERY_STRING= map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445" > test.png

That would take PostMan and the web server out of the picture. Is there any chance different versions of libpng are being used? What are you using to manage the tiles?

--Steve

On Wed, Nov 17, 2021 at 4:57 PM John Huotari via MapServer-users <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>> wrote:
I’m attempting to upgrade from MapServer 7.6.1 to 7.6.4 via compiled packages obtained from GISInternals.  I’m running it on Windows/IIS and whereas 7.6.1 was generating .PNG tiles perfectly for me, after upgrading to 7.6.4, the .PNGs being created appear to be corrupt.  I can replace the 7.6.4 exe and dlls with 7.6.1 versions and the PNG images generate fine again, so while there are quite a few places that could introduce an issue, with the exception of a change to MapServer everything would be identical in my stack between having the issue in 7.6.4 and not in 7.6.1.

The good headers from 7.6.1 look like this

89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52

and the corrupted ones from 7.6.4 look like this

89 50 4e 47 0d 0d 0a 1a 0d 0a 00 00 00 0d 49 48 44 52

It appears that the 0a values from the valid header are being converted to 0d 0a.  I might be wrong here, but it appears to me that something is interpreting the 0a as a line feed and given the code is running on Windows, is converting that LF into a CR LF.  The replacement doesn’t seem to be limited to the file header as I see the 7.6.4 version of the file is slightly larger (18,571 bytes instead of 18,407 bytes) and in spot checking, I’ve verified some additional 0d’s exist precede 0a within the data blocks of the .PNG.  Has anyone experienced anything like this or know of any fixes?

PNG Images produced on the server with shp2img are just fine, it’s only images produced by making a WMS request to mapserv.exe that have the issue.  An example WMS request would be

https://<Server<https://urldefense.com/v3/__https:/*3CServer__;JQ!!G6MNE2S8Nw!x_ksvCCfwXPiXznbuHZekzR6wQvjCERh2SpiB7YOgt5igg_PFub0fYqcSN2m4Zmt$> Name Removed>/mapserv.exe?map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445

Maybe I’m completely misdiagnosing the problem as these PNG image files just show up as corrupted within a browser – for example FireFox reports “The image <full URL here> cannot be displayed because it contains errors.”  The way I obtained the actual .PNG images to view in a binary editor was to use PostMan and save the body of the results.  Perhaps PostMan introduced the extra bytes when saving an unrecognizable format file to disk whereas it did not when saving a file it recognized as a valid PNG.  I can’t find anything different between the valid and invalid files beyond the extra 0d’s that have been added though, so I don’t think PostMan or anything else in the chain introduced them.

******************* PLEASE NOTE *******************
This message, along with any attachments, is for the designated recipient(s) only and may contain privileged, proprietary, or otherwise confidential information. If this message has reached you in error, kindly destroy it without review and notify the sender immediately. Any other use of such misdirected e-mail by you is prohibited. Where allowed by local law, electronic communications with Zurich and its affiliates, including e-mail and instant messaging (including content), may be scanned for the purposes of information security and assessment of internal compliance with company policy.
_______________________________________________
MapServer-users mailing list
MapServer-users at lists.osgeo.org<mailto:MapServer-users at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/mapserver-users<https://urldefense.com/v3/__https:/lists.osgeo.org/mailman/listinfo/mapserver-users__;!!G6MNE2S8Nw!x_ksvCCfwXPiXznbuHZekzR6wQvjCERh2SpiB7YOgt5igg_PFub0fYqcSI16O73t$>

******************* PLEASE NOTE *******************
This message, along with any attachments, is for the designated recipient(s) only and may contain privileged, proprietary, or otherwise confidential information. If this message has reached you in error, kindly destroy it without review and notify the sender immediately. Any other use of such misdirected e-mail by you is prohibited. Where allowed by local law, electronic communications with Zurich and its affiliates, including e-mail and instant messaging (including content), may be scanned for the purposes of information security and assessment of internal compliance with company policy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20211119/be3ce9a7/attachment-0001.html>