[mapserver-users] Corrupted PNG issue

Even Rouault even.rouault at spatialys.com
Fri Nov 19 07:11:51 PST 2021 

Ah, GDAL also changed... Zipping is done by GDAL, so could be a GDAL 
related change (there were adjustments in the past regarding ZIP64 that 
could result in some bytes being different). Did you try opening the zip 
with another utility (like 'unzip' on Linux) ?

This might be then a completely different issue than the one reported by 
John.

Le 19/11/2021 à 16:01, Seth G a écrit :
> Thanks Even, I'll give that a go (the bisect worked well for another 
> issue a few months ago).
>
> I presume the zipping issue is more likely to be in the MapServer 
> codebase than in GDAL?
> The GISInternals did switch from GDAL 2.4.4 to GDAL 3.3.3 from the 
> 7.4.3 to 7.6.4 branches.
>
> The zip exports work fine over a web browser rather than command line 
> which is strange. Maybe its related to modifications to the -nh switch.
>
> Seth
>
> --
> web:http://geographika.co.uk
> twitter: @geographika
>
>
> On Fri, Nov 19, 2021, at 2:07 PM, Even Rouault wrote:
>>
>> Could be a msIO_needBinaryStdout() missing somewhere. Seth, if you've 
>> the chance to run a git bisect session, that could probably give a 
>> strong hint of how to fix that.
>>
>> Le 19/11/2021 à 09:43, Seth G a écrit :
>>> Possibly unrelated but I ran into a similar issue exporting WFS to 
>>> zipped shapefiles.
>>> Working fine in 7-4-3 but broken in 7-6-4 (and current master) - the 
>>> zip files are corrupt, although they have an identical size.
>>>
>>> 7-zip reports "Headers Error Unconfirmed start of archive"
>>>
>>> The same command is used for both versions:
>>>
>>> mapserv -nh 
>>> "QUERY_STRING=map=my.map&service=WFS&REQUEST=GetFeature&TYPENAME=LayerName&version=2.0.0&outputformat=shapezip&srsName=EPSG:3857" 
>>> > output.zip
>>>
>>> I had a look with a hex editor but the start of the working and 
>>> corrupt zips seem identical.
>>>
>>> On the other hand all PNGs / WMS services are fine.
>>>
>>> Seth
>>>
>>>
>>> --
>>> web:http://geographika.co.uk <http://geographika.co.uk>
>>> twitter: @geographika
>>>
>>>
>>> On Thu, Nov 18, 2021, at 3:53 PM, Steve Lime wrote:
>>>> Hmmm... I've not run into or heard of this before although I'm not 
>>>> a windows user. I did a quick sanity check with a mapfile here and 
>>>> the latest 7.4 and 7.6 versions. While not exactly the same setup 
>>>> you have in terms of versions, they produce the exact same png image.
>>>>
>>>> What do you get for output if you use mapserv.exe at the command 
>>>> line? So something like:
>>>>
>>>>   mapserv.exe -nh "QUERY_STRING= 
>>>> map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445" 
>>>> > test.png
>>>>
>>>> That would take PostMan and the web server out of the picture. Is 
>>>> there any chance different versions of libpng are being used? What 
>>>> are you using to manage the tiles?
>>>>
>>>> --Steve
>>>>
>>>> On Wed, Nov 17, 2021 at 4:57 PM John Huotari via MapServer-users 
>>>> <mapserver-users at lists.osgeo.org 
>>>> <mailto:mapserver-users at lists.osgeo.org>> wrote:
>>>>
>>>>     I’m attempting to upgrade from MapServer 7.6.1 to 7.6.4 via
>>>>     compiled packages obtained from GISInternals.  I’m running it
>>>>     on Windows/IIS and whereas 7.6.1 was generating .PNG tiles
>>>>     perfectly for me, after upgrading to 7.6.4, the .PNGs being
>>>>     created appear to be corrupt.  I can replace the 7.6.4 exe and
>>>>     dlls with 7.6.1 versions and the PNG images generate fine
>>>>     again, so while there are quite a few places that could
>>>>     introduce an issue, with the exception of a change to MapServer
>>>>     everything would be identical in my stack between having the
>>>>     issue in 7.6.4 and not in 7.6.1.
>>>>
>>>>
>>>>     The good headers from 7.6.1 look like this
>>>>
>>>>
>>>>     89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52
>>>>
>>>>
>>>>     and the corrupted ones from 7.6.4 look like this
>>>>
>>>>
>>>>     89 50 4e 47 0d 0d 0a 1a 0d 0a 00 00 00 0d 49 48 44 52
>>>>
>>>>
>>>>     It appears that the 0a values from the valid header are being
>>>>     converted to 0d 0a.  I might be wrong here, but it appears to
>>>>     me that something is interpreting the 0a as a line feed and
>>>>     given the code is running on Windows, is converting that LF
>>>>     into a CR LF.  The replacement doesn’t seem to be limited to
>>>>     the file header as I see the 7.6.4 version of the file is
>>>>     slightly larger (18,571 bytes instead of 18,407 bytes) and in
>>>>     spot checking, I’ve verified some additional 0d’s exist precede
>>>>     0a within the data blocks of the .PNG.  Has anyone experienced
>>>>     anything like this or know of any fixes?
>>>>
>>>>
>>>>     PNG Images produced on the server with shp2img are just fine,
>>>>     it’s only images produced by making a WMS request to
>>>>     mapserv.exe that have the issue.  An example WMS request would be
>>>>
>>>>
>>>>     https://<Server Name
>>>>     Removed>/mapserv.exe?map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445
>>>>
>>>>
>>>>     Maybe I’m completely misdiagnosing the problem as these PNG
>>>>     image files just show up as corrupted within a browser – for
>>>>     example FireFox reports “The image <full URL here> cannot be
>>>>     displayed because it contains errors.”  The way I obtained the
>>>>     actual .PNG images to view in a binary editor was to use
>>>>     PostMan and save the body of the results.  Perhaps PostMan
>>>>     introduced the extra bytes when saving an unrecognizable format
>>>>     file to disk whereas it did not when saving a file it
>>>>     recognized as a valid PNG.  I can’t find anything different
>>>>     between the valid and invalid files beyond the extra 0d’s that
>>>>     have been added though, so I don’t think PostMan or anything
>>>>     else in the chain introduced them.
>>>>
>>>>
>>>>     ******************* PLEASE NOTE *******************
>>>>     This message, along with any attachments, is for the designated
>>>>     recipient(s) only and may contain privileged, proprietary, or
>>>>     otherwise confidential information. If this message has reached
>>>>     you in error, kindly destroy it without review and notify the
>>>>     sender immediately. Any other use of such misdirected e-mail by
>>>>     you is prohibited. Where allowed by local law, electronic
>>>>     communications with Zurich and its affiliates, including e-mail
>>>>     and instant messaging (including content), may be scanned for
>>>>     the purposes of information security and assessment of internal
>>>>     compliance with company policy.
>>>>     _______________________________________________
>>>>     MapServer-users mailing list
>>>>     MapServer-users at lists.osgeo.org
>>>>     <mailto:MapServer-users at lists.osgeo.org>
>>>>     https://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>>     <https://lists.osgeo.org/mailman/listinfo/mapserver-users>
>>>>
>>>> _______________________________________________
>>>> MapServer-users mailing list
>>>> MapServer-users at lists.osgeo.org 
>>>> <mailto:MapServer-users at lists.osgeo.org>
>>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users 
>>>> <https://lists.osgeo.org/mailman/listinfo/mapserver-users>
>>>>
>>>
>>>
>>> _______________________________________________
>>> MapServer-users mailing list
>>> MapServer-users at lists.osgeo.org  <mailto:MapServer-users at lists.osgeo.org>
>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users  <https://lists.osgeo.org/mailman/listinfo/mapserver-users>
>>>
>> -- 
>> http://www.spatialys.com  <http://www.spatialys.com>
>> My software is free, but my time generally not.
>> _______________________________________________
>> MapServer-users mailing list
>> MapServer-users at lists.osgeo.org <mailto:MapServer-users at lists.osgeo.org>
>> https://lists.osgeo.org/mailman/listinfo/mapserver-users 
>> <https://lists.osgeo.org/mailman/listinfo/mapserver-users>
>>
>
-- 
http://www.spatialys.com
My software is free, but my time generally not.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20211119/84869567/attachment.htm>

More information about the MapServer-users mailing list