<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>Hi Even,<br></div><div><br></div><div>No difference when adding in that function call, but I think it is on the right track so will try putting that call in in other places and debugging. <br></div><div><br></div><div>From the int msIO_needBinaryStdout() comments:<br></div><div><br></div><div>/*<br></div><div>/* But don't do it we are using FastCGI. We will take care of */<br></div><div>/* doing it in the libfcgi library in that case for the normal */<br></div><div>/* cgi case, and for the fastcgi case the _setmode() call */<br></div><div>/* causes a crash. */<br></div><div><br></div><div>I've no problems in my server setups as I'm using FastCGI. As this is on the commandline this is triggered, and I'm guessing it is the same issue for PNGs on the other thread if it is running under CGI rather than FastCGI.<br></div><div><br></div><div>Seth<br></div><div><br></div><div id="sig62266145"><div class="signature">--<br></div><div class="signature">web:http://geographika.co.uk<br></div><div class="signature">twitter: @geographika<br></div></div><div><br></div><div><br></div><div>On Tue, Nov 23, 2021, at 5:22 PM, Even Rouault wrote:<br></div><blockquote type="cite" id="qt" style=""><div>Hi Seth,<br></div><div> <br></div><blockquote type="cite" cite="mid:8615ba91-49e4-4b53-ad2b-db5d80a5f729@www.fastmail.com"><div>I tried with both of these in the Mapfile, and also setting
in the command window (SET CPL_CREATE_ZIP64=YES) but still
getting a corrupt zip:<br></div><div><br></div><div>CONFIG "CPL_CREATE_ZIP64" "NO"<br></div><div>CONFIG "CPL_CREATE_ZIP64" "YES"<br></div><div><br></div><div>Is there any ogr command to test the low level zip funcion in
GDAL?<br></div></blockquote><div>No<br></div><div> <br></div><blockquote type="cite" cite="mid:8615ba91-49e4-4b53-ad2b-db5d80a5f729@www.fastmail.com"><div>I thought "ogr2ogr foo.shp.zip something" command was a
higher level one which meant it might not be using the same
approach. <br></div></blockquote><div>I believe it should be representative enough of how MapServer uses
GDAL zip API.<br></div><div> <br></div><blockquote type="cite" cite="mid:8615ba91-49e4-4b53-ad2b-db5d80a5f729@www.fastmail.com"><div><br></div><div>I tried both the Windows unzip, 7-zip (x64) and opening on
Linux with Ark and unzip all with the same errors.<br></div><div><br></div><div>The strangest thing is that saving via IIS through the
browser is producing a valid zip using the same OUTPUTFORMAT..<br></div><div>I'm going to set up the msautotest and roads example in IIS
to test if that works. <br></div><div>Could it be related to a change in the Windows cmd shell?<br></div></blockquote><p>Maybe? But as I mentioned before, I'd thought this would rather
be some msIO_needBinaryStdout() call missing. Perhaps IIS already
sets stdout to binary mode ?<br></p><p>For the sake of testing, what if you add a call to
msIO_needBinaryStdout() in mapserv.c, let's say at <a class="qt-moz-txt-link-freetext" href="https://github.com/MapServer/MapServer/blob/main/mapserv.c#L243">https://github.com/MapServer/MapServer/blob/main/mapserv.c#L243</a> .
I'm not saying this is necessarily the right fix<br></p><p>Even<br></p><blockquote type="cite" cite="mid:8615ba91-49e4-4b53-ad2b-db5d80a5f729@www.fastmail.com"><div><br></div><div>Seth<br></div><div><br></div><div id="qt-sig62266145"><div class="qt-signature">--<br></div><div class="qt-signature">web:<a class="qt-moz-txt-link-freetext" href="http://geographika.co.uk">http://geographika.co.uk</a><br></div><div class="qt-signature">twitter: @geographika<br></div></div><div><br></div><div><br></div><div>On Sun, Nov 21, 2021, at 4:54 PM, Even Rouault wrote:<br></div><blockquote type="cite" id="qt-qt" style=""><p><br></p><div class="qt-qt-moz-cite-prefix">Le 21/11/2021 à 16:43, Seth G a
écrit :<br></div><blockquote type="cite" cite="mid:77f2ae3e-6514-4ae3-81b7-a9dbf0956255@www.fastmail.com"><div>Steve - in my case WMS are all fine, it is only related
to zipped outputs using OUTPUTFORMAT blocks. <br></div><div><br></div><div>I was trying to find a working commit for the bisect, but
went back to 7-2 without finding one. <br></div><div><br></div><div>I built MapServer main, and the latest 7-6, 7-4, and 7-2
branches and ran the msautotest:<br></div><div><br></div><div>cd /D D:\GitHub\mapserver\msautotest\wxs<br></div><div>mapserv -nh
"QUERY_STRING=map=wfs_ogr.map&service=WFS&REQUEST=GetFeature&TYPENAME=road&version=2.0.0&outputformat=shapezip"
> output.zip<br></div><div><br></div><div>All zips were invalid. However the same command does work
with the <a href="https://www.gisinternals.com/query.html?content=filelist&file=release-1911-x64-gdal-2-4-4-mapserver-7-4-3.zip">https://www.gisinternals.com/query.html?content=filelist&file=release-1911-x64-gdal-2-4-4-mapserver-7-4-3.zip</a> build. It looks like something changed between GDAL 2.4 and
3.3<br></div><div><br></div><div>Likely function is <span class="qt-qt-pl-c1">CPLCreateZip
(as this is used by MapServer as mentioned by Even in the
other thread</span>), found in: <a href="https://github.com/OSGeo/gdal/commits/v3.3.3/gdal/port/cpl_minizip_zip.cpp">https://github.com/OSGeo/gdal/commits/v3.3.3/gdal/port/cpl_minizip_zip.cpp</a><br></div><div><br></div><div>There are only a couple of commits between versions that
look like they might be relevant?<br></div><div><br></div><div><a href="https://github.com/OSGeo/gdal/commit/0e159e1ba8ee9e7b4a56bd0b53d16de0acf89485#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b">https://github.com/OSGeo/gdal/commit/0e159e1ba8ee9e7b4a56bd0b53d16de0acf89485#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b</a><br></div><div><br></div><div><a href="https://github.com/OSGeo/gdal/commit/edcdc5b0a57b4fab159b14180141ee42a0408830#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b">https://github.com/OSGeo/gdal/commit/edcdc5b0a57b4fab159b14180141ee42a0408830#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b</a><br></div><div><br></div><div>Is there a test / output zip file in the GDAL tests that
could be used to verify?<br></div></blockquote><p>Not necessarily generated by recent GDAL versions, and if
they were, most certainly from Linux builds<br></p><p>What is weird is that you reported that .shp.zip files
generated by recent gisinternals version with "ogr2ogr
foo.shp.zip something" were correct. <br></p><p>Hum actually I see that .shp.zip are generated with the
CPL_CREATE_ZIP64 environment variable set to NO (the default
is YES)<br></p><p>Can you try that ?<br></p><p>It might be that your .zip reader aren't Zip64 ready, or that
GDAL write them in a slightly invalid way<br></p><blockquote type="cite" cite="mid:77f2ae3e-6514-4ae3-81b7-a9dbf0956255@www.fastmail.com"><div>I have a working and invalid zip from msautotest if that
helps. <br></div></blockquote><div>yes, could be useful (but I suspect this is a difference
between regular ZIP vs Zip64 one)<br></div><div><br></div><blockquote type="cite" cite="mid:77f2ae3e-6514-4ae3-81b7-a9dbf0956255@www.fastmail.com"><div><br></div><div>Seth<br></div><div><br></div><div><br></div><div><br></div><div id="qt-qt-sig62266145"><div class="qt-qt-signature">--<br></div><div class="qt-qt-signature">web:<a class="qt-qt-moz-txt-link-freetext" href="http://geographika.co.uk">http://geographika.co.uk</a><br></div><div class="qt-qt-signature">twitter: @geographika<br></div></div><div><br></div><div><br></div><div>On Sun, Nov 21, 2021, at 3:51 PM, Steve Lime wrote:<br></div><blockquote type="cite" id="qt-qt-qt" style=""><div dir="auto"><div>This seems related... <br></div><div dir="auto"><a href="https://github.com/MapServer/MapServer/issues/6412">https://github.com/MapServer/MapServer/issues/6412</a> <br></div><div dir="auto"><br></div><div dir="auto">WMS fails but shp2img works, as does WFS.
Could it be WMS specific? Would be helpful to try
mode=map and see if that works.<br></div></div><div><div><br></div><div class="qt-qt-qt-gmail_quote"><div dir="ltr" class="qt-qt-qt-gmail_attr">On Sun, Nov 21,
2021 at 8:45 AM Seth G <<a href="mailto:sethg@geographika.co.uk">sethg@geographika.co.uk</a>>
wrote:<br></div><blockquote class="qt-qt-qt-gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><div>Thanks Jérome. and Even. Sorry I should have
checked for the headers in that zip first.<br></div><div>I'll attempt the bisecting now!<br></div><div><br></div><div>--<br></div><div>web:<a href="http://geographika.co.uk" rel="noreferrer" target="_blank">http://geographika.co.uk</a><br></div><div>twitter: @geographika<br></div><div><br></div><div>On Sun, Nov 21, 2021, at 1:12 PM, Even Rouault
wrote:<br></div><div>> Seth,<br></div><div>><br></div><div>> The .zip extension of this test file is a
bit misleading, as the HTTP <br></div><div>> headers are not stripped (testing HTTP
headers is something good to test <br></div><div>> in some tests). I've issued <br></div><div>> <a href="https://github.com/MapServer/MapServer/pull/6440" rel="noreferrer" target="_blank">https://github.com/MapServer/MapServer/pull/6440</a> to rename the expected <br></div><div>> file though to avoid the confusion<br></div><div>><br></div><div>> You can obtain a valid .zip file without
headers with<br></div><div>><br></div><div>> ../../build/mapserv -conf
../etc/mapserv.conf -nh <br></div><div>>
QUERY_STRING="map=wfs_ogr.map&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=road&featureid=road.1521&OUTPUTFORMAT=SHAPEZIP" <br></div><div>> > out.zip<br></div><div>><br></div><div>> Even<br></div><div>><br></div><div>> Le 21/11/2021 à 12:32, Seth G a écrit :<br></div><div>>> Hi all,<br></div><div>>><br></div><div>>> Following on from the post at <a href="https://lists.osgeo.org/pipermail/mapserver-users/2021-November/082429.html" rel="noreferrer" target="_blank">https://lists.osgeo.org/pipermail/mapserver-users/2021-November/082429.html</a> I've been trying to find where the cause of the
corrupt zips may be.<br></div><div>>><br></div><div>>> There is a msautotest to create zip
files and check against the expected output, however
the expected output also seems to be an invalid zip
file - I tried on both Windows and Linux.<br></div><div>>><br></div><div>>> Even stranger this is both in the main
branch (simply download the file with the following
URL to test), and also going back in the history to
2016.<br></div><div>>><br></div><div>>> <a href="https://github.com/MapServer/MapServer/blob/main/msautotest/wxs/expected/wfsogr10_shapezip.zip" rel="noreferrer" target="_blank">https://github.com/MapServer/MapServer/blob/main/msautotest/wxs/expected/wfsogr10_shapezip.zip</a><br></div><div>>><br></div><div>>> I had a working process using shapezips
in a 7.6 release of MapServer so its hard to
understand what could have changed, MapServer, GDAL,
or zip formats themselves?<br></div><div>>> I'll attempt a Git bisect but if the
issue is outside MapServer it won't find an issue.<br></div><div>>><br></div><div>>> If anyone is able to open the zip file
successfully please let me know (tried with Ark and
unzip on Linux and Windows inbuilt unzip and 7-zip
on Windows). Error seems to be "87 extra bytes at
beginning or within zip file".<br></div><div>>><br></div><div>>> Seth<br></div><div>>><br></div><div>>> --<br></div><div>>> web:<a href="http://geographika.co.uk" rel="noreferrer" target="_blank">http://geographika.co.uk</a><br></div><div>>> twitter: @geographika<br></div><div>>>
_______________________________________________<br></div><div>>> MapServer-dev mailing list<br></div><div>>> <a href="mailto:MapServer-dev@lists.osgeo.org" target="_blank">MapServer-dev@lists.osgeo.org</a><br></div><div>>> <a href="https://lists.osgeo.org/mailman/listinfo/mapserver-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/mapserver-dev</a><br></div><div>><br></div><div>> -- <br></div><div>> <a href="http://www.spatialys.com" rel="noreferrer" target="_blank">http://www.spatialys.com</a><br></div><div>> My software is free, but my time generally
not.<br></div><div>_______________________________________________<br></div><div>MapServer-dev mailing list<br></div><div><a href="mailto:MapServer-dev@lists.osgeo.org" target="_blank">MapServer-dev@lists.osgeo.org</a><br></div><div><a href="https://lists.osgeo.org/mailman/listinfo/mapserver-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/mapserver-dev</a><br></div></blockquote></div></div></blockquote><div><br></div></blockquote><pre class="qt-qt-moz-signature" cols="72">--
<a class="qt-qt-moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.
<br></pre></blockquote><div><br></div></blockquote><pre class="qt-moz-signature" cols="72">--
<a class="qt-moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.<br></pre></blockquote><div><br></div></body></html>