[GRASS-dev] Re: [GRASS GIS] #39: r.in.srtm fails to validate zip files

[Glynn Clements]

GRASS GIS wrote:

> #39: r.in.srtm fails to validate zip files
> ------------------------+---------------------------------------------------
>   Reporter:  kyngchaos  |       Owner:  hamish     
>       Type:  defect     |      Status:  assigned   
>   Priority:  major      |   Milestone:  6.3.0      
>  Component:  default    |     Version:  unspecified
> Resolution:             |    Keywords:             
> ------------------------+---------------------------------------------------
> Changes (by hamish):
> 
>   * status:  new => assigned
>   * owner:  grass-dev at lists.osgeo.org => hamish
>  * cc: grass-dev at lists.osgeo.org (added)
> 
> Comment:
> 
>  SUS says:
>    http://www.opengroup.org/onlinepubs/009695399/utilities/file.html
> 
>  --mime is not mentioned.
> 
> 
> 
>  re "why was it changed" in r19303: it never used `zip` to test, it was
>  using `ls` to look for a file with a .zip extension. (input= refers to the
>  tile name not the file name)
> 
>  {{{
>  -ls -1 "$FILE.hgt.zip" | grep zip > /dev/null
>  -if [ $? -ne 0 ] ; then
>  +# really a ZIP file?
>  +if [ "`file -ib $FILE.hgt.zip`" != "application/x-zip" ] ; then
>     echo "$FILE.hgt.zip is no zip file."
>     exit 1
>   fi
>  }}}
> 
>  for one thing "echo $foo.zip | grep zip" was always true.
> 
>  Perhaps the reason for the test had to do with downloaders creating
>  incomplete or 0 byte files, or having wrongly named files?

If incomplete downloads are an issue, file probably won't help if you
manage to download anything at all, as it normally only checks the
first few bytes of the file. My /usr/share/misc/file/magic file has:

# ZIP archives (Greg Roelofs, c/o zip-bugs at wkuvx1.wku.edu)
0	string		PK\003\004
>4	byte		0x09		Zip archive data, at least v0.9 to extract
>4	byte		0x0a		Zip archive data, at least v1.0 to extract
>4	byte		0x0b		Zip archive data, at least v1.1 to extract
>4	byte		0x14
>>30	ubelong		!0x6d696d65	Zip archive data, at least v2.0 to extract
>0x161	string		WINZIP          Zip archive data, WinZIP self-extracting

This suggests that the first 8 bytes are sufficient in most cases, 34
bytes for v2.0, or 360 bytes for a self-extracting archive.

>  I've removed `file` and added a call to `unzip -t` in 6.3 trunk.
> 
>  waiting to backport to 6.3.0's branch and close the bug until someone
>  tests the change please.

"unzip -vb" would probably be enough to catch aborted downloads. ZIP
files have the catalog at the end of the file, so a truncated file
would normally lack a catalog.

-- 
Glynn Clements <glynn at gclements.plus.com>