[gdal-dev] NITF Int16 image size limit

Wed Apr 7 01:23:47 PDT 2021

Hi,

Actually the error message doesn't look like there's a uint32 overflow 
as it displays a file offset that is beyond the 4 GB limit.

Did you check that your file size is at least 4324329780 + 4194302 = 4 
328 524 082 bytes ? I guess it is not, or then there's some mysterious 
error in the low level GDAL I/O layer.

So if your file is smaller than that,

- either it is corrupted (would be good if you could check that with a 
non-GDAL implementation)

- or there's a bug in the NITF driver in how it computes offsets. Could 
you report the values of the NITF_ABPP, NITF_IC and NITF_IMODE metadata 
reported by gdalinfo ?

Even

Le 07/04/2021 à 02:09, ni hao a écrit :
> gdalinfo output i:
> Driver: NITF/National Imagery Transmission Format
> Files: ##CH_CV_GRD\imagery\12499.ntf
> Size is 61180, 19694
> Coordinate System is `'
> GCP Projection =
> GEOGCS["WGS 84",   ... ]
> Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
> Band 2 Block=1024x1024 Type=UInt16, ColorInterp=Gray
>
> Also I am sure it is the UInt16 overflow, as the message is:
> ERROR 3: Unable to read 4194302 byte block from 4324329780
> and 2**32 = 4294967296
>
> ------------------------------------------------------------------------
> *From:* bradh at frogmouth.net <bradh at frogmouth.net>
> *Sent:* April 6, 2021 8:17 PM
> *To:* Even Rouault <even.rouault at spatialys.com>; ni hao 
> <ni_hao88 at hotmail.com>
> *Cc:* gdal-dev at lists.osgeo.org <gdal-dev at lists.osgeo.org>
> *Subject:* Re: [gdal-dev] NITF Int16 image size limit
> Also if you can show the headers (e.g. from gdalinfo), that might 
> provide indication of exactly which value is the issue.
>
> Brad
>
> On 7 Apr. 2021 04:00, Even Rouault <even.rouault at spatialys.com> wrote:
>
>     Please keep the mailing list copied (I re-added it)
>
>     This is not the most recent one (3.2.2 is), but skimming quickly
>     through release notes, I can't see anything directly related, so
>     it might still be current with the recent versions. Altough I see
>     that a bunch of "avoid unsigned integer overflow" type of fixes
>     have been done since 2.4, so some of them might have accidentally
>     fixed the issue.
>
>     If you can try with the latest version ( Docker images mentioned
>     at https://github.com/OSGeo/gdal/tree/master/gdal/docker
>     <https://github.com/OSGeo/gdal/tree/master/gdal/docker>, Conda,
>     etc .), that could be interesting.
>
>
>     Le 06/04/2021 à 19:51, ni hao a écrit :
>
>         Hi Even,
>
>         The version is 2.4.4.
>         ------------------------------------------------------------------------
>         *From:* Even Rouault <even.rouault at spatialys.com>
>         <mailto:even.rouault at spatialys.com>
>         *Sent:* April 6, 2021 2:46 PM
>         *To:* ni hao <ni_hao88 at hotmail.com>
>         <mailto:ni_hao88 at hotmail.com>; gdal-dev at lists.osgeo.org
>         <mailto:gdal-dev at lists.osgeo.org> <gdal-dev at lists.osgeo.org>
>         <mailto:gdal-dev at lists.osgeo.org>
>         *Subject:* Re: [gdal-dev] NITF Int16 image size limit
>
>         Shawn,
>
>         This sounds more like a unwanted integer overflow somewhere in
>         the NITF driver.  Is this with a recent GDAL ? If so, please
>         file a bug at https://github.com/OSGeo/gdal/issues/new
>         <https://github.com/OSGeo/gdal/issues/new> with all the
>         details needed. A link to the dataset would be ideal, but
>         otherwise please provide in the ticket description the output
>         of "gdalinfo your.nitf"
>
>         Even
>
>         Le 06/04/2021 à 19:35, ni hao a écrit :
>
>             Hi list,
>
>
>             I encountered problem ingesting a large NITF Int16 image
>             with GDAL:
>
>             NITF 2-band image in Int16 format, 4.3 GB. It has 19690
>             lines x 61180 pixels.   The upper 90% of the image looks
>             fine.   But the last 2000 lines fail to load by GDAL. 
>              That coincides with the 32-bit boundary.
>
>
>             Note that another larger NITF in 32-bit float, 10.1 GB
>             file loads fine.
>
>
>             Is there a hard limit on NITF Int16 file size?
>
>             Thanks,
>             Shawn
>
>
>             _______________________________________________
>             gdal-dev mailing list
>             gdal-dev at lists.osgeo.org  <mailto:gdal-dev at lists.osgeo.org>
>             https://lists.osgeo.org/mailman/listinfo/gdal-dev  <https://lists.osgeo.org/mailman/listinfo/gdal-dev>
>
>         -- 
>         http://www.spatialys.com  <http://www.spatialys.com>
>         My software is free, but my time generally not.
>
>     -- 
>     http://www.spatialys.com  <http://www.spatialys.com>
>     My software is free, but my time generally not.
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210407/6384b48a/attachment.html>