<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
<p>Hum, 4294967295 = 0xffffffff = max uint32 value. This really a
suspicious file size that looks like truncation at the 4 GB limit.
Are you really really sure that you tested the non-GDAL
implementation with that exact file, and not another version that
might have been truncated later ?<br>
</p>
<div class="moz-cite-prefix">Le 07/04/2021 à 14:59, ni hao a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:DM5PR02MB38628A04E03E4A902261FF12F1759@DM5PR02MB3862.namprd02.prod.outlook.com">
<div>
Hi Even,</div>
<div>
<br>
</div>
<div>
I am using GDAL 2.4.4. I may try the latest version as you
suggested.</div>
<div>
<span><br>
</span></div>
<div>
<span>The NITF file size is 4,294,967,295 bytes </span></div>
<div>
<br>
</div>
<div>
<span>The vendor used a non-GDAL implementation, and was able to
display all image.</span><br>
</div>
<div>
<span><br>
</span></div>
<div>
<span><span>Metadata:</span>
<div>
NITF_ABPP=16</div>
<div>
NITF_BLOCKA_BLOCK_COUNT=01</div>
<div>
NITF_BLOCKA_BLOCK_INSTANCE_01=01</div>
<div>
NITF_BLOCKA_FRFC_LOC_01=</div>
<div>
NITF_BLOCKA_FRLC_LOC_01=</div>
<div>
NITF_BLOCKA_LAYOVER_ANGLE_01=000</div>
<div>
NITF_BLOCKA_LRFC_LOC_01=</div>
<div>
NITF_BLOCKA_LRLC_LOC_01=</div>
<div>
NITF_BLOCKA_L_LINES_01=19694</div>
<div>
NITF_BLOCKA_N_GRAY_01=00000</div>
<div>
NITF_BLOCKA_SHADOW_ANGLE_01=180</div>
<div>
NITF_CCS_COLUMN=0</div>
<div>
NITF_CCS_ROW=0</div>
<div>
NITF_CLEVEL=07</div>
<div>
NITF_ENCRYP=0</div>
<div>
NITF_FBKGC= 0, 0, 0</div>
<div>
NITF_FDT=</div>
<div>
NITF_FHDR=NITF02.10</div>
<div>
NITF_FSCATP=</div>
<div>
NITF_FSCAUT=</div>
<div>
NITF_FSCLAS=U</div>
<div>
NITF_FSCLSY=CA</div>
<div>
NITF_FSCLTX=</div>
<div>
NITF_FSCODE=</div>
<div>
NITF_FSCOP=00000</div>
<div>
NITF_FSCPYS=00000</div>
<div>
NITF_FSCRSN=</div>
<div>
NITF_FSCTLH=</div>
<div>
NITF_FSCTLN=</div>
<div>
NITF_FSDCDT=</div>
<div>
NITF_FSDCTP=</div>
<div>
NITF_FSDCXM=</div>
<div>
NITF_FSDG=</div>
<div>
NITF_FSDGDT=</div>
<div>
NITF_FSREL=</div>
<div>
NITF_FSSRDT=</div>
<div>
NITF_FTITLE=</div>
<div>
NITF_IALVL=0</div>
<div>
NITF_IC=NC</div>
<div>
NITF_ICAT=SAR</div>
<div>
NITF_ICORDS=G</div>
<div>
NITF_IDATIM=</div>
<div>
NITF_IDLVL=</div>
<div>
NITF_IGEOLO=</div>
<div>
NITF_IID1=P1</div>
<div>
NITF_IID2=</div>
<div>
NITF_ILOC_COLUMN=0</div>
<div>
NITF_ILOC_ROW=0</div>
<div>
NITF_IMAG=1.0</div>
<div>
NITF_IMODE=P</div>
<div>
NITF_IREP=MULTI</div>
<div>
NITF_ISCATP=</div>
<div>
NITF_ISCAUT=</div>
<div>
NITF_ISCLAS=U</div>
<div>
NITF_ISCLSY=CA</div>
<div>
NITF_ISCLTX=</div>
<div>
NITF_ISCODE=</div>
<div>
NITF_ISCRSN=</div>
<div>
NITF_ISCTLH=</div>
<div>
NITF_ISCTLN=</div>
<div>
NITF_ISDCDT=</div>
<div>
NITF_ISDCTP=</div>
<div>
NITF_ISDCXM=</div>
<div>
NITF_ISDG=</div>
<div>
NITF_ISDGDT=</div>
<div>
NITF_ISORCE=</div>
<div>
NITF_ISREL=</div>
<div>
NITF_ISSRDT=</div>
<div>
NITF_ONAME=</div>
<div>
NITF_OPHONE=</div>
<div>
NITF_OSTAID=CSA</div>
<div>
NITF_PJUST=R</div>
<div>
NITF_PVTYPE=INT</div>
<div>
NITF_STYPE=BF01</div>
<span> NITF_TGTID=</span><br>
</span></div>
<hr tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><b>From:</b> Even Rouault
<a class="moz-txt-link-rfc2396E" href="mailto:even.rouault@spatialys.com"><even.rouault@spatialys.com></a><br>
<b>Sent:</b> April 7, 2021 5:23 AM<br>
<b>To:</b> ni hao <a class="moz-txt-link-rfc2396E" href="mailto:ni_hao88@hotmail.com"><ni_hao88@hotmail.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a> <a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev@lists.osgeo.org"><gdal-dev@lists.osgeo.org></a><br>
<b>Subject:</b> Re: [gdal-dev] NITF Int16 image size limit
<div> </div>
</div>
<div>
<p>Hi,</p>
<p>Actually the error message doesn't look like there's a uint32
overflow as it displays a file offset that is beyond the 4 GB
limit.</p>
<p>Did you check that your file size is at least 4324329780 +
4194302 = 4 328 524 082 bytes ? I guess it is not, or then
there's some mysterious error in the low level GDAL I/O layer.</p>
<p>So if your file is smaller than that,</p>
<p>- either it is corrupted (would be good if you could check
that with a non-GDAL implementation)</p>
<p>- or there's a bug in the NITF driver in how it computes
offsets. Could you report the values of the NITF_ABPP, NITF_IC
and NITF_IMODE metadata reported by gdalinfo ?<br>
</p>
<p>Even<br>
</p>
<div class="x_moz-cite-prefix">Le 07/04/2021 à 02:09, ni hao a
écrit :<br>
</div>
<blockquote type="cite">
<div><span>gdalinfo output i:</span><br>
</div>
<div><span>Driver: NITF/National Imagery Transmission Format
<div>Files: ##CH_CV_GRD\imagery\12499.ntf</div>
<div>Size is 61180, 19694</div>
<div>Coordinate System is `'</div>
<div>GCP Projection =</div>
GEOGCS["WGS 84", ... ]<br>
</span></div>
<div><span>Band 1 Block=1024x1024 Type=UInt16,
ColorInterp=Gray</span></div>
<div><span>Band 2 Block=1024x1024 Type=UInt16,
ColorInterp=Gray</span><br>
</div>
<div><br>
</div>
<div>Also I am sure it is the UInt16 overflow, as the message
is:</div>
<div>ERROR 3: Unable to read 4194302 byte block from
4324329780<br>
</div>
<div>and 2**32 = 4294967296</div>
<div><br>
</div>
<hr tabindex="-1">
<div id="x_divRplyFwdMsg" dir="ltr"><b>From:</b> <a
class="x_moz-txt-link-abbreviated"
href="mailto:bradh@frogmouth.net" moz-do-not-send="true">
bradh@frogmouth.net</a> <a
class="x_moz-txt-link-rfc2396E"
href="mailto:bradh@frogmouth.net" moz-do-not-send="true">
<bradh@frogmouth.net></a><br>
<b>Sent:</b> April 6, 2021 8:17 PM<br>
<b>To:</b> Even Rouault <a class="x_moz-txt-link-rfc2396E"
href="mailto:even.rouault@spatialys.com"
moz-do-not-send="true">
<even.rouault@spatialys.com></a>; ni hao <a
class="x_moz-txt-link-rfc2396E"
href="mailto:ni_hao88@hotmail.com" moz-do-not-send="true">
<ni_hao88@hotmail.com></a><br>
<b>Cc:</b> <a class="x_moz-txt-link-abbreviated"
href="mailto:gdal-dev@lists.osgeo.org"
moz-do-not-send="true">
gdal-dev@lists.osgeo.org</a> <a
class="x_moz-txt-link-rfc2396E"
href="mailto:gdal-dev@lists.osgeo.org"
moz-do-not-send="true">
<gdal-dev@lists.osgeo.org></a><br>
<b>Subject:</b> Re: [gdal-dev] NITF Int16 image size limit
<div> </div>
</div>
<div>
<div dir="auto">Also if you can show the headers (e.g. from
gdalinfo), that might provide indication of exactly which
value is the issue.
<div dir="auto"><br>
</div>
<div dir="auto">Brad</div>
</div>
<div class="x_x_gmail_extra"><br>
<div class="x_x_gmail_quote">On 7 Apr. 2021 04:00, Even
Rouault <a class="x_moz-txt-link-rfc2396E"
href="mailto:even.rouault@spatialys.com"
moz-do-not-send="true">
<even.rouault@spatialys.com></a> wrote:<br
type="attribution">
<blockquote class="x_x_quote">
<div>
<p>Please keep the mailing list copied (I re-added
it)</p>
<p>This is not the most recent one (3.2.2 is), but
skimming quickly through release notes, I can't
see anything directly related, so it might still
be current with the recent versions. Altough I see
that a bunch of "avoid unsigned integer overflow"
type of fixes have been done since 2.4, so some of
them might have accidentally fixed the issue.</p>
<p>If you can try with the latest version ( Docker
images mentioned at <a
href="https://github.com/OSGeo/gdal/tree/master/gdal/docker"
moz-do-not-send="true">
https://github.com/OSGeo/gdal/tree/master/gdal/docker</a>, Conda, etc
.), that could be interesting.<br>
</p>
<p><br>
</p>
<div>Le 06/04/2021 à 19:51, ni hao a écrit :<br>
</div>
<blockquote>
<div>Hi Even,</div>
<div><br>
</div>
<div>The version is 2.4.4.</div>
<hr>
<div dir="ltr"><b>From:</b> Even Rouault <a
href="mailto:even.rouault@spatialys.com"
moz-do-not-send="true">
<even.rouault@spatialys.com></a><br>
<b>Sent:</b> April 6, 2021 2:46 PM<br>
<b>To:</b> ni hao <a
href="mailto:ni_hao88@hotmail.com"
moz-do-not-send="true"><ni_hao88@hotmail.com></a>;
<a href="mailto:gdal-dev@lists.osgeo.org"
moz-do-not-send="true">gdal-dev@lists.osgeo.org</a>
<a href="mailto:gdal-dev@lists.osgeo.org"
moz-do-not-send="true">
<gdal-dev@lists.osgeo.org></a><br>
<b>Subject:</b> Re: [gdal-dev] NITF Int16 image
size limit
<div> </div>
</div>
<div>
<p>Shawn,</p>
<p>This sounds more like a unwanted integer
overflow somewhere in the NITF driver. Is
this with a recent GDAL ? If so, please file a
bug at
<a
href="https://github.com/OSGeo/gdal/issues/new"
moz-do-not-send="true">https://github.com/OSGeo/gdal/issues/new</a>
with all the details needed. A link to the
dataset would be ideal, but otherwise please
provide in the ticket description the output
of "gdalinfo your.nitf"</p>
<p>Even<br>
</p>
<div>Le 06/04/2021 à 19:35, ni hao a écrit :<br>
</div>
<blockquote>
<div>
<p>Hi list,</p>
<p><br>
</p>
<p>I encountered problem ingesting a
large NITF Int16 image with GDAL:</p>
<p>NITF 2-band image in Int16 format, 4.3
GB. It has 19690 lines x 61180 pixels.
The upper 90% of the image looks fine.
But the last 2000 lines fail to load by
GDAL. That coincides with the 32-bit
boundary. </p>
<p><br>
</p>
<p>Note that another larger NITF in 32-bit
float, 10.1 GB file loads fine. </p>
<p><br>
</p>
<p>Is there a hard limit on NITF Int16
file size?</p>
<div>
<div>
<div>
<div> </div>
<div>Thanks,<br>
Shawn</div>
</div>
</div>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" moz-do-not-send="true">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre>--
<a href="http://www.spatialys.com" moz-do-not-send="true">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote>
<pre>--
<a href="http://www.spatialys.com" moz-do-not-send="true">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="x_mimeAttachmentHeader"></fieldset>
<pre class="x_moz-quote-pre">_______________________________________________
gdal-dev mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org" moz-do-not-send="true">gdal-dev@lists.osgeo.org</a>
<a class="x_moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="x_moz-signature" cols="72">--
<a class="x_moz-txt-link-freetext" href="http://www.spatialys.com" moz-do-not-send="true">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>