[Gdal-dev] Format limitations in 64-bit architecture

[Simon Perkins]

Frank Warmerdam wrote:
> Skimming the MEM driver code, it does not appear to be careful about 
> having
> a very large backing image.  That is, offsets into it's memory image are
> computed using 32bit offsets.  If you want to have a very large MEM 
> dataset
> some improvements to the MEM driver will be required.
>
> I believe that NITF uses ascii encoded offsets for image chunks, and that
> these are wide enough to support files substantially larger than 4GB 
> though
> I haven't reviewed the limits.  GDAL does use the "large file API" for 
> access
> and I have had reports of people working with large NITF files.  But I 
> don't
> have much direct experience so there may be issues.
>
> Hmm, on review I see the NITFCollectSegmentInfo() keeps track of the
> offsets in a 32bit integer.  So now I am not at all confident that large
> files would necessarily work.  It would depend on the particulars of 
> the file.
> This is really something that should be fixed in the driver, but I'd 
> be more
> comfortable fixing it with a large example file to work with.

Thanks Frank for taking the time to actually look at the code... So, 
there's no reason in theory why MEM and NITF can't support files larger 
than 4GB (assuming a 64-bit OS in the case of MEM), but right now the 
drivers are not capable of doing that.

I would think that there's a particular need for the NITF support to be 
fixed up - I've seen several NITF files larger than 4GB - though I don't 
unfortunately have any to hand that I could lend you for debugging. 
Anyone out there have one they could lend to Frank? Or does someone have 
NITF creation software that could be used to make such a file?

Cheers,

Sy