[Gdal-dev] Format limitations in 64-bit architecture

[Frank Warmerdam]

Mateusz Loskot wrote:
> Mateusz Loskot wrote:
>> Simon Perkins wrote:
>>>> What about other formats such as NITF and MEM?  Because the data
>>>>  type size_t in a 32-bit system is defined as a 4-byte type, 
>>>> whereas in a 64-bit system it is defined as an 8-byte.   Does 
>>>> this mean the MEM format is not limited to 2 GB anymore once we 
>>>> work in the 64-bit system?
>>> That would seem like a reasonable assumption though I don't know 
>>> for sure. I would think that NITF should be able to handle larger 
>>> than 4GB files even on 32-bit OSes, on any modern filesystem.
>> Yes, I think it's correct too.
> 
> Unfortunately, this assumption above *may* be incorrect.
> Big Thanks to Norman Vine, who enlighten me about NITF format, and told
> that the size limit depends on size of pointer type used by C++ streams.
> The C++ standard specifies general requirements about type of file
> pointer used by streams, but the type size is not fixed there.
> 
> All constraints like type used to represent stream position
> (traits::pos_type) and offset (traits::off_type).
> and others, are determined by char_traits used during instantiation
> of particular stream type.
> Finally, position/offset in streams (not only file stream) is
> convertible to integral type with sign (sic).
> 
> According to the C++ standard, behavior of pos_type and off_type
> are implementation defined.
> 
> Assuming the C++ standard doesn't define 64 bit integers, maximum
> value of pos_type or off_type can be maximum value of int type,
> so these values are different on different platforms as well.

Folks,

For reference, GDAL "built in" formats like NITF use the VSI API for
file access, and in this case the VSI*L functions which use 64bit file
offsets on platforms where they are available (essentially all).  I am
not aware of any GDAL format drivers using the c++ stream type for io.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org