[Gdal-dev] GDALOpenInfo->fp == NULL for large files?

Nick Bower nick at nickbower.com
Mon Mar 12 21:08:18 EDT 2007


Hi - thanks Frank, but I did mention in my original post that the file
is just 85 megabytes big.  The platform is a HPC style Itanium cluster
running SuSe by the way.  Commenting out the fp = NULL in the block in
which VSIFOpenL happens makes things work, including the L1B driver's
use of VSIFOpen.  But the real issue would seem to be why is errno being
set by VSIFOpen?  I may be a Python coder, but what guarantees that
fp!=NULL && errno != (27, ENOENT, 79) simultaneously?  Convention
(whatever I'm doing, I seem to be breaking it)?  Perhaps the logic here
needs to be better.

            fp = VSIFOpen( pszFilename, "rb" );
            if( fp != NULL )
            {
                nHeaderBytes = (int) VSIFRead( pabyHeader, 1, 1024, fp
                );
                VSIRewind( fp );
            }
            else if( errno == 27 /* "File to large" */
                     || errno == ENOENT
                     || errno == 79 /* EOVERFLOW - value too large */ )
            {
                // Why are we in here if VSIFOpen works elsewhere???
                ....





On Mon, 12 Mar 2007 14:10:41 -0400, "Frank Warmerdam"
<warmerdam at pobox.com> said:
> Nick Bower wrote:
> > I'm trying to debug the L1B driver - it's failing because poOpenInfo->fp 
> > is NULL (starting a driver with this test is also in the driver 
> > tutorial, and a raw driver also):
> > 
> > GDALDataset *L1BDataset::Open( GDALOpenInfo * poOpenInfo )
> > 
> > {
> >    int i = 0;
> >    if( poOpenInfo->fp == NULL )
> >        return NULL; // Returns for no reason
> > etc...
> > 
> > So I tried to figure out why fp is NULL.  The reason may be below?  I'm 
> > not a C/C++ programmer to full understand this, so I can only get as far 
> > as suspecting the problem has something to do with VSIFOpen/VSIFOpenL 
> > conventions (let me know if I'm off).  With the latter, it is set to 
> > NULL after reading for some reason, thus meaning (as far as i can tell) 
> > the L1B, HDF4, raw etc drivers return in the first couple of lines of 
> > initialization.  The file itself is only 85 Meg, so is not actually that 
> > large.
> 
> Nick,
> 
> It is expected for poOpenInfo->fp to be NULL on some platforms when
> the file is larger than 2GB (or possibly 4GB).  I see the L1B driver
> is using the old technique of "adopting" the file pointer.  Formats
> wishing to support larger than 2GB should not do this and ought to be
> using the VSI*L API instead of the VSI* API.
> 
> Basically, the L1B driver is not "ready" for large files.
> 
> If you feel this should be rectified you could try updating the driver
> yourself, or filing a bug report.  Since we don't have L1B files larger
> than
> 2GB, it would likely be necessary to have to check any fixes made.
> 
> Best regards,
> -- 
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | President OSGeo,
> http://osgeo.org
> 
-- 
  Nick Bower
  nick at nickbower.com




More information about the Gdal-dev mailing list