[gdal-dev] GDALOpen crash on large GRIB files

[Sjur Kolberg]

Hello, list:

I have problems reading some large GRIB files from ECMWF. I have several files, 29MB and 50MB works fine, 141 MB and above crashes. The crash is within the first GDALOpen() function, and I track it through GRIBDataset::Open() to GDALDataset::Setband(). The error arises in the last statement given below, and because papoBands==NULL. By the two breakpoints I can tell that papoBands is not NULL when entering the SetBand() function, so it seems to be the VSIRealloc function that returns NULL. The error doesn't occur for the same band each time I try, but typically runs to band number somewhere around 50000. (bandno is a combined index of forecast reference time, forecast lead time and variable).

I run GDAL 1.6.3 on Windows XP, compiled with Visual Studio 2008. The error appears both in debug and release version. The application uses about 144 MB when it crashes, but has been up to a peak memory usage of 283 MB before the crash. I have 3.5 GB memory on the PC, of which 2.5 appears to be available. Both GDAL and the application is compiled with extended memory sizes and /LARGEADDRESSAWARE, but I am not sure about how this works. The error message is:

Unhandled exception at 0x30c36acd (gdal16d-vc9.dll) in RegModel.exe: 0xC0000005: Access violation writing location 0x00036c74.

Has anyone had similar problems?
Best regards,
Sjur K :-)



void GDALDataset::SetBand( int nNewBand, GDALRasterBand * poBand )

{

/* -------------------------------------------------------------------- */

/* Do we need to grow the bands list? */

/* -------------------------------------------------------------------- */

if( nBands < nNewBand || papoBands == NULL ) {

    int i;

    if( papoBands == NULL )

        papoBands = (GDALRasterBand **)  /* Breakpoint before this statement */

            VSICalloc(sizeof(GDALRasterBand*), MAX(nNewBand,nBands));

    else

    papoBands = (GDALRasterBand **)

        VSIRealloc(papoBands, sizeof(GDALRasterBand*) *

            MAX(nNewBand,nBands));

    for( i = nBands; i < nNewBand; i++ )   /* Breakpoint before this statement; condition: papoBands=0 */

        papoBands[i] = NULL;    /* Offending statement, papoBands==NULL.



State of poBand at error:

-  poBand 0x357f4228 {start=172766438 subgNum=0 longFstLevel=0x746a3e28 "0[-] SFC (Ground or water surface)" ...} GDALRasterBand *
+  [GRIBRasterBand] {start=172766438 subgNum=0 longFstLevel=0x746a3e28 "0[-] SFC (Ground or water surface)" ...} GRIBRasterBand
+  GDALMajorObject {nFlags=49 sDescription={...} oMDMD={...} } GDALMajorObject
+  poDS 0x3424f4b0 {fp=0x3424fb08 pszProjection=0x35b3ce68 "GEOGCS["Coordinate System imported from GRIB file",DATUM["unknown",SPHEROID["Sphere",6367470,0]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]" pszDescription=0xcdcdcdcd <Bad Ptr> ...} GDALDataset *
  nBand 56094 int
  nRasterXSize 0 int
  nRasterYSize 0 int
  eDataType GDT_Float64 GDALDataType
  eAccess GA_ReadOnly GDALAccess
  nBlockXSize 45 int
  nBlockYSize 1 int
  nBlocksPerRow 0 int
  nBlocksPerColumn 0 int
  bSubBlockingActive 0 int
  nSubBlocksPerRow 0 int
  nSubBlocksPerColumn 0 int
+  papoBlocks 0x00000000 GDALRasterBlock * *
  nBlockReads 0 int
  bForceCachedIO 0 int
+  poMask 0x00000000 {poDS=??? nBand=??? nRasterXSize=??? ...} GDALRasterBand *
  bOwnMask false bool
  nMaskFlags 0 int

Status of this GDALDataset:

  i -858993460 int
  nBands 49093 int
  nNewBand 49094 int
+  papoBands 0x00000000 GDALRasterBand * *
-  this 0x3424f4b0 {fp=0x3424fab8 pszProjection=0x35b31048 "GEOGCS["Coordinate System imported from GRIB file",DATUM["unknown",SPHEROID["Sphere",6367470,0]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]" pszDescription=0xcdcdcdcd <Bad Ptr> ...} GDALDataset * const
+  [GRIBDataset] {fp=0x3424fab8 pszProjection=0x35b31048 "GEOGCS["Coordinate System imported from GRIB file",DATUM["unknown",SPHEROID["Sphere",6367470,0]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]" pszDescription=0xcdcdcdcd <Bad Ptr> ...} GRIBDataset
+  GDALMajorObject {nFlags=33 sDescription={...} oMDMD={...} } GDALMajorObject
+  poDriver 0x00000000 {pfnOpen=??? pfnCreate=??? pfnDelete=??? ...} GDALDriver *
  eAccess GA_ReadOnly GDALAccess
  nRasterXSize 45 int
  nRasterYSize 33 int
  nBands 49093 int
+  papoBands 0x00000000 GDALRasterBand * *
  bForceCachedIO 0 int
  nRefCount 1 int
  bShared 0 int
+  oOvManager {poDS=0x00000000 poODS=0x00000000 osOvrFilename={...} ...} GDALDefaultOverviews






-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20091210/ca9262e3/attachment.html