[gdal-dev] With 1.6.2,still can't read ArcInfo binary grids: GDAL Error 3:Attempt to read past EOF inC:\Temp\GeoEcoTemp_Jason\tmpopx_s3\projected0a/../info/arc.dir.

[Roger Bivand]

On Sat, 8 Aug 2009, Even Rouault wrote:

>
> ----- Original Message ----- From: Jason Roberts
> To: gdal-dev at lists.osgeo.org
> Cc: 'Roger Bivand'
> Sent: Saturday, August 08, 2009 12:05 AM
> Subject: [gdal-dev] With 1.6.2,still can't read ArcInfo binary grids: GDAL 
> Error 3:Attempt to read past EOF 
> inC:\Temp\GeoEcoTemp_Jason\tmpopx_s3\projected0a/../info/arc.dir.
>
>
>> Hi all,
>
>> I am using rgdal to read ArcInfo binary grids produced by ArcGIS. The 
>> latest public version of rgdal, which binds with GDAL 1.6.1, fails with the
>> error message in the subject line of this email. I reported this a couple 
>> of weeks ago in 
>> http://lists.osgeo.org/pipermail/gdal-dev/2009-July/021369.html.
>
>> Today, the rgdal developer (Roger Bivand) provided me with an rgdal built 
>> against the 1.6.2 release candidate issued 31 July. The problem still 
>> exists.
>
> OK, I've looked more closely at 
> http://rgdal.cvs.sourceforge.net/viewvc/rgdal/rgdal/src/gdal-bindings.cpp?revision=1.32&view=markup 
> and I see RGDAL defines its own error handler for GDAL errors (around lines 
> 114-134). So the fix pushed in 1.6.2 that solves the issue for Python and 
> other bindings languages will not help for the RGDAL case as the error() call 
> probably causes a R exception to be thrown. As I've no knowledge of R, it is 
> just wild guessing.

Even, Jason,

Thanks, this looks helpful. I'll try to see how it fits into the error 
handler put in place from the R side. An R error() unwinds all the way up 
the call trace in C/C++ through to the R code; there is a warning() that 
could be used for non-fatal GDAL errors (though it isn't obvious which 
ones shouldn't be fatal, and since we're in a running R session, we cannot 
really allow anything not safe to persist (like a handle to a dataset not 
opened properly).

Jason, please send me off-list an example offending file (or link if big) 
so that I'll know when I'm making progress.

Roger

>
> RGDAL_OpenDataset() could be modified so that its own error handler is not 
> used during GDALOpen. Something like this might do it :
>
> ...
> CPLErrorReset();
> CPLPushErrorHandler(CPLDefaultErrorHandler);
> GDALDataset *pDataset = (GDALDataset *) GDALOpen(fn, RWFlag);
> CPLPopErrorHandler();
> // Similarly to SWIG bindings, the following lines will cause 
> RGDAL_OpenDataset() to fail on - unclearred - errors even if pDataset is not 
> NULL.
> // They could also be just removed. While pDataset != NULL, there's some hope 
> ;-)
> CPLErr eclass = CPLGetLastErrorType();
> if (pDataset != NULL && eclass == CE_Failure)
> {
>   GDALClose(pDataset);
>   pDataset = NULL;
>   __errorHandler(eclass, CPLGetLastErrorNo(), CPLGetLastErrorMsg());
> }
>
> if (pDataset == NULL)
>   error("Could not open file: %s\n", filename);
> ....
>
> Note this is uncompiled & untested, but that should avoid similar situations 
> in other drivers (non fatal errors triggered during dataset opening and 
> cleared by CPLErrorReset() ) to occur .
>
>> The problem had apparently been seen before. My original email prompted a 
>> review of the issue, including mention of it being tracked in tickets
>> #2447 and #3031. Evan made some checkins on 22 July that I thought would 
>> result in this being fixed in both the trunk and the next release of 1.6
>> (which would be 1.6.2). But I guess I did not fully understand it, and I 
>> missed the fact that neither of these issues appeared in Howard's 31 July
>> release candidate email.
>
> The tickets are not mentionned in Howard's email because the fix is only 
> partial and the tickets still open.
>
>> So: is there any chance the GDAL team can fix this in either 1.6.2 or, if 
>> it is now too late, 1.6.3?
>
>> It may be that this fell off the radar because the severity of the tickets 
>> (for example, #3031 is minor), or because there was a suggested workaround
>> that users could simply rename the "info" directory to prevent GDAL from 
>> failing. Well the issue is indeed severe if you are an ArcGIS user. In my
>> case, the rasters were produced by ArcGIS 9.3.1. Presumably ESRI is 
>> generating the rasters correctly. The workaround of renaming the info
>> directory is not a good idea. That renders all of the rasters in a 
>> directory inaccessible to ArcGIS. It is a major corruption of the ArcInfo 
>> binary grid
>> format to do so. I'm not sure why it was suggested in the first place, but 
>> it is a very bad idea.
>
> Eh! workarounds are workarounds and generally not very elegant. It enables 
> GDAL to avoids reading the VAT table, but if ArcGIS doesn't like it, that's 
> another story... The advantage with renaming is that you can undo it when 
> necessary... Anyway, you're free of ignoring it if you don't like it.
>
>> I'm sorry for not staying on top of this issue and somehow missing that it 
>> was not put into 1.6.2 and that the workaround is such as bad idea. In any
>> case, could you please consider fixing this in 1.6 soon? I can provide 
>> example data if needed.
>
> #3031 has already a link to sample data that triggers the error message. You 
> just need to find and convince someone that has time and knowledge of the AVC 
> driver.
>
>> Thanks,
>
>> Jason
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev 
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no