[Gdal-dev] Re: gdal fails on vrt (virtual raster) file

[Even Rouault]

Wolfgang,

I'm not sure at all, but I have maybe an idea of what is going wrong.
I ran recently into a similar problem with VRT made of several thousands of 
files, since when we open the VRT, we open each single dataset. This leads to 
a number of opened file descriptors > 1024 which is the usual limit for a 
Linux userland process. This has nothing to do with available RAM. The reason 
for it to work on Windows is that Windows has maybe a greater limit for 
opened files.
Does "gdalinfo /sonstiges/GIS/Frankreich/Frankreich250k/20/39_20.jpg" work ? 
If it does, it may confirm my theory.

I've recently posted a patch in GDAL trac 
(http://trac.osgeo.org/gdal/ticket/1781) that fixed the issue on my big VRTs 
but it's probably too experimental/nasty to be applied in its current state.

If you have the opportunity to build GDAL from subversion trunk, you could 
apply it and test if it improves the situation for you too.

Best regards.

On Thursday 06 September 2007 19:25:53 WolfgangZ wrote:
>
> Hi,
>
> finally I did some further tests on my data:
>
> One of the datasets worked directly on linux (FWTools 1.3.5) but a
> bigger one also fails with a different error (it takes some time to fill
> the swap file and therefore it take about 20 seconds to get the error):
>   ./gdal_translate -co COMPRESS=JPEG -co TILED=YES -a_srs EPSG:27582
> /sonstiges/GIS/Frankreich/Frankreich250k/referenz.vrt
> /sonstiges/GIS/Frankreich/Frankreich250k/Frankreich250k.tif
> ERROR 4:
> VSIFOpenL(/sonstiges/GIS/Frankreich/Frankreich250k/20/39_20.jpg) failed
> unexpectedly in jpgdataset.cpp
>
> after converting the tiles with gdal_translate -of JPEG in* out* gdal
> fails immediately (less than a second):
> GDALOpen failed - 4
> VSIFOpenL(/sonstiges/GIS/Frankreich/Frankreich250k/20/39_20a.jpg) failed
> unexpectedly in jpgdataset.cpp
>
> The strange thing is that exactly the same data set works well on
> windows!!! Memory usage is around 200MB.
> It suspect that on linux JPEG sources are generally importet completely
> into RAM, but I'm not a programmer just a user of gdal.
>
> (gdalinfo also fails on linux but works on win, filesize is 48600x46720)
>
> I hope these findings help to improve the (already great) software!
>
> Regards
> Wolfgang
>
> _______________________________________________
> Gdal-dev mailing list
> Gdal-dev at lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/gdal-dev