[gdal-dev] Can't use two datasets read-only from same files?

[Nik Sands]

Hi Even,

Thanks for your replies.  I think the number of files may be the problem.  Note that this is on the iPhone simulator (so effectively Mac OS X).

Yes, the "path" is a directory that contains shapefiles representing 352 layers.  Each layer has a .shp, .shx, .dbf, .prj, so 1409 files in total.  I'm not sure how many of these OGR needs to have open simultaneously.

Anyhow, I appreciate your reply.  I think I've hit the OS file limit.  I didn't even realise how many files were in that directory until you mentioned this possibility.

Cheers,
Nik.


On 23 Feb 2014, at 6:51 pm, Even Rouault <even.rouault at mines-paris.org> wrote:

> Le dimanche 23 février 2014 08:40:11, Even Rouault a écrit :
>> Nik,
>> 
>> what you try to do is perfectly valid and should work. I've just tried it
>> myself, but I can't reproduce any error. If you are working on Linux, could
>> you run your program with valgrind and look if it reports some errors ?
>> The only explanation I can imagine for now is that the file system on which
>> your shapefile is located doesn't support a file to be opened more than
>> once at a time, but this would be extremely surprising !
> 
> oh wait, is the "path" in your example a directory that contains lots of 
> shapefiles ? In that case, you might reach the maximum number of opened files by 
> a process. But... since GDAL 1.9, there is a mechanism in the shapefile driver 
> to limit the number of layers opened simultaneously to 100 per datasource (so 
> 200 files for .shp and .dbf). With 2 datasources, this would be 400 files which 
> is still below the 1024 limit on Linux.
> 
>> 
>> Best regards,
>> 
>> Even
>> 
>>> Hi GDAL devs,
>>> 
>>> I have a need to open the same set of Shapefiles as two independent
>>> datasets, BOTH of which are READ ONLY.  However, when I do this, and
>>> attempt to get the number of layers in the second data set, I get a whole
>>> 
>>> lot of errors of the form:
>>> 	ERROR 4: Failed to open file <shapefile>.shp.
>>> 	It may be corrupt or read-only file accessed in update mode.
>>> 
>>> ...for most (but not all) of the layers, and of course the dataset is
>>> incomplete and not useful.
>>> 
>>> I've crafted the following very simple example of this problem:
>>> 		 OGRDataSourceH firstDS = OGROpen(path, FALSE, NULL);
>>> 		 OGRDataSourceH secondDS = OGROpen(path, FALSE, NULL);
>>> 		 OGR_DS_GetLayerCount(firstDS);   //  THIS LINE WORKS FINE
>>> 		 OGR_DS_GetLayerCount(secondDS);  //  THIS LINE PRODUCES MANY ERRORS
>>> 
>>> If I OGR_DS_Destroy the first data set before running the
>>> OGR_DS_GetLayerCount on the second data set it works fine.
>>> 
>>> Is this expected?
>>> 
>>> I really need to have both datasets readable and useable simultaneously
>>> as they are both in completely independent parts of the application. 
>>> The first data set should be long-lived and needs to be persistent
>>> before, during and after the entire short life-cycle of the second data
>>> set.
>>> 
>>> The error messages imply that it is either corrupt or accessed in update
>>> mode.  But clearly it is not corrupt (at the file level) because it works
>>> fine in the first data set (and in the second if the first data set is
>>> destroyed first).  It is also being explicitly opened in READ-ONLY mode
>>> in both cases and therefore should not be in update mode.
>>> 
>>> I'm using gdal-1.10.1.
>>> 
>>> Any advice would be very much appreciated.
>>> 
>>> Cheers,
>>> Nik.
>>> 
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
> -- 
> Geospatial professional services
> http://even.rouault.free.fr/services.html
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev