[GRASSLIST:4245] Re: Cannot open driver pg - v.out.ogr

[David Orme]

Hi again,

Found a solution  - when I used v.out.ogr in a loop and wrote directly 
to the final destination directory then I got the "too many files" 
error after the folder got to around 350 files (approx. 80 shapefiles). 
If I send the output of v.out.ogr to /tmp and then move the files to 
the final directory, the whole list of files is processed. I don't 
understand why this should be so if anyone can enlighten me, that'd be 
great.

i.e. within a loop of 700 files:

v.out.ogr -c $fname dsn=./FinalDestDir layer=$fname type=area

fails after about 80 but...

v.out.ogr -c $fname dsn=/tmp layer=$fname type=area
mv /tmp/$fname* ./FinalDestDir

...works without hitch.

Cheers,
David


On 24 Aug 2004, at 11:00, David Orme wrote:

> Hi,
>
> I need to convert about 700 raster datasets into shapefiles -  I've 
> written a shell script that does this nicely, except that after 
> processing about 80 of the files through to shapefile I get the 
> following error. The script keeps running, it just fails to write out 
> the shapefiles from then on:
>
>> Converting Zosterops_lateralis_7454
>> CREATING SUPPORT FILES FOR Zosterops_lateralis_7454
>> 25 primitives registered
>> 13 areas built      %
>> 5 isles built
>> Number of nodes     :   17
>> Number of primitives:   25
>> Number of points    :   0
>> Number of lines     :   0
>> Number of boundaries:   18
>> Number of centroids :   7
>> Number of areas     :   13
>> Number of isles     :   5
>> Number of areas without centroid :   6
>> WARNING: The table <Zosterops_lateralis_7454> is now part of vector 
>> map
>>          <Zosterops_lateralis_7454> and may be deleted or overwritten 
>> by GRASS modules.
>> ERROR 4: Failed to open Shapefile 
>> `./Shapefiles/Zosterops_lateralis_7454.shp'.
>>
>> Too many open files: /usr/local/grass57/driver/db/
>> ERROR: Cannot open driver pg
>> REMOVE [Zosterops_lateralis_7454]
>>  raster
>>  header
>>  category
>>  color    MISSING
>>  history
>>  misc
>>  fcell    MISSING
>>  g3dcell  MISSING
>
> I've appended the shell script - there is a separate wrapper script 
> which just contains a 'while read' loop to pass in the variables that 
> I need to be added in to the attribute table and also a separate SQL 
> template that defines the attribute table structure and then updates 
> the entries using the provided variables. I'm using a home build of 
> GRASS 5.7 (grass57_exp_2004_01_10) on Mac OS X 10.3.5 and using 
> PostgreSQL 7.4.1.
>
> I can just restart GRASS and run batches of 80 or so but it would be 
> great to know where I'm going wrong - I obviously need to release the 
> files in some way.
>
> Thanks,
> David
>
> #!/bin/bash
>
> # variables are: infile, smname, smnumber, presence, origin, source 
> tag, sourcename, data_type
> # generalize with pg_db, out_dir?
>
> # requires a PostgreSQL dtabase called tmpgrass
>
> if /usr/local/pgsql/bin/psql -l | grep -q "tmpgrass"
> then
>
> 	# get the file basename for export
> 	fname=`basename $1`
> 	
> 	echo Converting $fname
>
> 	v.database driver=pg database=tmpgrass
> 	
> 	#check for output folder
> 	if ! ls | grep -q Shapefiles
> 	then
> 		mkdir ./Shapefiles
> 	fi
> 	
> 	r.in.arc $1 output=$fname type=CELL
> 			
> 	r.to.vect $fname output=$fname feature=area
> 		
> 	# get correct attribute table:
> 	# substitute species name into the sql commands into a temporary file
> 	tmpname=tmp.asc2grd.$$.sql
> 	sed  -e "s/SMNAME/$2/g" -e "s/SMNUM/$3/g"  -e "s/FNAME/$fname/g" -e 
> "s/PRCODE/$4/g" -e "s/ORCODE/$5/g" -e "s/SRCTAG/$6/g" -e 
> "s/SRCNAME/$7/g" -e "s/DATATYPE/$8/g" shpSqlTemplate.sql > $tmpname
> 	
> 	# and then execute the sql file
> 	db.execute driver=pg database=tmpgrass input=$tmpname
> 	
> 	# reconnect the vector data to the modified table
> 	v.db.connect -o map=$fname driver=pg database=tmpgrass table=$fname 
> key=id
> 	
> 	# dump the shapefile version
> 	v.out.ogr -c $fname dsn=./Shapefiles layer=$fname type=area
>
> 	# delete the temporary raster and sql file
> 	g.remove rast=$fname vect=$fname
> 	rm $tmpname
> 	
> else
> 	echo "PostgreSQL database tmpgrass does not exist:"
> 	echo "	- create using createdb tmpgrass"
> fi
>
>
>
>
>