[GRASS-stats] readVECT6 problem fo importing null data

[Markus Neteler]

Dear all,

Roger notified me about this issue (I sometimes read here as time
permits). I made some tests using out NC OSGeo data set
(http://www.grassbook.org/data_menu3rd.php):

# copy the map into your MAPSET and check for NULL
g.copy vect=lakes,mylakes
v.db.select mylakes
v.db.select mylakes where="FTYPE IS NULL"

# display the lakes, show undefined FTYPE lakes in red
g.region swwake_10m
d.mon x0
d.erase
d.vect mylakes where="FTYPE NOT NULL" type=area col=blue
d.vect mylakes where="FTYPE IS NULL" type=area col=red
# This works pretty well.

# Export test:
v.out.ogr mylakes dsn=mylakes.shp type=area

# Looking at it in openoffice:
cat,N,11,0,AREA,N,24,15,PERIMETER,N,24,15,FULL_HYDRO,N,24,15,FULL_HYDR2,N,24,15,FTYPE,C,80,FCODE,N,11,0,NAME,C,80
8,5310.04,334.89,9,55660,LAKE/POND,39000,
33,21739.17,709.53,34,54836,LAKE/POND,39000,
63,469.77,105.84,64,0,,0,
84,10564.66,402.43,85,55595,ROCK/ISLAND,44100,
93,7655.72,344.71,94,53621,DAM/WEIR,34300,
94,1357.21,178.47,95,0,,0,
...

No problem at all: some empty FTYPE fields as desired.
The problem must be elsewhere (or I misunderstand the problem,
didn't read all messages).

Best wishes,
Markus




Roger Bivand wrote:
> 
> On Thu, 29 Nov 2007, Jarosaw Jasiewicz wrote:
> 
>>
>>>
>>>  I can reproduce the problem, which seems related to DBFIsAttributeNULL
>>> not
>>>  being handled adequately on either side (both are using the OGR
>>> library).
>>>  Could you please try to write the temporary shapefile directly using
>>>  v.out.ogr, and then try to read it with readOGR() and with readShape*()
>>>  with * as Points, Lines, or Poly depending on what getinfo.shape()
>>> says?
>>>  I'll try to debug using the DBF code in the foreign package to start
>>> with
>>>  - numeric fields should have value /* NULL numeric fields have value
>>>  "****************" */ from dbfopen.c, but do not seem to.
>>> 
>> I tried what you sugested. I can't probably help more, but I found that:
>>
>> - v.out.ogr produce dbf with 0 instead null;
>> - readOGR() reads dbf with 0 instead null;
>> - if grass dbf table null are treated as 0, but displayed as null;
>>
>> I use RdbiPgSQL to import data with null values and nad readVECT6 to
>> import 
>> vector and use  substitute () to merge them (if needed). it is a
>> temporary 
>> solution
> 
> Jarek,
> 
> Thanks - I am adding comments to the readOGR() and readVECT6() help pages. 
> OGR does not promise to handle missing values at all, because there are no 
> standards as such - so this affects GRASS vector handling generally.
> 
> Two questions if you have time:
> 
> 1) can you try to read from PostgreSQL directly using readOGR() - I guess 
> that your GDAL/OGR includes the PostgreSQL/PostGIS driver?
> 
> 2) can you see whether the missing values are visible correctly within 
> GRASS, maybe with db.select (the v.db.univar script assumes that there 
> are no missing values)? If they are seen, then the problem is in 
> v.out.ogr not checking.
> 
> I can see isNull fields in dbmi.h; they seem to be used in the fetch.c 
> files for mysql, odbc, ogr, postgres, and sqlite drivers in db/drivers/. I 
> can only see this retrieved in vector/v.univar/main - but Martin Landa 
> probably added this recently, and he is active. There are traces in 
> lib/db/dbmi_base/value.c, lib/db/dbmi_base/xdrvalue.c, and 
> lib/db/dbmi_client/select.c.
> 
> I think that Martin will be interested in this, so maybe it's worth 
> checking. I do not run PostgreSQL, so I'm afraid I can't check myself. If 
> you do, maybe post to grass-dev?
> 
> Roger
> 
> 
>>
>> Jarek
>>
> 
> -- 
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand at nhh.no
> 

-- 
View this message in context: http://www.nabble.com/-GRASS-stats--readVECT6-problem-fo-importing-null-data-tf4876840.html#a14159473
Sent from the Grass - Stats mailing list archive at Nabble.com.