[GRASS-dev] Re: [GRASS-stats] Re: [R-sig-Geo] writing shapefiles / DBF files when input data contains NA

[Dylan Beaudette]

On Thursday 09 October 2008, Roger Bivand wrote:
> On Tue, 7 Oct 2008, Dylan Beaudette wrote:
> > On Tue, Oct 7, 2008 at 6:26 PM, Hamish <hamish_b at yahoo.com> wrote:
> >> Dylan:
> >>> It looks like the limiting factor in this equation is the
> >>> code used in v.out.ogr.
>
> Following Dylan's posting on the GDAL list, and Frank's response, I
> suggest the following simple patch to vector/v.out.ogr/main.c (here 6.3):
>
> $ diff -u main.c_old main.c
> --- main.c_old  2008-10-09 10:54:19.000000000 +0200
> +++ main.c      2008-10-09 11:30:34.000000000 +0200
> @@ -625,7 +625,9 @@
>                          colsqltype = db_get_column_sqltype(Column);
>                          colctype = db_sqltype_to_Ctype ( colsqltype );
>                          G_debug (2, "  colctype = %d", colctype );
> -                       switch ( colctype ) {
> +/* RSB 081009 emit unset fields */
> +                        if (!db_test_value_isnull(Value)) {
> +                         switch ( colctype ) {
>                               case DB_C_TYPE_INT:
>                                  OGR_F_SetFieldInteger( Ogr_feature, j,
> db_get_va lue_int(Value) );
>                                  break;
> @@ -639,7 +641,8 @@
>                                  db_convert_column_value_to_string (Column,
> &dbst
> ring);
>                                  OGR_F_SetFieldString( Ogr_feature, j,
> db_get_str ing (&dbstring) );
>                                  break;
> -                       }
> +                         }
> +                        } /* RSB */
>                      }
>                  }
>              }
>
> In 6.4 this is after line 717. Essentially it just uses
> db_test_value_isnull() not to set values in OGR fields if the DB field
> value is NULL, and follows Frank's suggestion.


OK-- I am using GRASS 6.4 SVN. Since this is the current (stable) developer 
release it might be good to stick with it. I am using r33792 .

Before making the suggested changes, I have tried exporting some point data 
with v.out.ogr:

# known working GRASS file with categories
v.out.ogr -e -c in=a_temp dsn=v.out.ogr_example layer=a_temp type=point

Exporting 25 points/lines...
 100%
0 features written
WARNING: 25 features found without category skip
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This looks like a new problem (?)... I know that these points have categories, 
why is v.out.ogr ignoring them?


Do these lines have anything to do with this:

v.out.ogr:434 
Vect_cat_get(Cats, field, &cat);
	    if (cat < 0 && !donocat) {	/* Do not export not labeled */
		nocatskip++;
		continue;
	    }

.... NO!
There is some kind of logic-related bug:

# does not export any features, 
# complaining that they don't have categories
v.out.ogr -c in=xxx dsn=some_folder layer=new_shpfile

# exported expected features
v.out.ogr -c in=xxx dsn=new_shpfile


OK- posted here:
http://trac.osgeo.org/grass/ticket/327

The suggested fix appears to result in the creation of a DBF file (in the case 
of shapefile) with properly encoded NULL.


Attached is a patch that applies this change to the develbranch-64 tree.

Should I make a ticket for this patch?

Cheers,

Dylan

> This matches code near line 939 in OGRGRASSLayer::SetAttributes()
> gdal/ogr/ogr_frmts/grass/ogrgrasslayer.cpp in the vector plugin, which
> uses:
>
> if ( !db_test_value_isnull(value) )
>
> I'm sure the patch needs checking, but with changes in the R rgdal package
> to support vector null data correctly, it ought to improve the interface.
>
> Best wishes,
>
> Roger
>
> >> maybe a silly question, but is a 3rd party format even needed here?
> >>
> >> $ ogrinfo --formats | grep -i grass
> >>  -> "GRASS" (readonly)
> >>
> >> at least in the one direction.
> >
> > Excellent question. I had also wondered about this. It looks like
> > there is a new argument in readVECT():
> >
> > # read in directly with GDAL/OGR -- no intermediate file:
> > x <- readVECT6('xxx', plugin=TRUE)
> >
> > This is quite fast and depends on the GDAL-GRASS plugin... However,
> > NULL data in a GRASS table is not imported correctly-- character
> > fields are imported as '', and numeric fields as 0.
> >
> > So... Is the error in GDAL itself?
> >
> > I tried inspecting a vector from GRASS with NULL data in some of the
> > columns from the table, using ogrinfo -al
> > location/mapset/vector/xxx/head
> >
> > OGRFeature(1):23
> >  cat (Integer) = 24
> >  cat_ (Integer) = 24
> >  str1 (String) = (null)
> >  xyz (Real) = (null)
> >  abc (Integer) = (null)
> >  POINT (591583 4925280 0)
> >
> > ... which seems to correctly report the NULL values.
> >
> > This leads me to suspect that something in readOGR() and writeOGR are
> > at fault in handling of NULL values.
> >
> > Unfortunately looking at the rgdal source code wasn't very productive
> > (my fault).
> >
> > Cheers,
> >
> > Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
-------------- next part --------------
A non-text attachment was scrubbed...
Name: v.out.ogr-null_value_support.patch
Type: text/x-diff
Size: 1654 bytes
Desc: not available
Url : http://lists.osgeo.org/pipermail/grass-dev/attachments/20081009/7aa40022/v.out.ogr-null_value_support.bin