[GRASS-dev] [GRASS GIS] #198: v.in.ascii: column scanning is borked

[GRASS GIS]

#198: v.in.ascii: column scanning is borked
------------------------+---------------------------------------------------
 Reporter:  hamish      |       Owner:  grass-dev at lists.osgeo.org
     Type:  defect      |      Status:  new                      
 Priority:  critical    |   Milestone:  6.4.0                    
Component:  Vector      |     Version:  svn-develbranch6         
 Keywords:  v.in.ascii  |    Platform:  All                      
      Cpu:  All         |  
------------------------+---------------------------------------------------
 Hi,

 this bug is related to the old RT bugs 2763 and 5209.
   http://intevation.de/rt/webrt?serial_num=2763
   http://intevation.de/rt/webrt?serial_num=5209

 and the clumsy empty last-column work-around in v.in.gpsbabel:
 http://trac.osgeo.org/grass/browser/grass/trunk/scripts/v.in.gpsbabel/v.in.gpsbabel#L298
  "FIXME: if last field (comments) is empty it causes a not-enough fields
 error in v.in.ascii"




 The column type scanning step in v.in.ascii's points mode no longer
 accepts empty columns as NULL, and imported tables have columns truncated.
 Note that passing empty values in double columns works in GRASS 6.2.3!

 It would be nice to allow numeric columns as empty or 'NULL' for an empty
 record, and allow "nan" or "inf" without the scanning function deciding
 that the column contains strings.
 (For varchar columns the word 'NULL' should not be stripped however)

 Input file:
 {{{
 cat << EOF > test.dat
 cat|x|y|name|value|count
 1|2.3|4.5|Foo|3.1415|4
 2|2.4|4.6|Bar|||
 EOF
 }}}


 Import without column declaration:
 {{{
 G64svn> v.in.ascii in=test.dat out=test_null_import skip=1 \
            cat=1 x=2 y=3 --verbose

 Scanning input for column types...
 Maximum input row length: 25
 Maximum number of columns: 6
 Minimum number of columns: 6
 Column: 1 type: integer
 Column: 2 type: double
 Column: 3 type: double
 Column: 4 type: string length: 3
 Column: 5 type: string length: 0
 Column: 6 type: string length: 0
 Importing points...
 Populating table...
 Building topology for vector map <test_null_import>...
 2 primitives registered
 Building areas:  100%
 0 areas built
 0 isles built
 Attaching islands:
 Attaching centroids:  100%
 Topology was built
 Number of nodes     :   2
 Number of primitives:   2
 Number of points    :   2
 Number of lines     :   0
 Number of boundaries:   0
 Number of centroids :   0
 Number of areas     :   0
 Number of isles     :   0
 v.in.ascii complete.

 G64svn> v.info -c test_null_import
 Displaying column types/names for database connection of layer 1:
 INTEGER|int_1
 DOUBLE PRECISION|dbl_1
 DOUBLE PRECISION|dbl_2
 CHARACTER|str_1
 }}}

  * what happened to columns 5 and 6?

 {{{
  Column: 5 type: string length: 0
  Column: 6 type: string length: 0
 }}}
  * Columns 5 and 6 incorrectly scanned as (empty) "string" type.

 Also, I am not sure if hiding the column scanning result behind --verbose
 mode is advisable, given that it is buggy and it is the first line of
 defense when the input file contains typos.


 Import with column declaration:
 {{{
 G64svn> v.in.ascii in=test.dat out=test_null_import skip=1 \
           cat=1 x=2 y=3 --verbose \
           columns='cat int, x double, y double, name varchar(10), value
 double, count int'


 Scanning input for column types...
 Maximum input row length: 25
 Maximum number of columns: 6
 Minimum number of columns: 6
 Column: 1 type: integer
 Column: 2 type: double
 Column: 3 type: double
 Column: 4 type: string length: 3
 Column: 5 type: string length: 0
 Column: 6 type: string length: 0
 WARNING: Table <test_null_import> linked to vector map <test_null_import>
          does not exist
 ERROR: Column number 5 defined as double has string values
 }}}

  * in addition to previous errors the "table does not exist" warning's
 meaning is a mystery.


 changing the empty {{{"||"}}} to "|NULL|" doesn't help, the scanning step
 declares it as a string column (length: 4) and refuses to continue.


 this is important code, so tread with greatest care.....


 Hamish

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/198>
GRASS GIS <http://grass.osgeo.org>