[GRASS-user] grass70 problem with gui

[Martin Landa]

Hi,

2013/12/12 Markus Metz <markus.metz.giswork at gmail.com>:
> OTOH, v.in.ascii format=point expects the same number of tokens in each line:
>
> 1|1||name
> 2|2|3|other_name
>
> must always result in 4 tokens. Here, double pipe works as expected
> only if delimiters are not merged.

right, we need to have native tokenize() subroutine for such cases. It
was not good idea to replace it directly with tokenize_strtok() even
as a test. Anyway my plan was to introduce a new G_tokenize3() or
G_tokenize_strtok() based on tokenize_strtok(). I believe that such
function would make sense on several places like in `r.in.xyz`.

>> BTW, `separator` parameter can be string with more than one character,
>> but native tokenize subroutine fails to tokenize such string with
>> multiple-character delimiter, eg. G_tokenize("firstSElast", "SE");
>> returns {'first', '', 'last'} and not {'first', 'last'} as someone
>> could expect.
>
> Someone else would expect that with "SE" as delimiter argument, the
> delimiter can be either "S" or "E".

right, as strtok() behaves.

> I would leave G_tokenize() as it is because it is used at too many
> different places which expect the current behaviour.

Agreed.

> It is not uncommon that ASCII input data need a bit of pre-processing
> (tr, sed), e.g. when fields have fixed lengths. Alternatively, add an

right, anyway I think it's still better to handle such issues on
module level. You can hardly ask Windows user to use `tr` or `sed`.
Being user friendly is not so bad idea.

> option (to respective modules, not to G_tokenize()) to expect fixed
> length fields defining the field lengths?

I still think that G_tokenize_strtok() would make sense to have. Then
`r.in.xyz` or `v.in.ascii` could have special flag which could handle
input files as I described (e.g. columns separated by two spaces).
What do you think?

Martin

-- 
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa