[GRASS-dev] extending v.in.ascii capacity to scan large number of columns (code change provided)

[Glynn Clements]

Yann Chemin wrote:

> 76 buflen = 400000; #Changed from 4000
> 77 buf = (char *)G_malloc(buflen);
> 78 buf_raw = (char *)G_malloc(buflen);
> 79 coorbuf = (char *)G_malloc(16*256); #Added 16*
> 80 tmp_token = (char *)G_malloc(16*256); #Added 16*
> 
> This change permitted me to read cleanly through an ascii file x,y,v1,v2...v4018
> and import each of the x,y,v combinations as 3D vector layers
> independently (without attribute table)

If you have a need for many columns, it would be better to change it
to support any number of columns rather than simply increasing the
hard-coded limit to a larger value.

For a moderately-large number of columns, you'd need a version of
G_getl() which realloc()s its buffer as needed.

To cope with very large numbers of columns, the pull parser (reading a
line at a time) should be replaced by a push parser (reading
characters and pushing completed tokens into a state machine), e.g.:

	for (;;) {
	    int c = fgetc(fp);
	    if (c == EOF) { /* end of file */ }
	    else if (c == '\n') { /* new record */ }
	    else if (c == separator) { /* new field */ }
	    else { /* append c to current field */ }
	}

-- 
Glynn Clements <glynn at gclements.plus.com>