[GRASS-dev] [GRASS-user] r.to.vect stats

[Moritz Lennert]

On 02/05/17 15:53, Vaclav Petras wrote:
>
> On Tue, May 2, 2017 at 3:24 AM, Moritz Lennert
> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>> wrote:
>
>         But I'm not against a wrapper. I added a prototype with limited
>         functionality to addons [1]. However, I'm not sure how to
>         account for
>         large data - that's what discouraged me from creating a wrapper. The
>         Python subprocess documentation says use communicate() but its
>         doc says
>         "The data read is buffered in memory, so do not use this method
>         if the
>         data size is large or unlimited." [2] Do I have to do the buffering
>         myself then or is it just fine? Is there some code like this in
>         GRASS?
>
>
>     Wouldn't using pipe_command() and feed_command() in the grass.script
>     be a solution ?
>
>
>
> I'm using pipe_command() which is just convenience function setting
> stdout=PIPE. Similarly feed_command() is just setting stdin=PIPE which
> I'm not using because I'm feeding the stdout of the other process
> directly (stdin=first_process.stdout). What I don't understand,
> regardless of using stdin=PIPE or stdin=first_process.stdout for the
> second process, is what should be next.

Do you really need the in_process.communicate() ? Here's what I used in 
a local script and it works, without communicate(). Then again, I don't 
think the data flowing through this pipe ever exceeded available memory.


         pin = gscript.pipe_command('v.db.select',
                                    map = firms_map,
                                    column="x,y,%s" % turnover_column,
                                    where="substr(%s, 1, 2) = '%s' AND 
%s >= 0" % (nace_column, nace2, turnover_column),
                                    flags='c',
                                    quiet=True)
         total_turnover_map = 'turnover_%s' % nace2
         p = gscript.start_command('r.in.xyz',
                                   input_='-',
                                   stdin=pin.stdout,
                                   method='sum',
                                   type_='DCELL',
                                   output=total_turnover_map,
                                   quiet=True,
                                   overwrite=True)
         if p.wait() is not 0:
             gscript.fatal("Error in r.in.xyz with nace %s" % nace2)


Moritz