[GRASS-dev] t.list slow

[Sören Gebbert]

Hi Radim,


2015-10-18 14:08 GMT+02:00 Radim Blazek <radim.blazek at gmail.com>:
> On Sun, Oct 18, 2015 at 1:42 PM, Sören Gebbert
> <soerengebbert at googlemail.com> wrote:
>> Hi Radim,
>> the reason why t.list and many other t.* modules are "slow" is the
>> initialization of the temporal framework. Several background processes
>> are started and the temporal databases are checked for each mapset in
>> the mapset searchpath.
>
> The time is almost constant regardless how many mapsets are in
> SEARCH_PATH and still about 500ms if SEARCH_PATH is empty so it must
> be something else.

My Python profiler shows me that the most time is spend by calling the
C-library access subprocess, some GRASS modules and in the temporal
database check.

>
>> You have to use the GRASS GIS temporal framework directly to make it
>> faster, so that the temporal framework is only initiated once and not
>> every time a t.* module is called.
>
> I would prefer to use it directly but I need a C API. Could be some
> basic functions like get_dataset_list() ported to C library? If not,
> is it some trick, some environment variable or something like that to
> avoid unnecessary initialization, which could make it faster?

Unfortunately not. The initialization is important and cant be
skipped, since a lot of functionality depends on it.
Almost 95% of the GRASS GIS temporal framework is implemented in
Python, hence all of the functionality.
There is no C/C++ API available to access the Python implementation.

A work around would be to implement a Python script that calls all the
Python API parts to generate the desired text output, so substitute
several t.list and t.rast.list calls.

Best regards
Soeren

> Radim
>
>> I have implemented such a functionality here:
>>  https://bitbucket.org/huhabla/grass-data-explorer/src/03e2fc5b74da163a2da5bfedca664afb542c4077/grassdataexplorerdialog.py?at=master&fileviewer=file-view-default#grassdataexplorerdialog.py-739
>>
>> The function from the temporal framework is: tgis.get_dataset_list()
>> This should be almost as fast as the direct sqlite interface.
>>
>> Best regards
>> Soeren
>>
>> 2015-10-18 13:15 GMT+02:00 Radim Blazek <radim.blazek at gmail.com>:
>>> Hi,
>>> I found t.list module to be quite slow, about 500ms. Simple "select *
>>> from strds_absolute_time;" takes about 8ms with sqlite3 or db.select.
>>> If I want to get list of more types for more mapsets (for example
>>> mapsets which are not in SEARCH_PATH) it takes seconds.
>>>
>>> Is there a chance to make it faster?
>>>
>>> Radim