[QGIS-Developer] only read name and group name of processing algorithms during startup

[Nyall Dawson]

On 19 March 2018 at 18:39, Rashad Kanavath <rashad.kanavath at c-s.fr> wrote:
> Nyall,
>
> could we have defineCharacteristicsFromFile method in
> QgsProcessingAlgorithm ?.
> the diff with canExecute is more of a working draft.


Hi Rashad,

Can you open a PR? I'm having trouble following the change as the
whitespace is messed up in the email.

Thanks!
Nyall

>
>
>
> On Thu, Mar 15, 2018 at 03:27:50PM +0100, Rashad Kanavath wrote:
>> Hello,
>>
>> On Wed, Mar 14, 2018 at 4:38 AM, Nyall Dawson <nyall.dawson at gmail.com>
>> wrote:
>>
>> > On 13 March 2018 at 20:56, Rashad Kanavath <rashad.kanavath at c-s.fr> wrote:
>> > > if a provier is enabled then qgis reads all algorithm's during
>> > > startup. This involves reading descriptor files and doing all parsing
>> > > required to make gui when user actually open algorithm (click on
>> > > treeitem).
>> > > So If I have like three provider enabled it takes more time to start
>> > qgis than
>> > > usual. Even though, I might open one or two later at some point.
>> > >
>> > > It is not necessary here to parse "all" data during startup.  name and
>> > > group name is only needed to fill provider-algorithm tree. This is
>> > > true for any provider which uses descriptor files (OTB, GRASS, SAGA
>> > etc..).
>> >
>> > Actually it's a bit trickier here -- models make things more complex
>> > as they self-validate on creation, and need full knowledge of the
>> > dependent algorithm's parameters and outputs.
>> >
>>
>> Yes. there are different cases like modeler, standard, batch, scripts.
>> I tried to utilize canExecute() in otb and seems to be working.
>> You already have a very flexible, strong API in qgis processing such
>> trickier feature like this one has become too easy!!.
>>
>> I had to call alg.canExecute() in two places and that's it.
>> diff attached below. With this loading time will be hopefully improved for
>> OTB, SAGA, GRASS GIS.
>>
>> index f5bdc40dc5..1cca57769a 100644
>> --- a/python/plugins/processing/modeler/ModelerParametersDialog.py
>> +++ b/python/plugins/processing/modeler/ModelerParametersDialog.py
>> @@ -70,6 +70,7 @@ class ModelerParametersDialog(QDialog):
>>          QDialog.__init__(self)
>>          self.setModal(True)
>>          # The algorithm to define in this dialog. It is an instance of
>> QgsProcessingModelAlgorithm
>> +        alg.canExecute()
>>          self._alg = alg
>>          # The model this algorithm is going to be added to
>>          self.model = model
>> @@ -80,6 +81,9 @@ class ModelerParametersDialog(QDialog):
>>          settings = QgsSettings()
>>          self.restoreGeometry(settings.value("/Processing/
>> modelParametersDialogGeometry", QByteArray()))
>>
>> +    def algorithm(self):
>> +        return self._alg
>> +
>>      def closeEvent(self, event):
>>          settings = QgsSettings()
>>          settings.setValue("/Processing/modelParametersDialogGeometry",
>> self.saveGeometry())
>> @@ -242,7 +246,6 @@ class ModelerParametersDialog(QDialog):
>>              outTypes = []
>>          elif not isinstance(outTypes, (tuple, list)):
>>              outTypes = [outTypes]
>> -
>>          return self.model.availableSourcesForChild(self.childId,
>> [p.typeName() for p in paramType if
>>
>>  issubclass(p, QgsProcessingParameterDefinition)],
>>                                                     [o.typeName() for o in
>> outTypes if
>> diff --git a/src/core/processing/models/qgsprocessingmodelchildalgorithm.cpp
>> b/src/core/processing/models/qgsprocessingmodelchildalgorithm.cpp
>> index f5f8074c88..01906a7f62 100644
>> --- a/src/core/processing/models/qgsprocessingmodelchildalgorithm.cpp
>> +++ b/src/core/processing/models/qgsprocessingmodelchildalgorithm.cpp
>> @@ -58,7 +58,7 @@ QgsProcessingModelChildAlgorithm &
>> QgsProcessingModelChildAlgorithm::operator=( c
>>
>>  const QgsProcessingAlgorithm *QgsProcessingModelChildAlgorithm::algorithm()
>> const
>>  {
>> -  return mAlgorithm.get();
>> +  return mAlgorithm && mAlgorithm->canExecute() ? mAlgorithm.get():
>> nullptr;
>>  }
>>
>>  void QgsProcessingModelChildAlgorithm::setModelOutputs( const
>> QMap<QString, QgsProcessingModelOutput> &modelOutputs )
>>
>> test code for otb provider:
>> https://gitlab.orfeo-toolbox.org/orfeotoolbox/qgis-otb-plugin
>> I can add modification to SAGA and GRASS if you are okay to move this to PR.
>>
>>
>> > > Issue is more about calling defineDescriptorFile() in Algorithm's
>> > > __init__ method. And ofcourse, one cannot avoid init when adding
>> > > algorithm and tree need to add an instance of algorithm. what can be
>> > > done?
>> >
>> > What I've been thinking/planning is to take advantage of task manager
>> > here. So basically:
>> >
>> > - on processing startup, there's no providers added
>> > - when the qgis interface is fully loaded then we fire up a background
>> > task which loads the providers, allowing them to fully build their
>> > available algorithms and parameters without blocking the startup
>> > - after the task completes, the tree is populated
>> >
>>
>> this is still loading all algorithm with parsing but things will be in
>> background to not block ui for users. correct?
>> In that case, you could include above diff and then add loading in
>> background task. IMHO that will be best.
>>
>>
>> > I'd like to see more plugins take this approach, so that qgis can load
>> > nice and quick and the slow stuff happens without annoying users...
>> >
>> > Nyall
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >
>>
>>
>>
>> --
>> Regards,
>>    Rashad
>
> --
> Take care,
> Rashad