[Qgis-developer] Tool for creating unit tests for SEXTANTE.

[Victor Olaya]

Tim

I am going to start writing tests with the idea of eventually having
them added to the QGIS test framework. There are some test that have
to be written the usual way (tests of inner classes and methods), but
this is just an idea to simplify how the tests for algorithms are
created. I guess that once we have a good set of algorithm tests, if
they pass, that wil be a good sign, since they cover many different
cases. Then I will start with the "normal" tests, and this can be
moved to the QGIS framework anytime.

I agree with Paolo that we could have something automated (Camilo
started writing code for that), but we would be leaving out many
algorithms if values are selected randomly or are harcoded for the
whole set of tests. For instance, if we are testing an algorithm to
create contour lines, the values of the contours, or the interval,
have to be selected accordingly to the value range in the raster
layer. Otherwise, the algorithm will fail. In most cases, by setting a
fixed layer and a fixed numerical value, cases like that are to be
found. In other words, using an analysis algorithm is not trivial and
requires knowledge, so using random values or a fixed one for all
algorithm is likely to not work in many cases.

I said it before, but tests are my priority for the hackfest, and hope
to work on that with other people who are interested on helping.

Cheers
Victor

2013/3/9 Tim Sutton <lists at linfiniti.com>:
> Hi
>
>
> On Thu, Mar 7, 2013 at 10:53 AM, Victor Olaya <volayaf at gmail.com> wrote:
>> Hi all,
>>
>> I have been thinking about how to ease the process of creating unit
>> tests for SEXTANTE algorithms. Ideally, there should be (at least) a
>> test for each algorithm. I do not fancy the idea of automating that
>> and just feeding whatever data to an algorithm and checking it
>> produced something, since that is not a good solution. Instead, I
>> would like to have meaningful tests, taking a sample dataset, running
>> an algorithm, and checking it produces a correct result.
>>
>> This is time-consuming, but I have created a small tool, so anyone,
>> even non-programmers, can use it and help us. Basically, you have to
>> run an algorithm using a test dataset that will be provided along with
>> SEXTANTE. Then you check that the resulting data is correct. If so,
>> you go to the SEXTANTE history, right click on that algorithm you have
>> just run, and select "create test". It will create the corresponding
>> python code that calls that algorithm with that same input values and
>> tests the output is exactly that one. That code can later be send
>> somehow, to be included in the testing suite.
>>
>> Before lauching this and tell people to start helping us, I would like
>> to hear your opinion on it, and if you have something to add, please
>> feel free to comment.
>>
>
> This is awesome stuff! It would be really create if these tests could
> be run as part of our standard test framework (e.g. invoked by make
> test), so that we could collect pass failure metrics accross different
> platforms. Is this already the case, or something you could add
> support for?
>
> Regards
>
> Tim
>
>> Thanks!
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> --
> Tim Sutton - QGIS Project Steering Committee Member (Release  Manager)
> ==============================================
> Please do not email me off-list with technical
> support questions. Using the lists will gain
> more exposure for your issues and the knowledge
> surrounding your issue will be shared with all.
>
> Visit http://linfiniti.com to find out about:
>  * QGIS programming and support services
>  * Mapserver and PostGIS based hosting plans
>  * FOSS Consulting Services
> Skype: timlinux
> Irc: timlinux on #qgis at freenode.net
> ==============================================