[GRASS-SVN] r61144 - sandbox/wenzeslaus/gunittest
svn_grass at osgeo.org
svn_grass at osgeo.org
Thu Jul 3 16:10:12 PDT 2014
Author: wenzeslaus
Date: 2014-07-03 16:10:12 -0700 (Thu, 03 Jul 2014)
New Revision: 61144
Modified:
sandbox/wenzeslaus/gunittest/testing.rst
Log:
gunittest: handling of differnet locations (theory) and few updates to current state
Modified: sandbox/wenzeslaus/gunittest/testing.rst
===================================================================
--- sandbox/wenzeslaus/gunittest/testing.rst 2014-07-03 22:48:19 UTC (rev 61143)
+++ sandbox/wenzeslaus/gunittest/testing.rst 2014-07-03 23:10:12 UTC (rev 61144)
@@ -13,9 +13,108 @@
the documentation of testing framework classes and scripts.
-Testing functions with doctest
-==============================
+Testing with gunittest
+======================
+The tests should be in files in a ``testsuite`` directory which is a subdirectory
+of the directory with tested files (module, package, library). Each testing file
+(test file) can have can have several testing classes (test cases).
+All test files names should have pattern ``test*.py``.
+
+GRASS GIS `gunittest` package and testing framework is similar to the standard
+Python ``unittest`` package, so the ways to build tests are very similar.
+
+ ::
+ import guinttest
+
+ class TestPython(gunittest.GrassTestCase):
+
+ def test_counting(self):
+ """Test that Python can count to two"""
+ self.assertEqual(1 + 1, 2)
+
+ if __name__ == '__main__':
+ gunittest.test()
+
+Each test file should be able to run by itself and accept certain set of command
+line parameters. This is ensured using `gunittest.test()`.
+
+To run (invoke) all tests in the source tree run::
+
+ python gunittest/main.py [gisdbase] location test_data_category
+
+All test files in all ``testsuite`` directories will be executed and
+a report will be created in a newly created ``testreport`` directory.
+You need to be in GRASS session to run the tests.
+
+The test_data_category parameter serves to filter tests accoring to data
+they can run successfully with. It is ignored for tests which does not have
+this specified.
+
+Each running test file gets its own mapset and current working directory
+but all run are in one location.
+
+.. warning:
+ The current location is ignored but you should run not invoke tests
+ in the location which is precious to you for the case that something fails.
+
+To run individual tests file you should be in GRASS session in GRASS NC sample
+location in a mapset of arbitrary name (except for the predefined mapsets).
+
+Your tests can rely on maps which are present in the GRASS NC sample location.
+But if you can provide tests which are independent on location it is better.
+
+Read the documentation of Python ``unittest`` package for a list of assert
+methods which you can use to test your results. For test of more complex
+GRASS-specific results refer to `GrassTestCase` class documentation.
+
+
+Tests of GRASS modules
+----------------------
+
+::
+
+ class TestRInfo(gunittest.GrassTestCase):
+
+ def test_elevation(self):
+ rinfo = Module('r.info', map='elevation', flags='g',
+ stdout_=subprocess.PIPE, run_=False)
+ self.assertModule(self.rinfo)
+ ...
+
+.. todo:
+ Add example of assertions of key-value results.
+
+.. todo:
+ Add example with module producing a map.
+
+::
+
+ class TestRInfoInputHandling(gunittest.GrassTestCase):
+
+ def test_rinfo_wrong_map(self):
+ map_name = 'does_not_exist'
+ rinfo = Module('r.info', map=, flags='g',
+ stdout_=subprocess.PIPE, run_=False)
+ self.assertModuleFail(rinfo)
+ self.assertTrue(rinfo.outputs.stderr)
+ self.assertIn(map_name, stderr)
+
+.. todo:
+ Create ``SimpleModule`` or ``TestModule`` class which will have the right
+ parameters for ``assertModule()`` and ``assertModuleFail()`` functions.
+
+
+Tests of C and C++ code
+-----------------------
+
+Tests of Python code
+--------------------
+
+
+Testing Python code with doctest
+--------------------------------
+
In Python, the easiest thing to test are functions which performs some computations
or string manipulations, i.e. they have sum numbers or strings on the input and
some others on the output.
@@ -48,51 +147,94 @@
the file is accessible through sys.path which is not true for case of GRASS modules.
Do not use use doctest for tests of edge cases, for tests which require
-generate complex data first, etc. In these cases use unittest.
+generate complex data first, etc. In these cases use `gunittest`.
-Testing functions with gunittest
-================================
+Data
+----
-GRASS GIS `gunittest` package and testing framework is similar to the standard
-Python ``unittest`` package, so the ways to build tests are very similar.
+Most of the tests requires some input data. However, it is good to write
+a test in the way that it is independent on the available data.
+In case of GRASS, we have we can have tests of functions where
+some numbers or strings are input and some numbers or string are output.
+These tests does not require any data to be provided since the numbers
+can be part of the test. Then we have another category of tests, typically
+tests of GRASS modules, which require some maps to be on the input
+and thus the output (and test) depends on the specific data.
+Again, it it best to have tests which does not require any special data
+or generally environment settings (e.g. geographic projection)
+but it is much easier to write good tests with a given set of data.
+So, an compromises must be made and tests of different types should be written.
- ::
+In the GRASS testing framework, each test file should be marked according to
+category it belongs to. Each category corresponds to GRASS location or locations
+where the test file can run successfully.
- from guinttest.case import GrassTestCase
+Universal tests
+ First category is *universal*. The tests in this category use some some
+ hard coded constants, generated data, random data, or their own imported
+ data as in input to function and GRASS modules. All the tests, input data
+ and reference results should be projection independent. These tests will
+ runs always regardless of available locations.
- class TestRUnivar(GrassTestCase):
+Standard names tests
+ Second category are tests using *standard names*. Tests rely on a
+ certain set of maps with particular names to be present in the location.
+ Moreover, the tests can rely also on the (semantic) meaning of the
+ names, i.e. raster map named elevation will always contain some kind of
+ digital elevation model of some area, so raster map elevation can be
+ used to compute aspect. In other words, these tests should be able to
+ (successfully) run in any location with a maps named in the same way as
+ in the standard testing location(s).
- def test_conting(self):
- """Test that Python can count"""
- self.assertEqual(1 + 1, 2)
+Standard data tests
+ Third category of tests rely on *standard data*. These tests expect that the
+ GRASS location they run in not only contains the maps with particular names
+ as in the *standard names* but the tests rely also on the data being the
+ same as in the standard testing location(s). However, the (geographic)
+ projection or data storage can be different. This is expected to be the
+ most common case but it is much better if the tests is one of the previous
+ categories (*universal* or *standard names*). If it is possible the
+ functions or modules with tests in this category should have also tests
+ which will fit into one of the previous categories, even though these
+ additional tests will not be as precise as the other tests.
-Note that GRASS-specific invoking of tests is not yet implemented
-but you should try different combinations of adding the following Python
-code to your test and running it from command line::
+Location specific tests
+ Finally, there are tests which requires certain concrete location. There
+ is (or will be) a set of standard testing locations each will have the same
+ data (maps) but the projections and data storage types will be different.
+ The suggested locations are: NC sample location in SPM projection,
+ NC in SPF, NC in LL, NC in XY, and perhaps NC in UTM, and NC in some
+ custom projection (in case of strange not-fitting projection, there is
+ a danger that the results of analyses can differer significantly).
+ Moreover, the set can be extened by GRASS locations which are using
+ different storage backends, e.g. PostGIS for vectors and PostgreSQL for
+ temporal database. Tests can specify one or preferably more of these
+ standard locations.
- if __name__ == '__main__':
- unittest.main()
+Specialized location tests
+ Additionally, an specialized location with a collection of strange,
+ incorrect, or generally extreme data will be provided. In theory, more
+ than one location like this can be created if the data cannot be
+ together in one location or if the location itself is somehow special,
+ e.g. because of projection.
-adding the directory or parent directory of your tests to ``PYTHONPATH``
-and using::
+Each category, or perhaps each location (will) have a set of external data
+available for import or other purposes. The standardization of this data
+is in question and thus this may be specific to each location or this
+can be a separate resource common to all tests using one of the standardized
+locations, or alternatively this data can be associated with the location
+with special data.
- python -m unittest test_runivar
+.. note::
+ The more general category you choose for your tests the more testing data
+ can applied to your tests and the more different circumstances can be tried
+ with your tests.
-or using automatic test discovery which might be sometimes tricky::
- python -m unittest discover some/path
-You should be in running GRASS session in GRASS NC sample location in a mapset
-of arbitrary name (except for the predefined mapsets).
-Your tests can rely on maps which are present in the GRASS NC sample location.
-But if you can provide tests which are independent on location it is better.
-Read the documentation of Python ``unittest`` package for a list of assert
-methods which you can use to test your results. For test of more complex
-GRASS-specific results refer to `GrassTestCase` class documentation.
-
.. note::
gunittest is under development but, so some things can change, however
this should not stop you from writing tests since the actual differences
@@ -112,7 +254,8 @@
Analyzing quality of source code
================================
-Besides testing, you can also use some tools to check the quality of your code.
+Besides testing, you can also use some tools to check the quality of your code
+according to various standards and occurrence of certain code patterns.
For C/C++ code use third party solution `Coverity Scan`_ where GRASS GIS
is registered as project number `1038`_. Also you can use `Cppcheck`_
@@ -150,4 +293,4 @@
.. _doctest: https://docs.python.org/2/library/doctest.html
.. _Coverity Scan: https://scan.coverity.com/
.. _1038: https://scan.coverity.com/projects/1038
-.. _Cppcheck:http://cppcheck.sourceforge.net/
+.. _Cppcheck: http://cppcheck.sourceforge.net/
More information about the grass-commit
mailing list