<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Kurt,<br>
<br>
During code sprint in Korea (FOSS4G 2015) I plan to play with new
approach of CMake fro GDAL. The one of experiments will be try to
use CTest. As I plan restructure the sources tree, I can try to
integrate you work on autotest2 and CTest. Also we can try to create
new test directory structure more compatible for test and sources
tree (this is about you wrote: Probably should move python code to
also match the C++ tree. e.g. tiff_read_test.py ->
autotest2/py/frmts/gtiff/tiff_read_test.py).<br>
How can get you tests? What do I need to use autotest2?<br>
<br>
<pre class="moz-signature" cols="72">Best regards,
Dmitry</pre>
<div class="moz-cite-prefix">05.09.2015 20:37, Kurt Schwehr пишет:<br>
</div>
<blockquote
cite="mid:CACmBxytfvJhpppzGeuhcZukQVgBCetAAO_WfVGLSMLepE5JDAA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>(Subject change to focus on testing)<br>
</div>
<div><br>
</div>
Hi all,
<div><br>
</div>
<div>First off... what GDAL has with autotest, travis-ci and
coverity is awesome!</div>
<div><br>
</div>
<div>
<div>Thoughts / discussion more than welcome!</div>
</div>
<div><br>
</div>
<div>For my production work, I'm not able to use the autotest
python code because of its non-unittest architecture. So... I
started creating python unittest and C++ gunit based tests. I
use autotest2 in Google's internal continuous integration
system in our main code base. I'm using Google's build
system... I've got nothing started for running the C++ tests
outside of Google.<br>
</div>
<div><br>
</div>
<div>Apologies for not even getting out at least samples of
autotest2 for folks to inspect and comment on. My intention
is to put what I have in a git repo and the to start
discussions as to what (if anything) GDAL community wants to
do with autotest2. I was hoping to get a lot more coverage
and get GDAL 2.x.x support, but that will have to come later.
It's only 14K lines at this point (optimistically 2-3% done),
but it has been a huge help for me especially with in
upgrading versions of gdal and catching bugs in support libs
& development toolchains. </div>
<div><br>
</div>
<div>The tests are more focused on test isolation than
autotest. This allows for a lot more parallelism in testing.
e.g. It's fair game to run all tests at the same time on the
same machine.</div>
<div><br>
</div>
<div>
<div>find . -name \*.py | xargs wc -l | tail -1</div>
<div> 10684 total</div>
<div><br>
</div>
<div>find . -name \*.cc -o -name \*.h | xargs wc -l | tail -1<br>
</div>
<div> 3734 total</div>
</div>
<div><br>
</div>
<div>Where GDAL's autotest is 204K lines:</div>
<div><br>
</div>
<div>
<div>find . -name \*.py | xargs wc -l | tail -1</div>
<div> 193994 total</div>
</div>
<div>
<div>find . -name \*.c\* -o -name \*.h | xargs wc -l | tail -1</div>
<div> 10471 total</div>
</div>
<div><br>
</div>
<div>Here are some samples:</div>
<div><br>
</div>
<div>C++ tests use C++11, gunit, google logging, gflags:
(Hoping for C++14 soon.. e.g. make_unique)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/13137d826763763fb031">autotest2/cpp/port/cpl_conv_test.cc</a> (Yes,
this is massively boring code)<br>
</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/c8ee86a6f6a1c1cc043b">autotest2/cpp/ogr/ogrpoint_test.cc</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/bc44b91a37cd621212c4">autotest2/cpp/ogr/ogrsf_frmts/geojson/geojson_test.cc</a></div>
<div><br>
</div>
<div>Python pretty much follows the autotest layout, but with
util files in the same directory. Assumes python 2.7 or >=
3.4 (have not tried py3 yet)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/c143927ca25d03a10265">autotest2/gcore/gcore_util.py</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/dd75f73cedf8f7b5357e">autotest2/gdrivers/gdrivers_util.py</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/a35b2bc8a7956ef1f620">autotest2/gdrivers/tiff_read_test.py</a> (I'm
leading towards moving driver tests in gcore to gdrivers)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/6cbdc3482055d2237ad2">autotest2/ogr/geojson_test.py</a></div>
<div><br>
</div>
<div>Probably should move python code to also match the C++
tree. e.g.</div>
<div><br>
</div>
<div> tiff_read_test.py ->
autotest2/py/frmts/gtiff/tiff_read_test.py<br>
</div>
<div><br>
</div>
<div>I'm (mostly) following Google's style guides. Public
versions here: <a moz-do-not-send="true"
href="https://google-styleguide.googlecode.com/svn/trunk/cppguide.html">C++</a> <a
moz-do-not-send="true"
href="https://google-styleguide.googlecode.com/svn/trunk/pyguide.html">Python</a></div>
<div><br>
</div>
<div>All C++ should be formatted with "clang-format
--style=Google"</div>
<div><br>
</div>
<div>What does autotest2 not do?</div>
<div><br>
</div>
<div>Would like to eventually do (unsorted):</div>
<div>- Test error handling on a range of corrupt data sources</div>
<div>- Fuzz testing, ASAN/MSAN/TSAN/Valgrind/Heap checks (I've
done some MSAN & heap checkers by hand)</div>
<div>- Performance testing - time and memory usage</div>
<div>- Test the C API at the C level</div>
<div>- Test platforms other than Linux (MS Win*, Mac OSX,
Android, iOS, other embedded oses, BSD*, Solaris, HPUX, etc)</div>
<div>- Have more detailed language binding tests for java, ruby,
perl, php</div>
<div>
<div>- Coverage checking<br>
</div>
</div>
<div>- Test parallel processing and multithreading</div>
<div>- Test networking (I need to think through isolation)</div>
<div>- Test multiple configurations (e.g. all drivers and
features enables vrs minimal build).</div>
<div>- Check which system calls are used by each driver for read
and for write</div>
<div>- Check i18n support.</div>
<div>- Check distribution packaging</div>
<div>- Validating that the given build options result in the
expect available features</div>
<div><br>
</div>
<div>Probably out of scope: </div>
<div>- Test for support from older platforms & C++ older
than C++11<br>
</div>
<div>- Actual sandbox checks</div>
<div>- Test other bindings to GDAL's C or C++ API such as Fiona
& Shapely <br>
</div>
<div>- Integration tests (e.g. GRASS, QGIS, mapserv, GeoDjango,
etc).</div>
<div>- ABI compatibility checks</div>
<div>- Older versions of dependent libs e.g. netcdf/hdf4/5,
kakadu, openjpeg, etc.</div>
<div><br>
</div>
<div>-kurt</div>
<div>Engineer at Google</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sat, Sep 5, 2015 at 7:48 AM,
Dmitry Baryshnikov <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bishop.dev@gmail.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:bishop.dev@gmail.com">bishop.dev@gmail.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi
Even,<br>
<br>
05.09.2015 17:10, Even Rouault пишет:<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Dmitry,<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">During
the code sprint in FOSS4G 2015 (Korea, Seoul) I plan
to start<br>
refactoring Cmake for GDAL (everybody are welcome<br>
<a moz-do-not-send="true"
href="http://2015.foss4g.org/programme/code-sprint/"
rel="noreferrer" target="_blank">http://2015.foss4g.org/programme/code-sprint/</a>).
This is good starting<br>
point to try release an idea to reformat source tree
(combine drivers on<br>
some principles - raster/vector/raster+vector). I
digging the mailing<br>
list, but didn't found discussion started by Even
about this.<br>
</blockquote>
Regarding unified drivers, it was a bit mentionned in<br>
<a moz-do-not-send="true"
href="https://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification"
rel="noreferrer" target="_blank">https://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification</a>
. Basically the<br>
PCIDSK drivers have been merged in frmts/pcidsk, the
PDF ones in frmts/pdf.<br>
And the raster side of GPKG has been added to the
existing<br>
ogr/ogrsf_frmts/geopackage<br>
Potential changes on the tree structure were left out
in the "Potential<br>
changes that are *NOT* included in this RFC"
paragraph.<br>
</blockquote>
</span>
I plan to experiment with this and if I get good results,
RFC will be written.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Also
we have<br>
new type of drivers - network. So, how it'll be best
to organise sources?<br>
This can be not only drivers, but the whole source
tree. How should the<br>
ideal GDAL source tree looks like?<br>
<br>
Also I plan:<br>
1. Move all internal libraries (zlib, libtiff,
libjpeg, etc.) to<br>
separate github repos to use CMake ExternalProject
feature.<br>
</blockquote>
Just to give some context: the point for the internal
libraries was to have a<br>
no-brainer way of building GDAL without any
prerequisite.<br>
- internal zlib is identical to its upstream v1.2.3
AFAIK<br>
- internal libtiff: most files are identical to
libtiff CVS, but a few ones<br>
(tiffconf.h, tif_config.h) have been modified for
integration with GDAL CPL, and<br>
tif_vsi.c is GDAL specific (I/O implementation) + a
build time hack for TIFF<br>
JPEG 12 bit support<br>
- internal libjpeg is mostly upstream libjpeg v6b +
one patch. There's also<br>
the build hack for libjpeg12<br>
</blockquote>
</span>
I only plan to move this internal libraries in separate
repos, not to link official ones. So this is only give
more structured sources tree.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">2.
Remove any other building systems<br>
</blockquote>
That sounds ambitious. Given the complexity and
maturity of our current build<br>
systems, I guess this would take some time to have
CMake catch up.<br>
</blockquote>
</span>
Yes, certainly. But anyhow current CMake branch not fully
consistent for current build system. So this have to be
done.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">3.
Try CTest for testing<br>
</blockquote>
What do you think it will bring w.r.t our current
testing system ? Do we want<br>
to be dependant of a particular build system for our
tests ?<br>
Regarding testing, I've somehow understood Kurt had
mentionned plans for a<br>
"gdalautotest2"<br>
</blockquote>
</span>
This is only subject of experiments. Let's try CTest and
see if it fits.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
Regarding all the above, I assume you mean in a fork
of yours ?<br>
</blockquote>
</span>
Yes. All experiments will be on forked GDAL in separate
branch.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">As
for me the ideal structure should looks like:<br>
+ apps<br>
+ autotests<br>
+ bindings<br>
+ core<br>
+ port<br>
+ ogr<br>
+ gcore<br>
</blockquote>
gnm core would go here too ?<br>
</blockquote>
</span>
Yes<span class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">+
cmake<br>
+ data<br>
+ docs<br>
+ doxygen<br>
+ readme<br>
+ drivers<br>
+ raster<br>
+ vector<br>
+ network<br>
+ combined<br>
+ CMakeLists.txt<br>
+ LICENSE<br>
<br>
So, at the root of sources tree we will have only 8
folders and 2 files.<br>
</blockquote>
Is the churn in the tree structure worth the effort ?
Be aware that there are a<br>
number of interdependencies between drivers, so this
might require fixing a<br>
number of source files. What advantages do you see in
a new structure ?<br>
</blockquote>
</span>
1. More ease to understand sources tree for novice.<br>
2. More useful for CMake macro. In current release there
are a lot of hardcoded things. Macro give more
flexibility.<br>
3. More ease to add some new check needed by separate
drivers.<br>
4. More configurable (ease selected depended libraries
installed in OS, or should be loaded via ExternalProject),
more dependence checks.<br>
5. May be CPack using in future to create distros.<span
class=""><br>
<blockquote class="gmail_quote" style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
I'm afraid that if you want to change multiple things
at a time (build system,<br>
testing mechanisms, tree structure), you will never
manage to get a working<br>
result. Incremental approaches when feasible are less
risky (but admitedly<br>
involve potentially a larger cumulated effort).<br>
</blockquote>
</span>
Yes, you may be right. But it seems to me that current
Cmake version is too complicated than it can be. If Ican
improve it it'll solve lot of problems, if not - ok this
will be only an unsuccessful experiment.<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Even<br>
<br>
</blockquote>
I do not insist, maybe it's a crazy idea. But as was the
discussion of unification, it seemed to me that this worth
trying during improvements Cmake build system.<br>
<br>
Best regards,<br>
Dmitry</blockquote>
</div>
<div class="gmail_signature"><br>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>