<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Karl,<br>
<br>
I think that some ready tests of GDAL are available. I'm not
familiar with autotest2, so some links to docs and sources are
welcome. Also I think that in gist you provided only some examples,
and somewhere else the repo exists. If not - I'll try to play with
this code.<br>
<pre class="moz-signature" cols="72">Best regards,
Dmitry</pre>
<div class="moz-cite-prefix">15.09.2015 00:42, Kurt Schwehr пишет:<br>
</div>
<blockquote
cite="mid:CACmBxyv1NMOjiDgNZG10Cw2Vrxd3XfO0jY5nUhLK9bNFZ65U5A@mail.gmail.com"
type="cite">
<div dir="ltr">Hi Dmitry,
<div><br>
</div>
<div>I didn't totally understand your question. I don't have
autotest2 really out yet. You are welcome to use the sample
test files that I posted in any way that the Apache 2.0
license allows.</div>
<div><br>
</div>
<div>-kurt</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Sep 6, 2015 at 8:27 AM, Dmitry
Baryshnikov <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:bishop.dev@gmail.com" target="_blank">bishop.dev@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hi Kurt,<br>
<br>
During code sprint in Korea (FOSS4G 2015) I plan to play
with new approach of CMake fro GDAL. The one of
experiments will be try to use CTest. As I plan
restructure the sources tree, I can try to integrate you
work on autotest2 and CTest. Also we can try to create new
test directory structure more compatible for test and
sources tree (this is about you wrote: Probably should
move python code to also match the C++ tree. e.g.
tiff_read_test.py ->
autotest2/py/frmts/gtiff/tiff_read_test.py).<br>
How can get you tests? What do I need to use autotest2?<br>
<br>
<pre cols="72">Best regards,
Dmitry</pre>
<div>05.09.2015 20:37, Kurt Schwehr пишет:<br>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">
<div>(Subject change to focus on testing)<br>
</div>
<div><br>
</div>
Hi all,
<div><br>
</div>
<div>First off... what GDAL has with autotest,
travis-ci and coverity is awesome!</div>
<div><br>
</div>
<div>
<div>Thoughts / discussion more than welcome!</div>
</div>
<div><br>
</div>
<div>For my production work, I'm not able to use
the autotest python code because of its
non-unittest architecture. So... I started
creating python unittest and C++ gunit based
tests. I use autotest2 in Google's internal
continuous integration system in our main code
base. I'm using Google's build system... I've
got nothing started for running the C++ tests
outside of Google.<br>
</div>
<div><br>
</div>
<div>Apologies for not even getting out at least
samples of autotest2 for folks to inspect and
comment on. My intention is to put what I have
in a git repo and the to start discussions as to
what (if anything) GDAL community wants to do
with autotest2. I was hoping to get a lot
more coverage and get GDAL 2.x.x support, but
that will have to come later. It's only 14K
lines at this point (optimistically 2-3% done),
but it has been a huge help for me especially
with in upgrading versions of gdal and catching
bugs in support libs & development
toolchains. </div>
<div><br>
</div>
<div>The tests are more focused on test isolation
than autotest. This allows for a lot more
parallelism in testing. e.g. It's fair game to
run all tests at the same time on the same
machine.</div>
<div><br>
</div>
<div>
<div>find . -name \*.py | xargs wc -l | tail -1</div>
<div> 10684 total</div>
<div><br>
</div>
<div>find . -name \*.cc -o -name \*.h | xargs wc
-l | tail -1<br>
</div>
<div> 3734 total</div>
</div>
<div><br>
</div>
<div>Where GDAL's autotest is 204K lines:</div>
<div><br>
</div>
<div>
<div>find . -name \*.py | xargs wc -l | tail -1</div>
<div> 193994 total</div>
</div>
<div>
<div>find . -name \*.c\* -o -name \*.h | xargs
wc -l | tail -1</div>
<div> 10471 total</div>
</div>
<div><br>
</div>
<div>Here are some samples:</div>
<div><br>
</div>
<div>C++ tests use C++11, gunit, google logging,
gflags: (Hoping for C++14 soon.. e.g.
make_unique)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/13137d826763763fb031"
target="_blank">autotest2/cpp/port/cpl_conv_test.cc</a> (Yes,
this is massively boring code)<br>
</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/c8ee86a6f6a1c1cc043b"
target="_blank">autotest2/cpp/ogr/ogrpoint_test.cc</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/bc44b91a37cd621212c4"
target="_blank">autotest2/cpp/ogr/ogrsf_frmts/geojson/geojson_test.cc</a></div>
<div><br>
</div>
<div>Python pretty much follows the autotest
layout, but with util files in the same
directory. Assumes python 2.7 or >= 3.4
(have not tried py3 yet)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/c143927ca25d03a10265"
target="_blank">autotest2/gcore/gcore_util.py</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/dd75f73cedf8f7b5357e"
target="_blank">autotest2/gdrivers/gdrivers_util.py</a></div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/a35b2bc8a7956ef1f620"
target="_blank">autotest2/gdrivers/tiff_read_test.py</a> (I'm
leading towards moving driver tests in gcore to
gdrivers)</div>
<div>- <a moz-do-not-send="true"
href="https://gist.github.com/schwehr/6cbdc3482055d2237ad2"
target="_blank">autotest2/ogr/geojson_test.py</a></div>
<div><br>
</div>
<div>Probably should move python code to also
match the C++ tree. e.g.</div>
<div><br>
</div>
<div> tiff_read_test.py ->
autotest2/py/frmts/gtiff/tiff_read_test.py<br>
</div>
<div><br>
</div>
<div>I'm (mostly) following Google's style
guides. Public versions here: <a
moz-do-not-send="true"
href="https://google-styleguide.googlecode.com/svn/trunk/cppguide.html"
target="_blank">C++</a> <a
moz-do-not-send="true"
href="https://google-styleguide.googlecode.com/svn/trunk/pyguide.html"
target="_blank">Python</a></div>
<div><br>
</div>
<div>All C++ should be formatted with
"clang-format --style=Google"</div>
<div><br>
</div>
<div>What does autotest2 not do?</div>
<div><br>
</div>
<div>Would like to eventually do (unsorted):</div>
<div>- Test error handling on a range of corrupt
data sources</div>
<div>- Fuzz testing, ASAN/MSAN/TSAN/Valgrind/Heap
checks (I've done some MSAN & heap checkers
by hand)</div>
<div>- Performance testing - time and memory usage</div>
<div>- Test the C API at the C level</div>
<div>- Test platforms other than Linux (MS Win*,
Mac OSX, Android, iOS, other embedded oses,
BSD*, Solaris, HPUX, etc)</div>
<div>- Have more detailed language binding tests
for java, ruby, perl, php</div>
<div>
<div>- Coverage checking<br>
</div>
</div>
<div>- Test parallel processing and multithreading</div>
<div>- Test networking (I need to think through
isolation)</div>
<div>- Test multiple configurations (e.g. all
drivers and features enables vrs minimal build).</div>
<div>- Check which system calls are used by each
driver for read and for write</div>
<div>- Check i18n support.</div>
<div>- Check distribution packaging</div>
<div>- Validating that the given build options
result in the expect available features</div>
<div><br>
</div>
<div>Probably out of scope: </div>
<div>- Test for support from older platforms &
C++ older than C++11<br>
</div>
<div>- Actual sandbox checks</div>
<div>- Test other bindings to GDAL's C or C++ API
such as Fiona & Shapely <br>
</div>
<div>- Integration tests (e.g. GRASS, QGIS,
mapserv, GeoDjango, etc).</div>
<div>- ABI compatibility checks</div>
<div>- Older versions of dependent libs e.g.
netcdf/hdf4/5, kakadu, openjpeg, etc.</div>
<div><br>
</div>
<div>-kurt</div>
<div>Engineer at Google</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sat, Sep 5, 2015 at
7:48 AM, Dmitry Baryshnikov <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bishop.dev@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:bishop.dev@gmail.com">bishop.dev@gmail.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi
Even,<br>
<br>
05.09.2015 17:10, Even Rouault пишет:<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Dmitry,<br>
<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">During
the code sprint in FOSS4G 2015 (Korea,
Seoul) I plan to start<br>
refactoring Cmake for GDAL (everybody
are welcome<br>
<a moz-do-not-send="true"
href="http://2015.foss4g.org/programme/code-sprint/"
rel="noreferrer" target="_blank">http://2015.foss4g.org/programme/code-sprint/</a>).
This is good starting<br>
point to try release an idea to
reformat source tree (combine drivers
on<br>
some principles -
raster/vector/raster+vector). I
digging the mailing<br>
list, but didn't found discussion
started by Even about this.<br>
</blockquote>
Regarding unified drivers, it was a bit
mentionned in<br>
<a moz-do-not-send="true"
href="https://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification"
rel="noreferrer" target="_blank">https://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification</a>
. Basically the<br>
PCIDSK drivers have been merged in
frmts/pcidsk, the PDF ones in frmts/pdf.<br>
And the raster side of GPKG has been
added to the existing<br>
ogr/ogrsf_frmts/geopackage<br>
Potential changes on the tree structure
were left out in the "Potential<br>
changes that are *NOT* included in this
RFC" paragraph.<br>
</blockquote>
</span> I plan to experiment with this and
if I get good results, RFC will be written.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Also
we have<br>
new type of drivers - network. So, how
it'll be best to organise sources?<br>
This can be not only drivers, but the
whole source tree. How should the<br>
ideal GDAL source tree looks like?<br>
<br>
Also I plan:<br>
1. Move all internal libraries (zlib,
libtiff, libjpeg, etc.) to<br>
separate github repos to use CMake
ExternalProject feature.<br>
</blockquote>
Just to give some context: the point for
the internal libraries was to have a<br>
no-brainer way of building GDAL without
any prerequisite.<br>
- internal zlib is identical to its
upstream v1.2.3 AFAIK<br>
- internal libtiff: most files are
identical to libtiff CVS, but a few ones<br>
(tiffconf.h, tif_config.h) have been
modified for integration with GDAL CPL,
and<br>
tif_vsi.c is GDAL specific (I/O
implementation) + a build time hack for
TIFF<br>
JPEG 12 bit support<br>
- internal libjpeg is mostly upstream
libjpeg v6b + one patch. There's also<br>
the build hack for libjpeg12<br>
</blockquote>
</span> I only plan to move this internal
libraries in separate repos, not to link
official ones. So this is only give more
structured sources tree.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">2.
Remove any other building systems<br>
</blockquote>
That sounds ambitious. Given the
complexity and maturity of our current
build<br>
systems, I guess this would take some
time to have CMake catch up.<br>
</blockquote>
</span> Yes, certainly. But anyhow current
CMake branch not fully consistent for
current build system. So this have to be
done.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">3.
Try CTest for testing<br>
</blockquote>
What do you think it will bring w.r.t
our current testing system ? Do we want<br>
to be dependant of a particular build
system for our tests ?<br>
Regarding testing, I've somehow
understood Kurt had mentionned plans for
a<br>
"gdalautotest2"<br>
</blockquote>
</span> This is only subject of experiments.
Let's try CTest and see if it fits.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
Regarding all the above, I assume you
mean in a fork of yours ?<br>
</blockquote>
</span> Yes. All experiments will be on
forked GDAL in separate branch.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">As
for me the ideal structure should
looks like:<br>
+ apps<br>
+ autotests<br>
+ bindings<br>
+ core<br>
+ port<br>
+ ogr<br>
+ gcore<br>
</blockquote>
gnm core would go here too ?<br>
</blockquote>
</span> Yes<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">+
cmake<br>
+ data<br>
+ docs<br>
+ doxygen<br>
+ readme<br>
+ drivers<br>
+ raster<br>
+ vector<br>
+ network<br>
+ combined<br>
+ CMakeLists.txt<br>
+ LICENSE<br>
<br>
So, at the root of sources tree we
will have only 8 folders and 2 files.<br>
</blockquote>
Is the churn in the tree structure worth
the effort ? Be aware that there are a<br>
number of interdependencies between
drivers, so this might require fixing a<br>
number of source files. What advantages
do you see in a new structure ?<br>
</blockquote>
</span> 1. More ease to understand sources
tree for novice.<br>
2. More useful for CMake macro. In current
release there are a lot of hardcoded things.
Macro give more flexibility.<br>
3. More ease to add some new check needed by
separate drivers.<br>
4. More configurable (ease selected depended
libraries installed in OS, or should be
loaded via ExternalProject), more dependence
checks.<br>
5. May be CPack using in future to create
distros.<span><br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
I'm afraid that if you want to change
multiple things at a time (build system,<br>
testing mechanisms, tree structure), you
will never manage to get a working<br>
result. Incremental approaches when
feasible are less risky (but admitedly<br>
involve potentially a larger cumulated
effort).<br>
</blockquote>
</span> Yes, you may be right. But it seems
to me that current Cmake version is too
complicated than it can be. If Ican improve
it it'll solve lot of problems, if not - ok
this will be only an unsuccessful
experiment.<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Even<br>
<br>
</blockquote>
I do not insist, maybe it's a crazy idea.
But as was the discussion of unification, it
seemed to me that this worth trying during
improvements Cmake build system.<br>
<br>
Best regards,<br>
Dmitry</blockquote>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">--
<div><a moz-do-not-send="true" href="http://schwehr.org"
target="_blank">http://schwehr.org</a></div>
</div>
</div>
</blockquote>
<br>
</body>
</html>