[gdal-dev] Participating on gdal2tiles

[Even Rouault]

On dimanche 22 janvier 2017 11:21:06 CET Grégory Bataille wrote:
> Yes it is. There was a commit on the test_gdal2tiles.py changing the
> checksum values for test 1 :)

Well, that's the life of a project that test results might change because of other changes. Not 
sure to understand if that's your issue, or if it is something Mac specific.

> 
> that's odd, it's pyflakes that is throwing me those warnings... well flake8
> actually that is pyflakes + pep8. I'll have a look anyway.
> And yes, the style commits need to be separate, no worries.
> 
> I did launch the vagrant by commenting this missing script. Now what is the
> vagrant box supposed to do? my understanding is that is comes with gdal
> installed (from trunk?)
> I think the gdal2tiles.py test (and maybe some other python ones) are not
> working in the vagrant box because those tests are looking for scripts to
> test (gdal2tiles.py for example) in relative from the test file
> test_gdal2tiles.py and because the Vagrantfile only maps the autotest
> folder from the host to the client VM, the source for those python scripts
> is not available in the VM.
> Is it on purpose? would you see any issue mapping the source gdal folder
> inside the vagrant box next to the autotest folder?

Honestly I did the initial crack on setting up the Vagrant things as it was suggested it might 
help other people, but I don't use it personnaly, so it has probably rotten up, or might not be 
optimally done. The root folder (ie the one with the gdal/ and autotest/ directories) should 
probably be done. Feel free to do appropriate changes.

> 
> Sorry for the questions, it's always hard to enter into a project.
> 
> Last one actually for this mail:
> I have an issue on a particular image. To test a fix (and non-regression in
> the future) I can either try to do some mocking (but seeing how
> gdal2tiles.py is not very modular, that might prove a nightmare) or use
> some specific image that I know present the issue. If I want to test based
> on an image, is it an ok practice to put the image in the repo (I can see
> that there are several already but that doesn't seem like a great idea to
> me). Any other common storage place (AWS S3 or whatever) that you would
> have available or recommend otherwise?

We must indeed be careful before adding new binaries (like images) in the repository to 
avoid its size to grow out of control. It is best if one can reuse existing images. Or create 
synthetic images that compress very well to few kilobytes or ten of kilobytes (sometimes the 
creation can be part of the test, so you don't have to store any binary in the repo). If the 
important thing is the size and georeferencing of the image, but not the pixel content, you 
can easily do a small black image with
"gdal_translate in.tif out.tif -scale 0 1 0 0 -co TILED=YES -co SPARSE_OK=YES" (with trunk)
Actually you can also do "gdal_translate in.tif out.vrt -of VRT" and edit the VRT to remove all 
<SimpleSource>. That will be even smaller.

We don't have a project AWS S3 bucket, but we have
http://download.osgeo.org/gdal/data/ where we can put some images. But it is a bit more 
cumbersome than having the images directly accessible from the SVN since you must do 
some checks (you can grep autotest/ for "http://download.osgeo.org/gdal/data/" to see how 
we use them)). And by default in the Travis-CI setups we don't enable those downloads from 
external resources because of the extra time it can take and reliability problems it causes.

One aspect that might impact test is the proj.4 version being used. We don't currently have a 
requirement on the proj.4 version used and I'm pretty sure we have different versions used in 
the various Travis-CI + AppVeyor configs. So we sometimes have to add alternate expected 
checksums when the test result is sensitive to proj.4 changes.

Writing tests might be non-trivial. I'm very well aware of the problem.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20170122/9dece6d3/attachment.html>