[mapserver-dev] the state of msautotest

[Smith, Michael ERDC-CRREL-NH]

If you need a dedicated server and hosting location for testing we could
provide this.


Mike

-- 
Michael Smith

Remote Sensing/GIS Center
US Army Corps of Engineers



On 9/12/12  7:14 AM, "thomas bonfort" <thomas.bonfort at gmail.com> wrote:

>After maybe a couple beers too many with Stephan the other day we came
>to talk about mapserver's autotests and it's current deficiencies with
>respect to how we are using them.
>
>To summarize, here are the principle issues that arose:
>
>- The tests aren't run on each commit, which is understandable as it
>takes an additional effort to do so
>- We have a large number of failing tests, which can be attributed to:
>  - the afore-mentioned reason: is a failing test a result of the last
>commit or was it already failing beforehand?
>  - image comparison tests can produce false positives, due to
>different cpu architectures and underlying dependency libraries
>(namely freetype)
>  - image and xml/gml tests can diverge depending on the compile-time
>configuration options that were chosen.
>- we've ended up in a chicken vs. egg situation where tests aren't
>being run because they are failing, and test fail in the long run
>because they are never being used/updated.
>
>To alleviate this, we'd like to set the ball rolling on these ideas:
>- provide a "reference" platform with a kitchen-sink (i.e. compiled
>with all options) mapserver instance. This could be an osgeo ad-hoc vm
>or a dedicated server provided by one of us or our users. (second
>option might be better given the limited resources available at osgeo)
>- go through all the current tests, make sure they pass on this
>reference platform, and update the expected result if not.
>- automatically run the whole test-suite on each commit, and provide a
>web-page or email alert when a test has failed. Does anyone have
>experience with a software solution that could provide that to us?
>
>Thoughts? It's certainly not a miracle solution as it will require
>acting upon when a test starts failing, however we believe that
>automating the runs and removing the false positives will be a nice
>step forward (and give more peace-of-mind to the release-manager when
>packaging a release :) )
>
>cheers,
>thomas
>_______________________________________________
>mapserver-dev mailing list
>mapserver-dev at lists.osgeo.org
>http://lists.osgeo.org/mailman/listinfo/mapserver-dev