[Qgis-developer] american fuzzy lop

[Even Rouault]

Le vendredi 06 mars 2015 10:44:48, Hugo Mercier a écrit :
> Hi,
> 
> Interesting.
> 
> The idea of fuzzing is to add random noise to inputs of a program in
> order to make it crash and exhibit hard-to-test branches of code.
> It is very well suited for programs that parse complex inputs.
> 
> For QGIS, the "inputs" are either project files or external data
> sources, where it relies on third party libraries and where the inputs
> are not always easy to represent as files (how to fuzz a postgis database
> ?)

> 
> So this may be interesting for testing project file parsing.

For XML, you need dedicated fuzzing tool. afl (as far as I know) is more 
dedicated to binary fuzzing, which make 99.99999% of what it produces invalid 
XML that will be rejected by the XML decoding library before it reaches the 
application level.

> But I think it would be better suited for external libraries like GDAL.

Indeed, I've used it a bit recently in GDAL, mainly on the GTiff driver and it 
did finds things. Note that afl needs generally to run for several minutes/hours 
to be able to find things (unless the code is particularly not robust in which 
case a few seconds will be enough to trigger bugs)

> 
> Anyway, automatic tools are always good to try if they can improve the
> overall quality.
> 
> Speaking about tests for external sources, it would be good to add (or
> complete) unit tests for "direct" providers (spatialite, postgis, etc.),
> probably as an option, since the setup is platform-dependent.
> 
> Le 06/03/2015 07:55, Paolo Cavallini a écrit :
> > It promises to find bugs and help fixing them.
> > 
> > Il 06 marzo 2015 07:53:57 CET, Nathan Woodrow <madmanwoo at gmail.com> ha
> > 
> > scritto:
> >     What does that even do?
> >     
> >     On Fri, 6 Mar 2015 at 16:52 Paolo Cavallini <cavallini at faunalia.it
> >     
> >     <mailto:cavallini at faunalia.it>> wrote:
> >         Hi all.
> >         Anybody knows about this?
> >         http://lcamtuf.coredump.cx/afl/
> >         Would it make sense to try and include it in our tests?
> >         All the best.
> 
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com