[postgis-devel] PostGIS (actually liblwgeom) integration with oss-fuzz

[Even Rouault]

Hi,

I've prototyped an integration of liblwgeom from PostGIS repository with oss-fuzz.

Quoting https://github.com/google/oss-fuzz/ ,
"""
Fuzz testing is a well-known technique for uncovering various kinds of programming errors in software.
Many of these detectable errors (e.g. buffer overflow) can have serious security implications.

We successfully deployed guided in-process fuzzing of Chrome components and found hundreds of
security vulnerabilities and stability bugs. We now want to share the experience and the
service with the open source community.

In cooperation with the Core Infrastructure Initiative, OSS-Fuzz aims to make common
open source software more secure and stable by combining modern fuzzing techniques
and scalable distributed execution.
""""

GDAL and proj.4 have joined oss-fuzz, for a few weeks  and this is really efficient. I've fixed between 300 and 400 bugs in GDAL...

So I just gave it a try with PostGIS, concentrating on liblwgeom, since it builds nicely in oss-fuzz environment
(plain "make" in top repository fails in oss-fuzz from some reason I haven't investigated)
As an example, I've created 2 fuzzers, one for lwgeom_from_wkb() and the other one for lwgeom_from_wkt().
More could be done, based on those examples. Left as an exercice to other developers.

Integration of a software with oss-fuzz is made of 2 parts:
- fuzzer entry points must be in the project repository : https://github.com/rouault/postgis/tree/ossfuzz/fuzzers
- a metadata file (project.yaml), a Dockerfile (download needed packages & PostGIS source code) and
a "bootstrap" build.sh script  must be integrated in OSS-Fuzz own repo too :
https://github.com/rouault/oss-fuzz/tree/postgis/projects/postgis

For now, I've done this in my own postgis and ossz-fuzz git forks as you can see, but ultimately if the
project agrees we should merge this into their respective official repos.

But pending that, you can already try this stuff locally
{{{
Make sure you have Docker installed

git clone  --branch postgis https://github.com/rouault/oss-fuzz
cd oss-fuzz

Build the Docker image:
python infra/helper.py build_image postgis

Build PostGIS and the fuzzer programs with the address sanitizer
python infra/helper.py build_fuzzers --sanitizer address postgis

Run one of the fuzzer (you can try with wkt_import_fuzzer too)
python infra/helper.py run_fuzzer postgis wkb_import_fuzzer
}}}

My local experiments show that lwgeom_from_wkt() seems to be rather robust,
but lwgeom_from_wkb() has a few bugs. For the record, I found and fixed (in my fork) this memory leak
(memory leak = leak in a case where lw_error() is not called) in
https://github.com/rouault/postgis/commit/cf179396b719223653eee56a01189339e0abcc0d
There's also a heap buffer overflow it just detected in
ptarray_from_wkb_state /src/postgis/liblwgeom/lwin_wkb.c:367

So if the community is interested in a closer integration in OSS Fuzz, next steps are :
1) someone with PostGIS commit rights merges https://github.com/rouault/postgis/commit/0181a28ab01764b4e6d11a5d2ffe7edce96498c6
into PostGIS SVN (as well as the bug fix https://github.com/rouault/postgis/commit/cf179396b719223653eee56a01189339e0abcc0d while you are it)
2) interested core PostGIS developers give me a @gmail.com email, so I add it in
https://github.com/rouault/oss-fuzz/blob/postgis/projects/postgis/project.yaml 
This way they will then have access to the bug reports that are embargoed for
90 days (or 30 days after OSS Fuzz has found them to be fixed)
3) I then modify https://github.com/rouault/oss-fuzz/blob/postgis/projects/postgis/Dockerfile to
point to PostGIS official github mirror instead of my fork
4) I then submit a pull request to https://github.com/google/oss-fuzz/ with my
https://github.com/rouault/oss-fuzz postgis branch . They may accept or not the application, but I guess they will accept.
If they don't, you can also play with it locally as I showed above. And this is strongly recommended
to do so when adding a new fuzzer for example.
5) once the project is accepted, monitor https://bugs.chromium.org/p/oss-fuzz/issues/list?q=postgis and fix the bugs !
6) add more fuzzers. Hint: in PostGIS "fuzzers" directory, "make dummyfuzzers" to check that your fuzzer builds.

Note: I don't volunteer to fix all bugs that will be found. I have already enough to do with GDAL... I wouldn't
mind if someone wants to be the declared maintainer in oss-fuzz projects/postgis/project.yaml and projects/postgis/Dockerfile

Note 2: if you look closely at https://github.com/rouault/postgis/blob/ossfuzz/fuzzers/wkb_import_fuzzer.cpp
you will notice that it is a bit messy since it stubs GEOS and geod_ symbols. This is due to the fact
that OSS-Fuzz requires that the fuzzer programs are completely statically linked, and Ubuntu doesn't ship
with static builds of geos (actually just libgeos.a but no libgeos_c.a) as far as I can see.
All this could be improved by adding a download of GEOS source code in
https://github.com/rouault/oss-fuzz/blob/postgis/projects/postgis/Dockerfile and building it manually in
https://github.com/rouault/oss-fuzz/blob/postgis/projects/postgis/build.sh

Thoughts ?

Even


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20170702/d42516aa/attachment.html>