[postgis-devel] PostGIS (actually liblwgeom) integration with oss-fuzz

[Paul Ramsey]

This seems very cool but also very involved and complicated :)
I think everyone is concomitantly afraid to step forward and pick it up.
It's got it all: external services, big chains of dependencies, docker :)
I think everyone is a little afraid to pick it up, lest they own it.
P.


On Sun, Jul 2, 2017 at 12:07 PM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Hi,
>
>
>
> I've prototyped an integration of liblwgeom from PostGIS repository with
> oss-fuzz.
>
>
>
> Quoting https://github.com/google/oss-fuzz/ ,
>
> """
>
> Fuzz testing is a well-known technique for uncovering various kinds of
> programming errors in software.
>
> Many of these detectable errors (e.g. buffer overflow) can have serious
> security implications.
>
>
>
> We successfully deployed guided in-process fuzzing of Chrome components
> and found hundreds of
>
> security vulnerabilities and stability bugs. We now want to share the
> experience and the
>
> service with the open source community.
>
>
>
> In cooperation with the Core Infrastructure Initiative, OSS-Fuzz aims to
> make common
>
> open source software more secure and stable by combining modern fuzzing
> techniques
>
> and scalable distributed execution.
>
> """"
>
>
>
> GDAL and proj.4 have joined oss-fuzz, for a few weeks and this is really
> efficient. I've fixed between 300 and 400 bugs in GDAL...
>
>
>
> So I just gave it a try with PostGIS, concentrating on liblwgeom, since it
> builds nicely in oss-fuzz environment
>
> (plain "make" in top repository fails in oss-fuzz from some reason I
> haven't investigated)
>
> As an example, I've created 2 fuzzers, one for lwgeom_from_wkb() and the
> other one for lwgeom_from_wkt().
>
> More could be done, based on those examples. Left as an exercice to other
> developers.
>
>
>
> Integration of a software with oss-fuzz is made of 2 parts:
>
> - fuzzer entry points must be in the project repository :
> https://github.com/rouault/postgis/tree/ossfuzz/fuzzers
>
> - a metadata file (project.yaml), a Dockerfile (download needed packages &
> PostGIS source code) and
>
> a "bootstrap" build.sh script must be integrated in OSS-Fuzz own repo too :
>
> https://github.com/rouault/oss-fuzz/tree/postgis/projects/postgis
>
>
>
> For now, I've done this in my own postgis and ossz-fuzz git forks as you
> can see, but ultimately if the
>
> project agrees we should merge this into their respective official repos.
>
>
>
> But pending that, you can already try this stuff locally
>
> {{{
>
> Make sure you have Docker installed
>
>
>
> git clone --branch postgis https://github.com/rouault/oss-fuzz
>
> cd oss-fuzz
>
>
>
> Build the Docker image:
>
> python infra/helper.py build_image postgis
>
>
>
> Build PostGIS and the fuzzer programs with the address sanitizer
>
> python infra/helper.py build_fuzzers --sanitizer address postgis
>
>
>
> Run one of the fuzzer (you can try with wkt_import_fuzzer too)
>
> python infra/helper.py run_fuzzer postgis wkb_import_fuzzer
>
> }}}
>
>
>
> My local experiments show that lwgeom_from_wkt() seems to be rather robust,
>
> but lwgeom_from_wkb() has a few bugs. For the record, I found and fixed
> (in my fork) this memory leak
>
> (memory leak = leak in a case where lw_error() is not called) in
>
> https://github.com/rouault/postgis/commit/cf179396b719223653eee56a011893
> 39e0abcc0d
>
> There's also a heap buffer overflow it just detected in
>
> ptarray_from_wkb_state /src/postgis/liblwgeom/lwin_wkb.c:367
>
>
>
> So if the community is interested in a closer integration in OSS Fuzz,
> next steps are :
>
> 1) someone with PostGIS commit rights merges https://github.com/rouault/
> postgis/commit/0181a28ab01764b4e6d11a5d2ffe7edce96498c6
>
> into PostGIS SVN (as well as the bug fix https://github.com/rouault/
> postgis/commit/cf179396b719223653eee56a01189339e0abcc0d while you are it)
>
> 2) interested core PostGIS developers give me a @gmail.com email, so I
> add it in
>
> https://github.com/rouault/oss-fuzz/blob/postgis/
> projects/postgis/project.yaml
>
> This way they will then have access to the bug reports that are embargoed
> for
>
> 90 days (or 30 days after OSS Fuzz has found them to be fixed)
>
> 3) I then modify https://github.com/rouault/oss-fuzz/blob/postgis/
> projects/postgis/Dockerfile to
>
> point to PostGIS official github mirror instead of my fork
>
> 4) I then submit a pull request to https://github.com/google/oss-fuzz/
> with my
>
> https://github.com/rouault/oss-fuzz postgis branch . They may accept or
> not the application, but I guess they will accept.
>
> If they don't, you can also play with it locally as I showed above. And
> this is strongly recommended
>
> to do so when adding a new fuzzer for example.
>
> 5) once the project is accepted, monitor https://bugs.chromium.org/p/
> oss-fuzz/issues/list?q=postgis and fix the bugs !
>
> 6) add more fuzzers. Hint: in PostGIS "fuzzers" directory, "make
> dummyfuzzers" to check that your fuzzer builds.
>
>
>
> Note: I don't volunteer to fix all bugs that will be found. I have already
> enough to do with GDAL... I wouldn't
>
> mind if someone wants to be the declared maintainer in oss-fuzz
> projects/postgis/project.yaml and projects/postgis/Dockerfile
>
>
>
> Note 2: if you look closely at https://github.com/rouault/
> postgis/blob/ossfuzz/fuzzers/wkb_import_fuzzer.cpp
>
> you will notice that it is a bit messy since it stubs GEOS and geod_
> symbols. This is due to the fact
>
> that OSS-Fuzz requires that the fuzzer programs are completely statically
> linked, and Ubuntu doesn't ship
>
> with static builds of geos (actually just libgeos.a but no libgeos_c.a) as
> far as I can see.
>
> All this could be improved by adding a download of GEOS source code in
>
> https://github.com/rouault/oss-fuzz/blob/postgis/
> projects/postgis/Dockerfile and building it manually in
>
> https://github.com/rouault/oss-fuzz/blob/postgis/projects/postgis/build.sh
>
>
>
> Thoughts ?
>
>
>
> Even
>
>
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20170704/821a65d9/attachment.html>