[Gdal-dev] Re: Rendering optimization [Re: Looking for huge vector dataset]

aaime aaime at openplans.org
Fri Mar 9 04:28:37 EST 2007

* On  6-Mar-2007 at 11:55PM PST, Mateusz Loskot said:
>>> >>>> 1. load the whole geometry data into memory, into simple
>>> >>>> structures  (arrays) - this is similar to how Manifold works
>> >>> This is doomed to lead into problems once you try to open a theme
>> >>> with 5 million polygons...
> >> Yup, in many cases it is.  However, both approaches introduce some
> >> problems.  In all-in-memory approach, the obvious assumption is a
> >> user has a *lot* of RAM space, 4-8 GB.  But in turn, it provides the
> >> fastest access to features (accessing RAM is faster than disk or
> >> network).
> This is sort of a blue sky idea, but we build overviews of rasters for
> exactly this sort of purpose. I can easily imagine designing some kind
> of OGR-based tool that constructs a directory of progressively
> generalized versions of a master vector dataset according to a
> hierarchy of scales, and then offers an access API that selects the
> generalization for a scale appropriate to the task at hand. I can't
> speculate whether this sort of tool would belong in GDAL/OGR the same
> way that gdaladdo etc. does.

My 2 cents from Geotools/Geoserver land. We are using both tecniques in
fact, that is:
* our renderer subsystem gathers data from the datastores (the equivalent
   of OGR data sources) and do generalize them on the fly using a simple
   distance based algorithm before passing them to the rendering pipeline.
   This buys us some 30%-50% speedup, more if data needs to be reprojected
   because we try to generalize before reprojecting
* in Geoserver we have a demo site that uses TIGER and GNIS data sets for
   USA. Roads are stored in a postgis table where the original geometry
   is complemented by three other geometric columns, progressively more
   generalized. We use an SLD to decide which geometry to pick depending
   on the scale.
   The generalization process has been performed ad hoc, but in fact
   nothing prevents to write a tool that does it for every kind of data
   (and maybe that adds a marker column stating on which zoom level the
   features becomes visible).

Hope this helps

More information about the Gdal-dev mailing list