Large shape files and databases. WAS: [Qgis-user] Very slow performance with large SHP files, with over 200k objects

[Matt Boyd]

Hi Richard,
the postgis extensions to the Postgresql database work very well in my
experience.

If you're using windows a google search will point you at somewhere to
get the Postgresql server software from. Often it comes packaged with
PgAdminIII which includes an "application stackbuilder" which will
guide you through setting up a database with the database extensions
enabled. Just create a template first, then use that to create the
database you'll use.

Once you've the database created you can create tables etc using the
postgis manager plugin OR you can use the SPIT plugin to import your
shapefiles. My linux install insists on creating a new GID but I just
live with it.

A side note, PGAdmin will let you change table and database names
easily, I've had this break something (don't ask me what) and get odd
error messages. Moral of this is not to be tempted to create a test
database then do a bunch of work in it and decide to use it as your
main database.

This isn't quite the step by step howto but those are out there and
hopefully this points you in the direction with some questions to ask
of google.

Regards
Matt

On Mon, Mar 7, 2011 at 9:34 PM, Ramon Andinach <custard at westnet.com.au> wrote:
>
> On 07/03/2011, at 16:20 , <jr.morreale at enoreth.net> <jr.morreale at enoreth.net> wrote:
>
>> On Mon, 7 Mar 2011 16:13:32 +0800, Thiru Chandran <tiruchirapalli at gmail.com> wrote:
>>> Looks like QGIS is not able to handle very large SHP files. Here we are
>>> talking about SHP files with over 200K polygons and the one that we are over
>>> 500K objects.
>>>
>>> Even to open, it takes quite a long, say  more than 40 seconds.
>>>
>>> The issue is with search, if you do any attribute search then, its as good
>>> as 5 mins to respond.
>>>
>>> I have seen some earlier threads complaining/having the same problem.. well
>>> any patch / progress on this?
>>>
>>> this is a critical requirement in one of the application and without the
>>> performance, we really cant make it.
>>>
>>> hope to see some light on this ... can we expect this in v 1.7?
>>>
>>>
>>> Richard.
>>>
>>> Northern Territory
>>> Australia
>>
>> Even if the threading branch brings some real improvements over 1.6, you would get a better result by converting your dataset to a database such as spatialite on whihc you can set indexes and spatial idx.
>
> Right. Wondered where that sort of thing became useful.
>
> Are there some really basic step by half-step guides to setting up something like this?
> (And I don't mean here's the protocol spec, and you can figure it out from there)
>
> Also, are there comparisons out there of the different DBs that qgis can use?
>
> -ramon._______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-user
>