[postgis-users] osm2pgsql error, out of memory, though i use -s

[Frederik Ramm]

Torsten,

On 12/02/10 11:58, LF wrote:
> i've set up PostGIS and tried to import planet-101103.

I suggest that you ask on one of the OpenStreetMap lists (osm-talk or 
osm-dev) as people there will be more familiar with osm2pgsql.

> Though i use the option "-s" and i hav 4Gb RAM in my system i get an error
> "out of memory", please have a look below, the command output is at the bottom
> of this mail.

I am surprised that you get an out of memory exception; osm2pgsql should 
not use much more than 1 GB in this configuration. Have you perhaps 
configured your PostgreSQL in way that eats a lot of your memory? Can 
you give us the output of the "free" command before you run osm2pgsql?

> Please also note that i got another error during import (TopologyException),
> not sure if this is critical?

It can be ignored.

> I imported planet-101103 using this command:
>
> bzcat /local/ftp/osm2/planet-101103.osm.bz2 | osm2pgsql -S
> //tmp/osm2pgsql/osm2pgsql/default.style -d gis -m -s -
>
> Does this leave me with unusable data in PostGIS?

As you have seen, it leaves you with no data in PostGIS. That's because 
the import happens in a transaction which, when aborted, leaves no trace.

> Is there a way i can recover from this?  The actual import took 5 days.  It
> doesn't look to me like this, as there is no data in PostGIS, but i thought
> i'd better ask, if there is a chance to not again import data for another 5
> days.

You will have to start over. Some information on osm2pgsql performance 
can be found here: 
http://www.geofabrik.de/media/2010-07-10-rendering-toolchain-performance.pdf 
but the bottom line is, more memory (along with a proper setting of -C 
as has been suggested already) will definitely help.

Bye
Frederik