[pgrouting-users] std::bad_alloc with USA OSM dataset

[Ben Brehmer]

Hello,

Currently attempting to get PgRouting to work on a USA dataset. I have created the routing table (source, target, id, the_geom etc) with osm2po as all other routing converters (ie. osm2pgrouting) failed due to out of memory. Currently I  am attempting to run a shortest_path_astar query. The result is a crashed Database and the following Postgres log: 

terminate called after throwing an instance of 'std::bad_alloc'
  what():  St9bad_alloc
LOG:  server process (PID 1170) was terminated by signal 6: Aborted
LOG:  terminating any other active server processes
FATAL:  the database system is in recovery mode
LOG:  all server processes terminated; reinitializing
LOG:  database system was interrupted; last known up at 2010-12-12 05:17:42 UTC
LOG:  database system was not properly shut down; automatic recovery in progress
LOG:  record with zero length at 1F/FCF370A8
LOG:  redo is not required


I suspect my data may be the problem as it is all of USA (from Cloudmade OSM file). My max source/target ids are quite large:
Max source/target id: 25,638,274

I have confirmed length and valid geometries on all rows in my sample query. I have confirmed valid source/target/ids on all rows in test query. 

Without being a knowledgable C programmer, how would you suggest I go about debugging this? I have tried changing 'int' types in a_star.c to 'long' types, without luck.  Without setting up an entire C development environment, is there a way to add debug statements to the C code (and then re-compile)? Tried increasing Postgres logging to the Maximum (debug5) and didn't receive any more useful information.


Any help is greatly appreciated!  For more detailed system info, please read below.

Thanks!

Ben





Currently running on a 64 bit amazon (Large instance):
Linux version 2.6.34.7-56.40.amzn1.x86_64 (mockbuild at build-31005.build) (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22 18:48:49 UTC 2010

Also tried the Amazon 32 bit Linux instance (Small instance type) and received a "std::length_error, vector::_M_fill_insert" error:
Linux version 2.6.34.7-56.40.amzn1.i686 (mockbuild at build-31003.build) (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22 18:48:33 UTC 2010

Tried both Pgrouting 1.05 and 1.04. 

I am running Postgres 8.4, PostGIS 1.5 as follows:
#select version();
                                                     version                                                      
------------------------------------------------------------------------------------------------------------------
 PostgreSQL 8.4.5 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48), 64-bit

#select postgis_version();
            postgis_version            
---------------------------------------
 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

Sample Query:

SELECT * FROM shortest_path_astar(' 
                                                        SELECT
                                                                roads.osm_id as id, roads.source::integer AS source, roads.target::integer AS target, (CASE WHEN roads.cost =0 THEN 1 ELSE roads.cost END)::float8 AS cost, roads.x1, roads.y1, roads.x2, roads.y2   
                                                        FROM 
                                                                ways roads
                                                        WHERE 
                                                                (the_geom && ST_Expand(ST_SetSRID(ST_MakeBox2d(ST_Point(-73.945798873896,40.644763731167), ST_Point(-73.955540657038,40.649745616898)),4326), 0.01) )
                                                         ', 42479854, 42497359, false, false) route ;





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/pgrouting-users/attachments/20101211/a653cc26/attachment.html