[pgrouting-users] Re: std::bad_alloc with USA OSM dataset

[Ben Brehmer]

As a follow up to my previous email, 


I created a new routing table containing the edges that would have been considered in my previous query (on the entire USA). Then ran assign_vertix_id() to rebuild the source/target ids, and issued the same query with success.  

This seems to indicate that the source, target and/or id fields are too large in my USA level routing table. This goes inline with this ticket, which has the same error I encountered on my previous 32 bit instance: http://download.osgeo.org/pgrouting/forum/pgrouting.postlbs.org/ticket/149.html


Cheers,
Ben

On 2010-12-11, at 9:54 PM, Ben Brehmer wrote:

> Hello,
> 
> Currently attempting to get PgRouting to work on a USA dataset. I have created the routing table (source, target, id, the_geom etc) with osm2po as all other routing converters (ie. osm2pgrouting) failed due to out of memory. Currently I  am attempting to run a shortest_path_astar query. The result is a crashed Database and the following Postgres log: 
> 
> terminate called after throwing an instance of 'std::bad_alloc'
>   what():  St9bad_alloc
> LOG:  server process (PID 1170) was terminated by signal 6: Aborted
> LOG:  terminating any other active server processes
> FATAL:  the database system is in recovery mode
> LOG:  all server processes terminated; reinitializing
> LOG:  database system was interrupted; last known up at 2010-12-12 05:17:42 UTC
> LOG:  database system was not properly shut down; automatic recovery in progress
> LOG:  record with zero length at 1F/FCF370A8
> LOG:  redo is not required
> 
> 
> I suspect my data may be the problem as it is all of USA (from Cloudmade OSM file). My max source/target ids are quite large:
> Max source/target id: 25,638,274
> 
> I have confirmed length and valid geometries on all rows in my sample query. I have confirmed valid source/target/ids on all rows in test query. 
> 
> Without being a knowledgable C programmer, how would you suggest I go about debugging this? I have tried changing 'int' types in a_star.c to 'long' types, without luck.  Without setting up an entire C development environment, is there a way to add debug statements to the C code (and then re-compile)? Tried increasing Postgres logging to the Maximum (debug5) and didn't receive any more useful information.
> 
> 
> Any help is greatly appreciated!  For more detailed system info, please read below.
> 
> Thanks!
> 
> Ben
> 
> 
> 
> 
> 
> Currently running on a 64 bit amazon (Large instance):
> Linux version 2.6.34.7-56.40.amzn1.x86_64 (mockbuild at build-31005.build) (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22 18:48:49 UTC 2010
> 
> Also tried the Amazon 32 bit Linux instance (Small instance type) and received a "std::length_error, vector::_M_fill_insert" error:
> Linux version 2.6.34.7-56.40.amzn1.i686 (mockbuild at build-31003.build) (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22 18:48:33 UTC 2010
> 
> Tried both Pgrouting 1.05 and 1.04. 
> 
> I am running Postgres 8.4, PostGIS 1.5 as follows:
> #select version();
>                                                      version                                                      
> ------------------------------------------------------------------------------------------------------------------
>  PostgreSQL 8.4.5 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48), 64-bit
> 
> #select postgis_version();
>             postgis_version            
> ---------------------------------------
>  1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
> 
> Sample Query:
> 
> SELECT * FROM shortest_path_astar(' 
>                                                         SELECT
>                                                                 roads.osm_id as id, roads.source::integer AS source, roads.target::integer AS target, (CASE WHEN roads.cost =0 THEN 1 ELSE roads.cost END)::float8 AS cost, roads.x1, roads.y1, roads.x2, roads.y2   
>                                                         FROM 
>                                                                 ways roads
>                                                         WHERE 
>                                                                 (the_geom && ST_Expand(ST_SetSRID(ST_MakeBox2d(ST_Point(-73.945798873896,40.644763731167), ST_Point(-73.955540657038,40.649745616898)),4326), 0.01) )
>                                                          ', 42479854, 42497359, false, false) route ;
> 
> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/pgrouting-users/attachments/20101211/e095f1c2/attachment.html