[pgrouting-users] Re: std::bad_alloc with USA OSM dataset

[Daniel Kastl]

Hi Ben,

It's probably the ID being to large, yes.
osm2po generates an ID that also contains some information about the grid it
belongs to, as far as I remember.

So if your total number of road links is OK and it would fir with integer,
then it's better to renumber ID's, because it will also affect calculation
speed. You can keep the ID that osm2po generated and just add a new column
and renumber also source and target column accordingly.

Daniel



2010/12/12 Ben Brehmer <ben at geooasis.com>

> As a follow up to my previous email,
>
>
> I created a new routing table containing the edges that would have been
> considered in my previous query (on the entire USA). Then ran
> assign_vertix_id() to rebuild the source/target ids, and issued the same
> query with success.
>
> This seems to indicate that the source, target and/or id fields are too
> large in my USA level routing table. This goes inline with this ticket,
> which has the same error I encountered on my previous 32 bit instance:
> http://download.osgeo.org/pgrouting/forum/pgrouting.postlbs.org/ticket/149.html
>
>
> Cheers,
> Ben
>
> On 2010-12-11, at 9:54 PM, Ben Brehmer wrote:
>
> Hello,
>
> Currently attempting to get PgRouting to work on a USA dataset. I have
> created the routing table (source, target, id, the_geom etc) with osm2po as
> all other routing converters (ie. osm2pgrouting) failed due to out of
> memory. Currently I  am attempting to run a shortest_path_astar query. The
> result is a crashed Database and the following Postgres log:
>
> *terminate called after throwing an instance of 'std::bad_alloc'*
> *  what():  St9bad_alloc*
> *LOG:  server process (PID 1170) was terminated by signal 6: Aborted*
> *LOG:  terminating any other active server processes*
> *FATAL:  the database system is in recovery mode*
> *LOG:  all server processes terminated; reinitializing*
> *LOG:  database system was interrupted; last known up at 2010-12-12
> 05:17:42 UTC*
> *LOG:  database system was not properly shut down; automatic recovery in
> progress*
> *LOG:  record with zero length at 1F/FCF370A8*
> *LOG:  redo is not required*
>
>
> I suspect my data may be the problem as it is all of USA (from Cloudmade
> OSM file). My max source/target ids are quite large:
> Max source/target id: 25,638,274
>
> I have confirmed length and valid geometries on all rows in my sample
> query. I have confirmed valid source/target/ids on all rows in test query.
>
> Without being a knowledgable C programmer, how would you suggest I go about
> debugging this? I have tried changing 'int' types in a_star.c to 'long'
> types, without luck.  Without setting up an entire C development
> environment, is there a way to add debug statements to the C code (and then
> re-compile)? Tried increasing Postgres logging to the Maximum (debug5) and
> didn't receive any more useful information.
>
>
> Any help is greatly appreciated!  For more detailed system info, please
> read below.
>
> Thanks!
>
> Ben
>
>
>
>
>
> Currently running on a 64 bit amazon (Large instance):
> *Linux version 2.6.34.7-56.40.amzn1.x86_64 (mockbuild at build-31005.build)
> (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22
> 18:48:49 UTC 2010*
>
> Also tried the Amazon 32 bit Linux instance (Small instance type) and
> received a "*std::length_error, vector::_M_fill_insert*" error:
> *Linux version 2.6.34.7-56.40.amzn1.i686 (mockbuild at build-31003.build)
> (gcc version 4.4.4 20100525 (Red Hat 4.4.4-5) (GCC) ) #1 SMP Fri Oct 22
> 18:48:33 UTC 2010*
>
> Tried both Pgrouting 1.05 and 1.04.
>
> I am running Postgres 8.4, PostGIS 1.5 as follows:
> *#select version();*
> *                                                     version
>                                          *
> *
> ------------------------------------------------------------------------------------------------------------------
> *
> * PostgreSQL 8.4.5 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC)
> 4.1.2 20080704 (Red Hat 4.1.2-48), 64-bit*
> *
> *
> *#select postgis_version();*
> *            postgis_version            *
> *---------------------------------------*
> * 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1*
>
> Sample Query:
>
> SELECT * FROM shortest_path_astar('
>                                                         SELECT
>
>  roads.osm_id as id, roads.source::integer AS source, roads.target::integer
> AS target, (CASE WHEN roads.cost =0 THEN 1 ELSE roads.cost END)::float8 AS
> cost, roads.x1, roads.y1, roads.x2, roads.y2
>                                                         FROM
>                                                                 ways roads
>                                                         WHERE
>                                                                 (the_geom
> &&
> ST_Expand(ST_SetSRID(ST_MakeBox2d(ST_Point(-73.945798873896,40.644763731167),
> ST_Point(-73.955540657038,40.649745616898)),4326), 0.01) )
>                                                          ', 42479854,
> 42497359, false, false) route ;
>
>
>
>
>
>
>
> _______________________________________________
> Pgrouting-users mailing list
> Pgrouting-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/pgrouting-users
>
>


-- 
Georepublic UG & Georepublic Japan
eMail: daniel.kastl at georepublic.de
Web: http://georepublic.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/pgrouting-users/attachments/20101213/b34d490c/attachment-0001.html