[postgis-tickets] [PostGIS] #4654: Analyze regress tests with valgrind

PostGIS trac at osgeo.org
Mon Mar 30 14:14:09 PDT 2020 

#4654: Analyze regress tests with valgrind
--------------------------+---------------------------
  Reporter:  Algunenano   |      Owner:  Algunenano
      Type:  enhancement  |     Status:  new
  Priority:  high         |  Milestone:  PostGIS 3.1.0
 Component:  postgis      |    Version:  master
Resolution:               |   Keywords:
--------------------------+---------------------------
Description changed by Algunenano:

Old description:

> After https://trac.osgeo.org/postgis/ticket/4652 it seems obvious to me
> there is a need to run the regression tests, either once or ideally as
> part of the CI, under valgrind to detect issues.
>
> I've done a test and got multiple worrisome errors
>
> There are some around computing stats:
> ```
> mar 27 20:25:24 Mordor postgres[396099]: ==396099== Conditional jump or
> move depends on uninitialised value(s)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    at 0x813D16B:
> gbox_is_valid (gbox.c:221)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x80C864B:
> compute_gserialized_stats_mode (gserialized_estimate.c:1469)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x3FCD5A:
> do_analyze_rel (analyze.c:529)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x3FBC9C:
> analyze_rel (analyze.c:260)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x4BF51E:
> vacuum (vacuum.c:413)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x4BEDA6:
> ExecVacuum (vacuum.c:199)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x745296:
> standard_ProcessUtility (utility.c:670)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x744933:
> ProcessUtility (utility.c:360)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x74426A:
> PortalRunUtility (pquery.c:1175)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x7434A1:
> PortalRunMulti (pquery.c:1321)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x742ADD:
> PortalRun (pquery.c:796)
> mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x73DFD2:
> exec_simple_query (postgres.c:1215)
> ```
>
> ```
> mar 27 20:21:18 Mordor postgres[395155]: ==395155== Conditional jump or
> move depends on uninitialised value(s)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    at 0x81184B1:
> next_float_down (lwgeom_api.c:63)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x813E720:
> gbox_float_round (gbox.c:784)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x8141584:
> gserialized2_get_gbox_p (gserialized2.c:601)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x80C861B:
> compute_gserialized_stats_mode (gserialized_estimate.c:1457)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x80C6351:
> compute_gserialized_stats (gserialized_estimate.c:1871)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x3FCD5A:
> do_analyze_rel (analyze.c:529)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x3FBC9C:
> analyze_rel (analyze.c:260)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x4BF51E:
> vacuum (vacuum.c:413)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x4BEDA6:
> ExecVacuum (vacuum.c:199)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x745296:
> standard_ProcessUtility (utility.c:670)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x744933:
> ProcessUtility (utility.c:360)
> mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x74426A:
> PortalRunUtility (pquery.c:1175)
> ```
>
> This one seem like a cast:
> ```
> mar 27 20:23:26 Mordor postgres[395843]: ==395843== Conditional jump or
> move depends on uninitialised value(s)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    at 0x813D39B:
> gbox_merge (gbox.c:276)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
> lwcollection_calculate_gbox_geodetic (lwgeodetic.c:3020)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
> lwgeom_calculate_gbox_geodetic (lwgeodetic.c:3056)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81418E0:
> gserialized2_from_lwgeom (gserialized2.c:1179)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x8109068:
> geography_serialize (lwgeom_pg.c:305)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAA3E:
> gserialized_geography_from_lwgeom (geography_inout.c:115)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAB91:
> geography_in (geography_inout.c:177)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9240F5:
> InputFunctionCall (fmgr.c:1548)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9244C7:
> OidInputFunctionCall (fmgr.c:1651)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E867A:
> stringTypeDatum (parse_type.c:659)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C2459:
> coerce_type (parse_coerce.c:304)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C1E25:
> coerce_to_target_type (parse_coerce.c:103)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E4770:
> transformAssignedExpr (parse_target.c:577)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==
> mar 27 20:23:26 Mordor postgres[395843]: ==395843== Conditional jump or
> move depends on uninitialised value(s)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    at 0x813D3AC:
> gbox_merge (gbox.c:277)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
> lwcollection_calculate_gbox_geodetic (lwgeodetic.c:3020)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
> lwgeom_calculate_gbox_geodetic (lwgeodetic.c:3056)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81418E0:
> gserialized2_from_lwgeom (gserialized2.c:1179)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x8109068:
> geography_serialize (lwgeom_pg.c:305)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAA3E:
> gserialized_geography_from_lwgeom (geography_inout.c:115)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAB91:
> geography_in (geography_inout.c:177)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9240F5:
> InputFunctionCall (fmgr.c:1548)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9244C7:
> OidInputFunctionCall (fmgr.c:1651)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E867A:
> stringTypeDatum (parse_type.c:659)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C2459:
> coerce_type (parse_coerce.c:304)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C1E25:
> coerce_to_target_type (parse_coerce.c:103)
> mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E4770:
> transformAssignedExpr (parse_target.c:577)
>
> ```
>

> Right now I get quite a bit of warnings from Postgresql syscalls (mostly
> the same one repeated over and over), so I first need to filter those
> out, and then find errors and match them with the queries that triggered
> them.
>
> The tests become really slow under valgrind, so I'm not sure how to best
> approach them (probably test each file independently).

New description:

 After https://trac.osgeo.org/postgis/ticket/4652 it seems obvious to me
 there is a need to run the regression tests, either once or ideally as
 part of the CI, under valgrind to detect issues.

 I've done a test and got multiple worrisome errors

 There are some around computing stats:
 {{{
 mar 27 20:25:24 Mordor postgres[396099]: ==396099== Conditional jump or
 move depends on uninitialised value(s)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    at 0x813D16B:
 gbox_is_valid (gbox.c:221)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x80C864B:
 compute_gserialized_stats_mode (gserialized_estimate.c:1469)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x3FCD5A:
 do_analyze_rel (analyze.c:529)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x3FBC9C:
 analyze_rel (analyze.c:260)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x4BF51E: vacuum
 (vacuum.c:413)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x4BEDA6:
 ExecVacuum (vacuum.c:199)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x745296:
 standard_ProcessUtility (utility.c:670)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x744933:
 ProcessUtility (utility.c:360)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x74426A:
 PortalRunUtility (pquery.c:1175)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x7434A1:
 PortalRunMulti (pquery.c:1321)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x742ADD:
 PortalRun (pquery.c:796)
 mar 27 20:25:24 Mordor postgres[396099]: ==396099==    by 0x73DFD2:
 exec_simple_query (postgres.c:1215)
 }}}

 {{{
 mar 27 20:21:18 Mordor postgres[395155]: ==395155== Conditional jump or
 move depends on uninitialised value(s)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    at 0x81184B1:
 next_float_down (lwgeom_api.c:63)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x813E720:
 gbox_float_round (gbox.c:784)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x8141584:
 gserialized2_get_gbox_p (gserialized2.c:601)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x80C861B:
 compute_gserialized_stats_mode (gserialized_estimate.c:1457)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x80C6351:
 compute_gserialized_stats (gserialized_estimate.c:1871)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x3FCD5A:
 do_analyze_rel (analyze.c:529)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x3FBC9C:
 analyze_rel (analyze.c:260)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x4BF51E: vacuum
 (vacuum.c:413)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x4BEDA6:
 ExecVacuum (vacuum.c:199)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x745296:
 standard_ProcessUtility (utility.c:670)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x744933:
 ProcessUtility (utility.c:360)
 mar 27 20:21:18 Mordor postgres[395155]: ==395155==    by 0x74426A:
 PortalRunUtility (pquery.c:1175)
 }}}

 This one seem like a cast:
 {{{
 mar 27 20:23:26 Mordor postgres[395843]: ==395843== Conditional jump or
 move depends on uninitialised value(s)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    at 0x813D39B:
 gbox_merge (gbox.c:276)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
 lwcollection_calculate_gbox_geodetic (lwgeodetic.c:3020)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
 lwgeom_calculate_gbox_geodetic (lwgeodetic.c:3056)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81418E0:
 gserialized2_from_lwgeom (gserialized2.c:1179)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x8109068:
 geography_serialize (lwgeom_pg.c:305)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAA3E:
 gserialized_geography_from_lwgeom (geography_inout.c:115)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAB91:
 geography_in (geography_inout.c:177)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9240F5:
 InputFunctionCall (fmgr.c:1548)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9244C7:
 OidInputFunctionCall (fmgr.c:1651)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E867A:
 stringTypeDatum (parse_type.c:659)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C2459:
 coerce_type (parse_coerce.c:304)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C1E25:
 coerce_to_target_type (parse_coerce.c:103)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E4770:
 transformAssignedExpr (parse_target.c:577)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==
 mar 27 20:23:26 Mordor postgres[395843]: ==395843== Conditional jump or
 move depends on uninitialised value(s)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    at 0x813D3AC:
 gbox_merge (gbox.c:277)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
 lwcollection_calculate_gbox_geodetic (lwgeodetic.c:3020)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81470E9:
 lwgeom_calculate_gbox_geodetic (lwgeodetic.c:3056)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x81418E0:
 gserialized2_from_lwgeom (gserialized2.c:1179)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x8109068:
 geography_serialize (lwgeom_pg.c:305)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAA3E:
 gserialized_geography_from_lwgeom (geography_inout.c:115)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x80CAB91:
 geography_in (geography_inout.c:177)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9240F5:
 InputFunctionCall (fmgr.c:1548)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x9244C7:
 OidInputFunctionCall (fmgr.c:1651)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E867A:
 stringTypeDatum (parse_type.c:659)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C2459:
 coerce_type (parse_coerce.c:304)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3C1E25:
 coerce_to_target_type (parse_coerce.c:103)
 mar 27 20:23:26 Mordor postgres[395843]: ==395843==    by 0x3E4770:
 transformAssignedExpr (parse_target.c:577)

 }}}


 Right now I get quite a bit of warnings from Postgresql syscalls (mostly
 the same one repeated over and over), so I first need to filter those out,
 and then find errors and match them with the queries that triggered them.

 The tests become really slow under valgrind, so I'm not sure how to best
 approach them (probably test each file independently).

--

-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/4654#comment:2>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.

More information about the postgis-tickets mailing list