[postgis-tickets] a bug when table row count larger than 0 and less than 5

[zhuodao.zwj]

I have a table contain a column of geometry TYPE like this:
 
```
gds=# \d testgeom;
  Column  |   Type   | Modifiers
----------+----------+-----------
 the_geom | geometry |
```
 
The table testgeom has 3 records and I run the sql as follow:
 
```
 select the_geom from testgeom where ST_Intersects(the_geom,ST_GeomFromText( 'POLYGON((80.4748535 32.2128011,80.4858398 32.2128011,80.4858398 32.2035053,80.4748535 32.2035053,80.4748535 32.2128011))',4326))
                                                                               )
```
 
Unfortunately, it raise a core:
 
```
#0  0x00007f081fca48a5 in raise () from /lib64/libc.so.6
#1  0x00007f081fca6085 in abort () from /lib64/libc.so.6
#2  0x00000000008f7e5e in ExceptionalCondition (
    conditionName=0xa86098 "!(rel->rows > 0 || ((rel)->cheapest_total_path != ((void *)0) && (((((const Node*)(((rel)->cheapest_total_path)))->type) == T_AppendPath) && ((AppendPath *) ((rel)->cheapest_total_path))->subpaths == "..., errorType=0xa8600b "FailedAssertion", fileName=0xa86000 "allpaths.c", lineNumber=350)
    at assert.c:54
#3  0x00000000006d5930 in set_rel_size (root=0x2c6c138, rel=0x2c834c0, rti=5, rte=0x2c7bc60) at allpaths.c:350
```
I find rel->rows is not a number . I gdb it and find that the zero is the divisor in the gserialized_estimate.c:1899:

```
cell_size[d] = (max[d] - min[d]) / nd_stats->size[d]; 
```

The nd_stats->size comes from the column stanumbersN table pg_statistic as follow:
 
```
stanumbers1 | {2,0,0,0,0,80.4356,32.1458,0,0,80.5024,32.3317,0,0,7,7,7,7,1,7,7}
```

So I find function compute_gserialized_stats_mode  in the gserialized_estimate.c:1425
 
```
histo_cells_target = Min(histo_cells_target, (int)(total_rows/5));
```

When total_rows is greater than 0 and less than 5, histo_cells_target will be zero and it will cause the error.
 
But I wonder why this histo_cells_target can be zero. Is it a bug ? Can I fix it by this:
 
```
histo_cells_target = Min(histo_cells_target, Max(1, (int)(total_rows/5)));
```
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-tickets/attachments/20170324/8b9fcf6b/attachment.html>