[GRASS-dev] [GRASS GIS] #2643: d.vect png with icon=basic/circle and long paths segfaults

[GRASS GIS]

#2643: d.vect png with icon=basic/circle and long paths segfaults
----------------------+-----------------------------------------------------
 Reporter:  pertusus  |       Owner:  grass-dev@…              
     Type:  defect    |      Status:  new                      
 Priority:  major     |   Milestone:  7.0.1                    
Component:  Display   |     Version:  svn-releasebranch70      
 Keywords:            |    Platform:  Linux                    
      Cpu:  x86-64    |  
----------------------+-----------------------------------------------------

Comment(by glynn):

 Replying to [comment:3 pertusus]:

 > I poked around a bit in gdb, and my first impression is that the error
 is already in draw_line, it is never checked if x1 could be equal to x2,
 which means dx = 0, so at line 59, in
 {{{
             y = y1 + (x - x1) * dy / dx;

 }}}
 > y is set to nan and the segfault is a consequence of that.
 >
 > I have no idea whether x1 and X2 are supposed never to be equal or not.


 Line 59 is within the "else" clause of the condition
 {{{
     if (fabs(y1 - y2) > fabs(x1 - x2)) {
 }}}

 The only way for that condition to be false when fabs(x1-x2) is zero (i.e.
 x1==x2) is if fabs(y1-y2) is also zero (i.e. y1==y2).

 IOW, this case only occurs when x1==x2 AND y1==y2, i.e. both endpoints are
 identical.

 It should suffice to add

 {{{
     if (x1 == x2 && y1 == y2)
         return;
 }}}

 to the start of that function.

 Although it's possible that if both differences are very close to zero
 (e.g. so small as to be denormals), rounding error could result in the
 dependent variable being garbage, that shouldn't actually cause a problem,
 as either positive or negative infinity would fail the clip-rectangle test
 in store_xy(). It's only for NaN where the issue arises.

 In fact, it may also suffice to invert the logic of the clip-rectangle
 test, i.e. change
 {{{
     if (x < png.clip_left || x >= png.clip_rite || y < png.clip_top || y
 >= png.clip_bot)
         return;
 }}}
 to
 {{{
     if (!(x >= png.clip_left && x < png.clip_rite && y >= png.clip_top &&
 y < png.clip_bot))
         return;
 }}}

 The rationale is that any comparison involving NaN is always false (except
 for !=, which is always true, even for NaN!=NaN). So a NaN value should
 fail an "inside" test as well as the current "outside" test.

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/2643#comment:4>
GRASS GIS <http://grass.osgeo.org>