[GRASS-dev] [GRASS GIS] #2036: Failed watershed analysis on Grass

[GRASS GIS]

#2036: Failed watershed analysis on Grass
--------------------------+-------------------------------------------------
  Reporter:  mehmeto      |       Owner:  grass-dev@…              
      Type:  defect       |      Status:  closed                   
  Priority:  normal       |   Milestone:  6.4.4                    
 Component:  Raster       |     Version:  6.4.2                    
Resolution:  fixed        |    Keywords:  LFS, r.watershed         
  Platform:  MSWindows 7  |         Cpu:  x86-64                   
--------------------------+-------------------------------------------------

Comment(by mmetz):

 Replying to [comment:20 hamish]:
 >
 > mmetz:
 > > It's not so easy for Windows. You need to use the LFS API explicitely,
 i.e.
 > > off64_t, fseeko64(), ftello64, lseek64(), _stati64, etc.
 >
 > so more #ifdefs are needed in G_ftell() and G_fseek(), then more modules
 in g6 need to use those functions? (right now only r.in.bin does) If so it
 doesn't seem so hard.

 In g6, G_ftell() and G_fseek() do not have LFS on Windows because they use
 fseeko and ftello, not fseeko64 and ftello64. Besides, it does not make
 sense to enable LFS on module level if the libraries do not have LFS
 (which they do not have on wingrass 6). This is why LFS is globally
 enabled/disabled in g7 by configure which in turn creates config.h and
 Platform.make. Libraries and modules do no longer need any additional
 switches.

 > As a future maintenance goal, full 64bit file support on Windows for
 6.4.x seems to me a rather important thing to work towards. (From both the
 grass code, msys, and osgeo4w ends)

 Considering that ticket #1131 (Global LFS for wingrass) was opened 3 years
 ago and closed 10 days ago, which triggered you to ask

 "it depends: is LFS working in MS Windows? i.e. has it been tested with
 the latest trunk code and passed? We shouldn't close tickets on
 assumptions."

 I suggest to 1) release the GRASS GIS 7 tech-preview asap, 2) leave
 wingrass LFS for g7. LFS is not so easy for Windows.

 Interesting that for a change you ask for low-level modification of g
 6.4.x which I regard as too invasive.
 >
 >
 > > For trunk, the -m flag could become the default, in order to avoid
 tickets
 > > like this.
 >
 > mmph, I'm not a fan of that, rather just document the memory issues in
 the man page and have it go fast in the typical cases.

 I prefer modules to finish successfully, even if it takes some time,
 instead of first freezing the machine by going into swap, then failing
 with an out-of-memory error.

 > (how much RAM will the average computer have when g7 is middle aged?)

 Replacing "g7" with "currently cutting edge software under development",
 this is an easy question. The answer has been the same for the last
 decades and will be the same for the foreseeable future: not enough, even
 for your average HPC. This is the reason for the design of the GRASS
 raster library since its inception and the reason for the existence of the
 segment and rowio libraries.

 I assume that the data available will continue to prevent processing all
 in RAM. For example, the SRTM data have been collected in February 2000,
 and as of today only few hardware + software combos exist that can process
 the whole SRTM dataset.

 I guess the reason why an all-in-RAM r.watershed version exists at all is
 that the all-in-RAM version was really slow and the disk-swap version was
 even slower. This has been changed, the current disk-swap version is
 magnitudes faster than the pre g-6.4 all-in-RAM version.

 Anyway, as long as wingrass is compiled as a 32 bit application, it can
 only access the amount of RAM available to a 32 bit application with
 correspondingly sized pointers. Another reason to use the low-memory
 module versions by default and release the g7 tech preview asap.

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/2036#comment:21>
GRASS GIS <http://grass.osgeo.org>