[Qgis-developer] GRASS modules

[Benjamin Ducke]

OK, I have attached my notes about trying to compile and run
QGIS 0.8.1 on Windows with Cygwin. It's messy (thanks to Windows)
to say the least.
But I managed to get everything to compile. However, trying to start
QGIS just resulted in a completely silent crash.
Maybe someone with more expertise on Windows C++ development could pick
it up from there and just get it to the point where QGIS (0.9?) starts
and runs with GRASS Cygwin.
I would then produce an up-to-date set of GRASS 6.3 binaries.

Benjamin

Tim Sutton wrote:
> Hi
> 
> I hevent tried with cygwin - but if they prove to work reliably with
> QGIS and they are official GRASS products we should certainly consider
> using them as a drop in replacement for the ones we currently have in
> QGIS-win. To my mind it would be the most ideal situation if the QGIS
> folks were not trying to maintain a separate build of GRASS binaries -
> its a waste of effort and I think a source of numerous bugs and issues
> since we are not experts in GRASS.
> 
> Tisham how will dropping in cygwin for you?
> 
> Regards
> 
> Tim
> 
> 2007/8/11, Benjamin Ducke <benjamin.ducke at ufg.uni-kiel.de>:
>> Please also keep in mind silly bug #738 which basically
>> renders GRASS on native Win32 useless for any
>> professional level work involving vector attribute
>> data.
>>
>> I have tried finding the cause of this for weeks,
>> but in vain. Guess my Win32 system API skills are
>> just not good enough.
>>
>> Btw., Cygwin versions of current GRASS 6.3.cvs run
>> just fine.
>>
>> Has anyone here ever tried to get QGIS 0.9 running
>> with Cygwin based GRASS binaries?
>>
>> Benjamin
>>
>> Paolo Cavallini wrote:
>>> Hi.
>>> Tim uploaded our new GRASS modules-thanks!
>>> Now QGISsers can do *a lot* of stuff with GRASS. Especially (but not
>>> only) for win users this is, I think, a big step forward.
>>> A couple of things remain to be fixed, however:
>>> - the very frequent crash while loading grass layers (a regression)
>>> - attribute fields are not shown in the module window
>>> I think these should be fixed before releasing 0.9.
>>> All the best.
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Qgis-developer mailing list
>>> Qgis-developer at lists.qgis.org
>>> http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-developer
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.qgis.org
>> http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-developer
>>
> 
> 
-------------- next part --------------
Installing GRASS 6.3.cvs with Cygwin on Win32

Everything works slower under Cygwin than native Win32, but the 
unix environment provided but Cygwin is very complete and robust 
so that it is easier to get things running the same as under 
Linux.

The most detailed instructions about GRASS on Cygwin are still 
here: [http://geni.ath.cx/grass.html]. Also, see this old Wiki 
page with lots of hints: [http://geni.ath.cx/wiki_cygwin_grass.html]
.

The most up-to-date instructions and Cygwin packages are here: [http://grass.itc.it/grass62/binary/mswindows/]
.

These installation instructions are for users with administrator 
privileges that can do system-wide installations of the software. 
I have not tested this for restricted user accounts.

You will likely need a good text editor that can also handle Unix 
EOL characters correctly. Download and install PSPad from [www.pspad.com]
. It will make your Windows life much easier!

  TODO

• Explore the possibility to install Cygwin with normal user 
  privileges for current user only (setting ``Just Me'' in 
  Cygwin's setup.exe options).

• User home directory is per default in c:\cygwin\<username> but 
  this can be changed! Change it to the regular Windows home 
  folder? Problem: this has spaces!

• Add SQLite support: get sqlite.exe to compile!

• Test PostgreSQL support

• Add ODBC support (possible?)!

• Motif support for GRASS?

• MySQL support?

• Freetype support for GRASS.

• Python bindings for QGIS 0.8.1 (see INSTALL file in QGIS source 
  dir).

1 Cygwin Installation<sec:Cygwin-Installation>

Cygwin DLL version 1.5.24-2

1.1 Installing the base system

Download setup.exe from [www.cygwin.com] to ``inst''.

Run ``setup.exe''

Select ``Download without Installing'' and download packages to 
directory ``inst\cygwin-packages''.

Adjust Internet Connection settings if necessary.

Choose a Download site (http://linux.rz.ruhr-uni-bochum.de).

In addition to the default selection, add these packages:

• Archive

  – unzip

  – zip

• Database

  – postgresql

  – postgresql-devel

• Devel

  – autoconf

  – automake 1.9

  – binutils

  – bison

  – byacc

  – cvs

  – doxygen

  – flex

  – gcc

  – gettext

  – gettext-devel

  – libfreetype2-devel

  – libiconv-devel

  – libncurses-devel

  – libtool1.5

  – libxml2

  – libxml2-devel

  – make

  – nasm

  – patchutils

  – pcre

  – pcre-devel

  – pkg-config

  – readline

  – subversion

  – swig

• Editors

  – mc

  – nano

  – nedit

• Graphics

  – ImageMagick

  – gd

  – ghostscript

  – ghostscript-x11

  – gnuplot

  – jpeg

  – libgd-devel

  – libgeotiff

  – libgeotiff-devel

  – libproj-devel

  – libtiff-devel

  – netpbm

  – proj

  – tiff

• Interpreters

  – python

• Libs

  – gsl: Runtime

  – gsl-devel

  – libgdbm4

  – ncurses

  – popt

  – sunrpc

• Math

  – fftw3

  – fftw3-dev

• Text

  – gv

• Utils

  – screen

• Web

  – wget

Now download the packages. If the download should be incomplete: 
it can be restarted from another source. The package selection 
will be preserved.

Now run setup.exe again to install all downloaded packages: 
Select ``Install from Local Directory''.

Next, choose the Installation Directory. A wise choice is 
C:\cygwin (default). Make sure that ``Default Text File Type'' is 
set to ``Unix/binary''.

Next, set the ``Local Package Directory'' to where the packages 
where downloaded to ``inst\cygwin-packages''. Set the ``All'' 
category to ``Install''. All downloaded packages should now be 
selected for installation. Install them! If the installer 
recommends to install additional packages (dependencies): just 
confirm.

The installation may take a few minutes as a lot of Unix software 
will be installed.

Also add C:\cygwin\bin to the Windows Environment Variable ``Path''
. This way you will be able to run the many useful Cygwin Unix 
tools from anywhere on the command line.

1.2 Updating and adding packages<sub:Updating-and-adding>

Just rerun setup.exe and select ``Download without installing''. 
You will be able to download more packages. Then rerun setup.exe 
again, this time ``Install from Local Directory'''' and make sure 
to select the newly downloaded packages. Keep all other 
installation setting (path!) the same: the new software will be 
added to your installation.

Once you are done installing Cygwin, update the binutils (linker 
and friends) if you plan to compile any software yourself. Create 
a directory binutils, change into it, then do:

cvs -z 9 -d :pserver:anoncvs at sourceware.org:/cvs/src login 

Enter "anoncvs" as the password.

cvs -z 9 -d :pserver:anoncvs at sourceware.org:/cvs/src co binutils

Compilation of binutils:

cd src

automake

./configure --prefix=/usr

make

make install

1.3 Cygwin usage hints

Nothing so far.

2 Installing GRASS and dependencies

UPDATE THE binutils PACKAGE (see [sub:Updating-and-adding])!!! 
Otherwise linking GRASS' Vlib with the GDAL lib will fail!

All required source code packages need to be downloaded to the 
Cygwin home directory of a user with admin privileges (if Cygwin 
has been setup for system-wide usage).

Windows only looks for shared libraries (DLLs) in executable file 
directories or in the same directory where the application was 
started! This is a heavy design flaw that Cygwin is unfortunately 
also subjected to. So remember to copy compiled DLLs (e.g. GDAL!) 
to /usr/bin as well!! If some program just dies w/o any error 
message, it's usually because of some DLL that's not in the path. 
You can use cygcheck.exe <program> to test for such problems.

The C compiler supplied by Cygwin is GCC version 3.4.4.

2.0.1 SQLite

TODO: does not work yet (see below), so for now no SQLite 
support!

Version 3.4.0 from [http://www.sqlite.org].

SQLite only works as a static lib under Cygwin and the 
configuration script is a bit buggy:

./configure --prefix=/usr --disable-tcl

make

Warning appears: 

libtool: link: warning: undefined symbols not allowed in 
i686-pc-cygwin shared libraries

Even though shared libraries were disabled in the configure 
script!

This leads to a failure when attempting:

make install

Error message:

It is possible to produce the sqlite3 shared lib, however:

make clean

./configure --prefix=/usr --disable-tcl

make sqlite3.dll

cp sqlite3.dll /usr/bin/

Copy DLLs to /usr/bin (see general notes at start of this 
section) as well!

We also need to copy the C development headers by hand:

cp *.h /usr/include

Now we have all we need from SQLite in order to compile GRASS and 
QGIS. Unfortunately, the command line program to manage an SQLite 
database (sqlite.exe) is missing. But might be possible to 
download and use the Windows binary for sqlite.exe from the 
SQLite homepage!

2.1 UnixODBC

Tested version 2.2.12 from u[www.unixodbc.org]:

./configure --prefix=/usr --disable-gui

make

Aborts with: 

gcc -g -O2 -DENABLE_INI_CACHING -o isql.exe isql.o 
../DriverManager/.libs/libodbc.a -ldl /usr/lib/libiconv.dll.a 
../extras/.libs/libodbce

dline -lpthread

isql.o: In function `OptimalDisplayWidth':

/home/Administrator/unixODBC-2.2.12/exe/isql.c:236: undefined 
reference to `_SQLColAttribute'

/home/Administrator/unixODBC-2.2.12/exe/isql.c:237: undefined 
reference to `_SQLColAttribute'

...

2.1.1 GEOS

Version 2.2.3

Get it from [http://geos.refractions.net/].

./configure --prefix=/usr

make

make install

2.1.2 GDAL<sub:GDAL>

Upddate the binutils PACKAGE (see [sub:Updating-and-adding])!!! 
Otherwise linking GRASS' Vlib with the GDAL lib will fail!

Version 1.4.2 (and gdal-grass 1.4.1) from [www.gdal.org].

Install GDAL w/o GRASS support. Some configure options need to be 
set to produce shared DLL libraries:

./configure --prefix=/usr --enable-shared=yes --without-libtool 
--without-python

IMPORTANT: Make sure you do not create a static version of GDAL 
by accident. If there is a libgdal.a lying around in /usr/lib 
already: make sure to delete it, otherwise, you probably not even 
be able to configure GRASS correctly!

This should enable support for PostgreSQL and GEOS but not GRASS. 
GRASS support will be enabled later using gdal-grass.

make

make install

Make a copy of libgdal.dll.1.4.2 and call it libgdal.dll.

Copy DLLs to /usr/bin (see general notes at start of this 
section)!

2.1.3 GRASS 6.3.CVS

We need a GRASS version that runs w/o X11 and Tcl/Tk so we have a 
GUI-independent core that we can connect with QGIS.

For GRASS, we use latest CVS sources (Fri, July 13th 2007), as 
these are most extensively modified to work on Windows.

Change into home dir.

touch .cvspass

export 
CVSROOT=:pserver:grass-guest at intevation.de:/home/grass/grassrepository 

cvs login

Password is 'grass'.

cvs -z3 checkout grass6

This will put the most recent sources into the directory grass6.

Subsequent updates:   

cvs up -dP

We cannot configure GRASS w/o GDAL anymore, as GDAL is now a 
mandatory lib. 

Change into grass6 source directory. 

./configure --with-readline --without-nls

(NLS enables language localizations resulting in horrible GUI 
strings.)

This should enable pretty much the full GRASS feature set 
(except: see TODO section at top of this document. Also BLAS and 
a few more things are missing that are useless, anyway).

make

This should create all enabled modules currently in CVS w/o 
compilation errors!

make install

will put everything into /usr/local/grass-6.3.cvs inside your 
Cygwin install directory.

Now we need to install the GRASS driver for GDAL. Version 1.4.1 
of gdal-grass should work fine:

./configure --with-grass=/usr/local/grass-6.3.cvs/ 
--with-gdal=/usr/bin/gdal-config --prefix=/usr

make

make install

This concludes the basic GRASS setup.

2.2 Cygwin tuning

Make a copy of c:\cygwin\usr\X11R6\bin\startxwin.bat as 
c:\cygwin\GRASS.bat.

change 

%RUN% xterm -e /usr/bin/bash -l

to

%RUN% xterm -e /usr/local/bin/grass63

The X server will keep running after GRASS was exited. However, 
this just gives an annoying error message when running GRASS.bat 
again without first manually quitting the X server.

2.3 Bugs and pitfalls

• MAJOR BUG: Resizing an X monitor sends it into an endless ``
  Redraw'' sleep state (but only if some d.* was sent to it 
  before). However, gis.m monitors work.

• The instructions assume that Cygwin is installed in C:.

• It does not seem to go well to have two different versions of 
  Cygwin running on the same machine.

• Many problems with the Tcl/Tk login GUI

  – The TclTk login screen complains about ``invalid mapsets'' if 
    the location is stored on a network drive and there are 
    permission problems (?). However, text login works fine.

  – Clicking on ``Exit'' in the TclTk login GUI still starts 
    GRASS!

  – Clicking on ``Projection Values'' to define a new Location 
    just gives an error message ``Invalid switch -''/usr''''. The 
    problem originates from this line:

  exec -- cmd.exe /c start $env(GISBASE)/etc/set_data

  in the file etc/gis_set.tcl. Apparently, the ``start'' command 
  interpretes the GRASS installation path as a command switch 
  because of the leading ``/''!

  HOTFIX: replace with

  exec -- xterm.exe -e $env(GISBASE)/etc/set_data 

  – This line is actually meant for MINGW GRASS! It wrongly 
    assumes that Cygwin GRASS works just the same, so this fix 
    cannot go into CVS. Instead, we must differentiate between 
    MINGW and Cygwin here.

  – Also, the set_data text screen checks for the path of the 
    database starting with ``/''. This makes sense on a Unix 
    system, but in the case of Cygwin it just forces uses to 
    learn about the /cygdrive/drive# convention, which is a 
    nuissance.

• Browsing documentation: not clear how to invoke the system's 
  HTML browser. For now, users can just open the local index.html 
  file with their favourite browser.

2.4 TODO

• Get NVIZ to work: 

  – nviz lives in etc/nviz2.2/nviz.exe. However, there is also an 
    nviz.bat in bin!

  – complains about not being able to find init.tcl in a (pretty 
    dumb) searchpatch. Linking /usr/share to /share helps, but 
    nviz still crashes after showing some initial GUI elements. 
    Cygcheck

  – nviz.bat complains: ``The system cannot find the path 
    specified''

• Compile with SQLite support.

• Compile with UnixODBC support.

• Compile with MySQL support.

3 QGIS with Cygwin GRASS

This is really tricky. Basically because Windows is a horribly 
designed operating system with minimal support for standards such 
as Posix and compilation of complex software is a nightmare. The 
problem is that we need to compile a mix of Cygwin libraries 
(i.e. with a Unix emulation layer that includes sockets etc.) and 
``pure'' Win32 API and C runtime. It can be done but you must pay 
attention to every detail as described below.

QGIS and Qt4 are compilable on Windows using the free MinGW. As 
opposed to Cygwin, MinGW just links binaries to the original 
Win32 runtime libraries. It does not provide a Unix emulation 
layer like Cygwin does.

3.1 Preparations

Download QGIS 0.8.1 sources from www.qgis.org. Unpack the sources 
into your Cygwin home folder. The INSTALL file has notes about 
compilation on Windows using MingW.

Download the Qt open source edition for Windows including MinGW 
from here: [http://trolltech.com/developer/downloads/qt/windows]. 
Run the installer and install the Qt software (default: 
C:\Qt\4.3.0). Select to download and install MinGW into c:\MinGW. 
This installs a version of MinGW that Qt is sure to work with.

To make your life easier, also download and install MSYS 
(1.0.11-2004.04.30-1.exe) from [www.ming.org] and install to the 
directory C:\msys. The install script will try to ``normalize'' 
your MinGW environment. Just answer ``y'' and supply the path to 
your MinGW installation (c:\MinGW). NOTE: You need to remove or 
(better) rename sh.exe (the Unix shell) in C:\msys\bin as this 
will interfere with the CMake build process later!

To complete the MinGW environment, you need the flex and bison 
utilities. THEORETICALLY, they should be availabe here: [http://sourceforge.net/project/showfiles.php?group_id=2435&package_id=82724]
. THEORETICALLY, you should just have to download the packages, 
unpack (e.g. with [www.izarc.com]) and add the contained files to 
the same folders (bin, lib, ...) in your MSYS directory (not 
MinGW!). BUT THIS WON'T WORK!

The problem is that bison will not find its macro definitions and 
always spit out a message like this: 

/msys/share/bison/m4sugar/m4sugar.m4': No such file or directory

Bison only works if you start MSYS via it's own startup script 
msys.bat. This will set the path correctly. HOWEVER, this will 
also attempt to start sh.exe and since we can't allow that (see 
above), we are f*****!

The current solution is a bit awkward: rename Cygwin's shell 
(c:\cygwin\sh.exe) to sh.org, then finish the QGIS compilation 
process, afterwars rename it to sh.exe again to get it working 
again (sorry...).

For compilation to work, we must add some folders to the Windows 
PATH environment, so linking to DLLs can be done. Edit 
C:\Qt\4.3.0\bin\qtvars.bat and add the following line to the 
sequence of ``set PATH'' statements (in this order!):

set PATH=%PATH%;C:\MinGW\bin;C:\cygwin\bin

set PATH=%PATH%;c:\cygwin\lib

set PATH=%PATH%;C:\cygwin\usr\local\grass-6.3.cvs\lib

Also add C:\Qt\4.3.0\bin to the Windows Environment Variable ``
Path''. This will make it easier to run the basic Qt tools from 
anywhere.

You can skip all the other tools mentioned in the QGIS INSTALL 
file as you already have the Cygwin versions of these available 
at this point.

Now we need Cmake, a tool to configure multi platform source 
codes that is needed by QGIS. Download from [http://www.cmake.org/files/v2.4/cmake-2.4.6-win32-x86.exe]
 and install. The installer suggests a path with spaces by 
default, so you might want to change this to something more 
simple like c:\cmake just ot be on the safe side. This is a 
special Windows version with GUI. Let the installer add it to the 
system PATH if you like.

Again, edit C:\Qt\4.3.0\bin\qtvars.bat and add the following line 
to the sequence of ``set PATH'' statements (adjust path if you 
chose to install Cmake somewhere else):

set PATH=%PATH%;"C:\Program Files\cmake\bin"

3.2 Compilation of QGIS

Start a regular Windows command line (cmd.exe). Then cd into your 
Cygwin home dir and to the dir where you have previously 
downloaded and unpacked the QGIS 0.8.1 sources.

Execute qtvars.bat to set environment variables correctly.

Configure the source: create a directory for the build inside the 
QGIS source dir and change into it. Then run CMake:

md build

cd build

cmakesetup ..

The GUI should show correct source and build dirs. Press ``
Configure'' and choose ``MinGW Makefile'' (works on Windows XP; 
use ``MSYS Makefile'' for Windows 2000).

The configuration should find everything OK except some libraries 
and include files which are in the Cygwin dir and must be 
specified manually (SELECT ``Show Advanced Values!''):

BISON_EXECUTABLE = C:/cygwin/bin/bison.exe

CMAKE_CXX_COMPILER = C:/MinGW/bin/g++.exe

CMAKE_CXX_FLAGS = '-Ic:/MinGW/include'

CMAKE_C_COMPILER = C:/MinGW/bin/gcc.exe

CMAKE_C_FLAGS = '-Ic:/MinGW/include'

CMAKE_EXE_LINKER_FLAGS = '-Lc:/MingW/lib'

CMAKE_MAKE_PROGRAM = C:/MinGW/mingw32-make.exe

CMAKE_MODULE_LINKER_FLAGS = '-Lc:/MingW/lib'

CMAKE_SHARED_LINKER_FLAGS = '-Lc:/MingW/lib'

FLEX_EXECUTABLE = C:/msys/bin/flex.exe

GDAL_INCLUDE_DIR = C:/cygwin/usr/include

GDAL_LIBRARY = C:/cygwin/lib/libgdal.dll

GEOS_INCLUDE_DIR = C:/cygwin/usr/include

GRASS_PREFIX = C:/cygwin/usr/local/grass-6.3.cvs

PROJ_INCLUDE_DIR = C:/cygwin/usr/include

SQLITE_INCLUDE_DIR = C:/cygwin/usr/include

Press ``Configure'' again. Some more dependencies will not be 
found and must be set manually:

EXPAT_INCLUDE_DIR = C:/cygwin/usr/include

GSL_INCLUDE_DIR = C:/cygwin/usr/include/gsl

GSL_PREFIX = C:/cygwin/usr

POSTGRES_INCLUDE_DIR = C:/cygwin/usr/include

Press ``Configure'' again. There should be no more missing 
critical dependencies. Make sure that:

WITH_BINDINGS = OFF (Python disabled for now)

WITH_GRASS = ON

Select ``OK''. When the configuration is done, compile and 
install with:

make

make install

Create a folder grass in the qgis install dir. Copy all files 
from the GRASS installation compiled under Cygwin (located in 
/usr/local/grass-6.3.cvs) in there. Move all files from the 
grass/lib directory into the QGIS main installation dir so 
Windows will find them.

Copy GDAL DLLs and executables from the Cygwin installation into 
the QGIS main dir:

proj.exe:

Copy Cygwin DLLs (from c:\cygwin\bin) into the QGIS main dir:

cyggeos-2.dll

cygwin1.dll

cygproj-0.dll

cygpq.dll

cygcrypt-0.dll

cygcrypt-0.9.8.dll

cygintl-3.dll

cygiconv-2.dll

cygssl-0.9.8.dll

libgdal.dll

cyggeos_c-1.dll

cyggeotiff-1.dll

cygtiff-5.dll

cygz.dll

cygjpeg-62.dll

cygjasper-1-701.dll

cygpng12.dll

cygintl-8.dll

sqlite3.dll

Also, remember to rename Cygwin's sh.org back to sh.exe so you 
will have a shell to work with!