<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:243609752;
mso-list-type:hybrid;
mso-list-template-ids:172630912 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Tamas,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Apologies in advance for repeating stuff below that you already know.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Assuming we have the scenario of a Windows Python programmer wants to use GDAL from Python like he uses other Python packages, and is not interested in running GDAL command-line utilities or accessing GDAL by other means, then all he needs is the GDAL Python bindings coupled with the GDAL DLLs and associated supporting files. He doesn’t really care about those DLLs and supporting files; he just wants the Python bindings to work with a minimum of steps. Ideally, he would just be able to start the GDAL installer, click Next the next button four times, accepting all the default choices like he does with any other Python package installer, watch the progress bar, and then be able to import the osgeo modules from Python when it is complete.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>In this scenario, Python is already installed. Multiple versions of it may be installed. Python supports side-by-side installation of any number and combination of versions. The default installation location is C:\PythonXY where XY are the major and minor version numbers. If the user accepted the defaults when he installed Python itself, that’s where it will be. (That somewhat violates Microsoft’s best practice of storing programs in C:\Program Files. I’m sure there’s a long thread on that somewhere in the Python mailing list archives, but I’ve never seen it.) Certain applications that embed Python may choose a different location. For example, ArcGIS 10 includes a copy of Python 2.6 and ESRI decided to store it in C:\Python26\ArcGIS10.0. I’m not sure what that was done, but it doesn’t really matter.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Python’s installer maintains a list of which versions are installed and the installation locations in the Windows Registry (a system-wide database of config info). When a package installation program runs, assuming it was built with the standard Python distutils technology I mentioned, it prompts the user with a list of installed Python instances and asks which one he wants to install he package to. The user picks one and the package files are installed to the “site-packages” directory within that Python instance. This will be, for example, C:\Python26\Lib\site-packages.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>If a Python package includes extension modules, i.e. Python modules written in C/C++ rather than Python, they are compiled by Python to .pyd files. These are actually DLLs; they just have the extension .pyd rather than .dll and must contain a certain entry point that Python can call after loading them. GDAL includes several of these, such as _gdal.pyd, _gdal_array.pyd, and so on. These modules “implicltly” link to the GDAL DLLs such as gdal17.dll. Therefore, in order for Python to successfully load _gdal.pyd when the Python program imports the osgeo.gdal package, gdal17.dll has to be locatable by Windows, along with the other modules that gdal17.dll itself depends on.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>There are several ways that gdal17.dll might be locatable. Here is what Windows does: <a href="http://msdn.microsoft.com/en-us/library/7d83bc18%28v=vs.80%29.aspx">http://msdn.microsoft.com/en-us/library/7d83bc18%28v=vs.80%29.aspx</a>. Unfortunately, none of those are optimal for GDAL’s Python bindings. Under the first option, the executable module will typically but not always be C:\PythonXY\python.exe. If an embedding application loads the Python interpreter, it will be whatever executable that program is (e.g. C:\Program Files\ArcGIS\bin\ArcMap.exe). So this is not a good choice.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>The second choice, the current directory, might be something to try. The GDAL Python bindings, e.g. gdal.py, could be modified to call a Python function to change the current directory to wherever the GDAL DLLs are, then import the _gdal.pyd, then change directory back.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>The Windows system directory (C:\Windows\system32) and Windows directory (C:\Windows) are probably not good. You guys don’t want to put all of your DLLs in there.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>The directories listed in the PATH environment variable are the typical solution. This is probably what most Python programmers do these days, based on instructions from the GDAL team. They put GDAL in C:\gdal17 or something and use the Windows Control Panel to modify the system PATH variable, so that all Windows processes will have C:\gdal17 in the PATH. But in our scenario, the Python programmer doesn’t really care about the GDAL DLLs. He’s not planning to build something in another language that links to them. He just wants his Python stuff to work with a minimum of fuss. So this is less than optimal from his point of view.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Here are some possible alternatives:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>The minimum case: build an installation package for the Python bindings but still require the user to manually store the GDAL binaries someplace and set PATH, PROJ_LIB, GDAL_DATA, etc. This will at least give a GUI installer to get the bindings installed, even if they still have to manually install GDAL.<o:p></o:p></span></p><p class=MsoListParagraph><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Build an installation package as above. Have it install the GDAL DLLs as a subdirectory of the osgeo directory, e.g. C:\PythonXY\Lib\site-packages\osgeo\bin. Modify gdal.py to set os.environ['PATH'] = os.environ['PATH'] + ';' gdalInstallDir to modify the PATH to include that directory prior to importing _gdal.pyd. The PATH will be modified for the running process only, for the duration of that process.<o:p></o:p></span></p><p class=MsoListParagraph><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><span style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Same as #2 but rather than modifying gdal.py to set the PATH variable, instead create a new Python extension module called _gdal_dll_helper.pyd. The job of this C extension module is simply to get gdal.dll and other DLLs loaded without resorting to modifying the system PATH which can sometimes have weird consequences (I can explain more if needed). The extension module would call the Windows <a href="http://msdn.microsoft.com/en-us/library/ms686203%28v=vs.85%29.aspx">SetDllDirectory</a> function, call LoadLibrary to explicitly load gdal17.dll into the current process, then call SetDllDirectory again to set the DLL directory back to what it was previously. Then, when gdal.py wants to load _gdal.pyd, gdal17.dll is already loaded and the binding succeeds.<o:p></o:p></span></p><p class=MsoListParagraph><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><span style='mso-list:Ignore'>4.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Statically link all necessary code to _gdal.pyd, _gdal_array.pyd, etc. This would eliminate the need to store the GDAL DLLs on the machine at all, as all code would be in the bindings .pyd DLLs. This would likely be a big job, and probably not even possible given the variety of libraries that GDAL leverages.<o:p></o:p></span></p><p class=MsoListParagraph><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>I know #2 and #3 sound scary but they can be done cleanly. I currently use a variation of #3 in my own project that embeds GDAL and its Python bindings.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>You also asked:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal>The individual packages may specify the required minimum version of the referred packages loaded by the .NET runtime. Is this something that can be done with the python environment as well?<span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>As far as I know, there is not really a clean, standardized way to do that. There are some innovations related to the “easy_install” thing I mentioned earlier, but nothing has become standard in Python itself, at least with the 2.x releases.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'>Jason<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Tamas Szekeres [mailto:szekerest@gmail.com] <br><b>Sent:</b> Thursday, January 06, 2011 7:24 AM<br><b>To:</b> Jason Roberts<br><b>Cc:</b> gdal-dev@lists.osgeo.org; Christopher Barker<br><b>Subject:</b> Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'>Jason,<br><br>I appreciate the expertise for all of you along with this thread, I could already gather quite some useful information from here for this reason. I must mention that my programming practice in Python can be considered as zero, this is the main reason that my issues may have trivial solutions for the hardcore pythonists but not trivial to me. Apologies for this inconvenience :-)<br><br>Getting back to the original topic, you mention that the gdal binaries should be installed somewhere an set PATH, GDAL_DATA, PROJ_LIB and GDAL_DRIVER_PATH as a systemwide setting. This is where the problems of mine are starting. Modifying the PATH globally is a bad practice in 99% of the cases. The only case I'm aware of which may not be a problem when we make sure that only one version of such files (dll-s and executables) will ever be installed to a particular system. But this is not the case with the gdal binaries as I would expect at least a development or a stable version (and their x86/x64 variants) to coexist which should be used by the same user. The same problem may arise when we would like to install multiple versions to the site packages directory, how the versions of the files are maintained by the python runtime? In this regard I could mention something like what have been done with the .NET framework with the multiple versions of the packages installed simultaneously in the global assembly cache. The individual packages may specify the required minimum version of the referred packages loaded by the .NET runtime. Is this something that can be done with the python environment as well?<br><br>(As opposed to this, the dumb solution of having a starting script to open a command prompt (and setting PYTHONPATH properly) would ensure multiple versions to be used at the same time, since those settings are applyed to the cmd process solely.)<br><br>Best regards,<br><br>Tamas<br><br><o:p></o:p></p></div></body></html>