[mapserver-users] MapScript import errors (Module not found) in Python venv with global install

[Seth G]

Glad it's working for you, and good to know it can work alongside QGIS. 
The GISInternal Python MapScripts I think are built with SWIG 2.0, whereas I compiled those wheels with the latest version of SWIG 4.0.2/master build. That may explain why the GISInternals don't work with more recent Python versions. 

Seth

--
web:https://geographika.net
twitter: @geographika


On Mon, May 2, 2022, at 3:38 PM, maëlle lapriel wrote:
> Hi Seth,
> 
> It is working fine with Python 3.6 and the corresponding GISInternals build until I try it in a venv, however it's not working on any other Python version (and matching GISInternals) regardless of venv or not. Regarding venv and env variables, it doesn't seem that anything gets wiped when activating a venv.
> 
> Interestingly, I got it working on all tested Python versions (3.6, 3.7, 3.9) by installing MapScript using your wheels instead of using the MapScript that ships with MapServer, both in venv and outside, and together with QGIS loaded. I have no idea what exactly the difference is ultimately, but at least I have a working environment now, so many thanks!
> 
> On Fri, 29 Apr 2022 at 10:15, Seth G <sethg at geographika.co.uk> wrote:
>> __
>> Hi,
>> 
>> As you wrote, the version of Python needs to match the version of Python used by GISInternals (which is different dependent on the package) e.g. https://www.gisinternals.com/packageinfo.php?file=release-1928-x64-gdal-3-4-2-mapserver-7-6-4.zip - "python - Python37-AMD64"
>> 
>> Is everything working fine until you setup a virtual environment? This could be because activating a Python virtual environment on Windows seems to wipe out any previously set environment variables in the session. Try activating the virtual environment, and then running SDKShell.bat (or simply setting the environment variables you need from that file).
>> 
>> SET PATH=C:\MapServer\bin;%PATH%
>> SET PROJ_LIB=C:\MapServer\proj7\SHARE
>> SET MAPSERVER_DLL_PATH=C:\MapServer\bin
>> 
>> I have some experimental MapScript Wheels [1] built for various versions of Python with a recent MapServer development build, which could work with a release-1930-x64 development build [2]. 
>> 
>> Seth
>> 
>> [1] https://github.com/geographika/python-mapscript/tree/main/wheels
>> [2] https://www.gisinternals.com/query.html?content=filelist&file=release-1930-x64-gdal-mapserver.zip
>> --
>> web:https://geographika.net
>> twitter: @geographika
>> 
>> 
>> On Fri, Apr 29, 2022, at 9:08 AM, maëlle lapriel wrote:
>>> Hi Seth,
>>> 
>>> I've considered this to be the problem, however I can rule out any dependency conflicts already since I also tried to simply just get MapScript running in a Python env without any other imports.
>>> Am I assuming right that for just using MapScript alone, I can just run the SDKShell.bat to have all the variables set, then take a Python version that matches the GISInternals build and I'm good to go? Because that already yields mentioned import errors, with every possible build or Python version except 3.6 and MapServer 7.6.1, which also stops working as soon as I set up a venv. 
>>> 
>>> Thank you very much for your help.
>>> 
>>> Cheers
>>> 
>>> On Thu, Apr 28, 2022, 18:25 Seth G <sethg at geographika.co.uk> wrote:
>>>> __
>>>> Hi Maëlle,
>>>> 
>>>> MapScript (and MapServer) use many of the same dependent DLLs as QGIS - GDAL, Proj, GEOS etc. which would then get loaded into memory by Python. 
>>>> For everything to work correctly, all these dependencies would have to be the same version, and (I think this is still the case), compiled with the same version of Microsoft Visual C (the 1928, 1931 etc. use when naming the GISInternals zips). 
>>>> It probably would be possible to get everything build and working, but you'd likely have to compile everything yourself. 
>>>> Is it possible to modify your workflow so a PyQGIS environment dumps out data to disk and then a MapScript environment works on this data? 
>>>> I'd imagine PyQGIS can probably do everything MapScript can, except for manipulate Mapfiles - you could take a look at mappyfile for this, which is Python only has has no dependent DLLs.
>>>> 
>>>> Seth
>>>> 
>>>> --
>>>> web:https://geographika.net
>>>> twitter: @geographika
>>>> 
>>>> 
>>>> On Thu, Apr 28, 2022, at 11:42 AM, maëlle lapriel wrote:
>>>>> I'm trying to write a PyQGIS standalone script which utilizes MapScript for certain tasks. In a first attempt, I basically did this:
>>>>> 
>>>>> - I installed MapScript globally with pip and using the Python 3.9.5 interpreter which ships with QGIS 3.22.6 (also includes wheel)
>>>>> - Created a venv with --system-site-packages
>>>>> - Downloaded the appropriate Windows binary build of MapServer from GISInternals (release-1928-x64-gdal-3-4-mapserver-7-6) and extracted into C:\MapServer\
>>>>> - Set the MAPSERVER_DLL_PATH to C:\MapServer\bin
>>>>> 
>>>>> However, whenever I try to load MapScript, it yields the following:
>>>>> 
>>>>> Traceback (most recent call last):
>>>>>   File "<string>", line 1, in <module>
>>>>>   File "C:\QGIS\apps\Python39\lib\site-packages\mapscript\__init__.py", line 2, in <module>
>>>>>     from .mapscript import *
>>>>>   File "C:\QGIS\apps\Python39\lib\site-packages\mapscript\mapscript.py", line 13, in <module>
>>>>>     from . import _mapscript
>>>>> ImportError: DLL load failed while importing _mapscript: The specified module could not be found.
>>>>> 
>>>>> As to where the README over at github indicates that this means MapServer.dll can't be found.
>>>>> 
>>>>> In order to rule out possible issues, I also tried the following:
>>>>> 
>>>>> - Not creating a venv, but using it with the QGIS python version directly
>>>>> - Installing MapScript into normal installations of python 3.7, 3.8, 3.9, 3.10
>>>>> - Using it together with the latest builds of MapServer (1930)
>>>>> - Not installing MapScript with pip but copying over mapscript.py and _mapscript.pyd from the respective MapServer builds
>>>>> - Adding C:\MapServer\bin to PATH instead of the new env variable
>>>>> 
>>>>> All with the same result. The only ever combination I got it working is when installing the MapScript globally using python3.6/pip and the old MapServer release-1911 (7.6.1), where I sadly can't import PyQGIS in the version I need.
>>>>> 
>>>>> What could be the issue here? 
>>>>> 
>>>>> Kind regards, Maëlle
>>>>> _______________________________________________
>>>>> MapServer-users mailing list
>>>>> MapServer-users at lists.osgeo.org
>>>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>>> 
>>>> 
>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20220502/be4070e9/attachment.htm>