Multiple user problem -- C# MapScript Web Application (Is
mapserver thread safe?)
Ned Harding
nharding at EXTENDTHEREACH.COM
Tue Aug 8 11:34:59 EDT 2006
My experience with these libraries is that they are not thread safe for
a large production environment. We were wasting more and more time on
open source & 3rd party libraries that were supposedly thread safe and
were not. We actually gave up on thread safety and changed our core
architecture so that we have a separate process (that stays alive)
handle each thread on a web server so that the issues go away. I would
highly recommend a web architecture like that because it is much more
stable and requires much less work debugging these threading issues and
also can deliver better performance since you don't have any blocking
between threads.
Ned.
-----Original Message-----
From: UMN MapServer Developers List [mailto:MAPSERVER-DEV at LISTS.UMN.EDU]
On Behalf Of Tamas Szekeres
Sent: Tuesday, August 08, 2006 8:14 AM
To: MAPSERVER-DEV at LISTS.UMN.EDU
Subject: [UMN_MAPSERVER-DEV] Multiple user problem -- C# MapScript Web
Application (Is mapserver thread safe?)
Steve,
As digging into the problem increasingly i am getting more and more
disappointed. By now i am not sure whether mapscript is safe for
multithreading at all (even if thread support is switched on).
For example the most essential gd library uses a number of static
variables without using any protection when accessing them. The proj
library also has a number of static variables an we have protection only
against pj_init in the mapserver code. For example i am considering
pj_transform is unsafe as well (as using static pj_errno for example).
The problems existing in the libraries cause the same issues as if they
were in the mapserver code directly. Each of the dll is loaded into the
process "globally" so any static variable becomes processwide global
variable.
Has anyone reviewed the libraries with this regard?
I have collected the potential problems of the two libraries mentioned
before. Some of the variables might be const static not causing such
problems, apologies for those.
In the gd library:
gd-2.0.33\gd_gif_in.c(12):static int verbose_set = 0;
gd-2.0.33\gd_gif_in.c(13):static int verbose;
gd-2.0.33\gd_gif_in.c(68):} Gif89 = { -1, -1, -1, 0 };
gd-2.0.33\gd_gif_in.c(244): static unsigned char buf[256];
gd-2.0.33\gd_gif_in.c(308): static unsigned char buf[280];
gd-2.0.33\gd_gif_in.c(309): static int curbit,
lastbit, done, last_byte;
gd-2.0.33\gd_gif_in.c(360): static int fresh = FALSE;
gd-2.0.33\gd_gif_in.c(362): static int code_size,
set_code_size;
gd-2.0.33\gd_gif_in.c(363): static int max_code,
max_code_size;
gd-2.0.33\gd_gif_in.c(364): static int firstcode, oldcode;
gd-2.0.33\gd_gif_in.c(365): static int clear_code, end_code;
gd-2.0.33\gd_gif_in.c(366): static int table[2][(1<<
MAX_LWZ_BITS)];
gd-2.0.33\gd_gif_in.c(367): static int stack[STACK_SIZE],
*sp;
gd-2.0.33\gd_png.c(50):static jmpbuf_wrapper gdPngJmpbufStruct;
gd-2.0.33\gdft.c(31):static int fontConfigFlag = 0;
gd-2.0.33\gdft.c(777):static gdCache_head_t *fontCache;
gd-2.0.33\gdft.c(778):static FT_Library library;
gd-2.0.33\gdkanji.c(120): static int whatcode = ASCII;
gd-2.0.33\gdkanji.c(499): static unsigned char tmp[BUFSIZ];
gd-2.0.33\gdkanji.c(595): static unsigned char tmp_dest[BUFSIZ];
In the proj library:
proj-4.4.9\src\biveval.c(7): static projUV
proj-4.4.9\src\cs2cs.c(69):static projPJ fromProj, toProj;
proj-4.4.9\src\cs2cs.c(71):static int
proj-4.4.9\src\cs2cs.c(76): static char
proj-4.4.9\src\cs2cs.c(83):static struct FACTORS facs;
proj-4.4.9\src\cs2cs.c(84):static double (*informat)(const char *,
proj-4.4.9\src\geod.c(15): static int
proj-4.4.9\src\geod.c(20): static char
proj-4.4.9\src\geod.c(129):static char *pargv[MAX_PARGS];
proj-4.4.9\src\geod.c(130):static int pargc = 0;
proj-4.4.9\src\geod.c(135): static int eargc = 0, c;
proj-4.4.9\src\geod_for.c(7): static double
proj-4.4.9\src\geod_for.c(9): static int
proj-4.4.9\src\nad2nad.c(16): static int
proj-4.4.9\src\nad2nad.c(19): static char
proj-4.4.9\src\nad2nad.c(54):} input = { proj-4.4.9\src\nad2nad.c(58):},
output = {
proj-4.4.9\src\pj_apply_gridshift.c(79): static int debug_count = 0;
proj-4.4.9\src\pj_gridinfo.c(66):static int byte_order_test = 1;
proj-4.4.9\src\pj_gridlist.c(50):static PJ_GRIDINFO *grid_list = NULL;
proj-4.4.9\src\pj_gridlist.c(54):static int last_nadgrids_max
= 0;
proj-4.4.9\src\pj_gridlist.c(55):static int
last_nadgrids_count = 0;
proj-4.4.9\src\pj_gridlist.c(56):static PJ_GRIDINFO **last_nadgrids_list
= NULL;
proj-4.4.9\src\pj_gridlist.c(57):static char *last_nadgrids =
NULL;
proj-4.4.9\src\pj_init.c(71):static paralist *start;
proj-4.4.9\src\PJ_mod_ster.c(102): static COMPLEX /* Miller Oblated
Stereographic */
proj-4.4.9\src\PJ_mod_ster.c(116): static COMPLEX /* Lee Oblated
Stereographic */
proj-4.4.9\src\PJ_mod_ster.c(130): static COMPLEX /* 48 United
States */
proj-4.4.9\src\PJ_mod_ster.c(147): static COMPLEX
proj-4.4.9\src\PJ_mod_ster.c(177): static COMPLEX
proj-4.4.9\src\PJ_nzmg.c(48): static COMPLEX
proj-4.4.9\src\PJ_nzmg.c(56): static double
proj-4.4.9\src\pj_open_lib.c(51):static int path_count = 0;
proj-4.4.9\src\pj_open_lib.c(52):static char **search_path = NULL;
proj-4.4.9\src\pj_open_lib.c(53):static char * proj_lib_name =
proj-4.4.9\src\PJ_robin.c(10):static struct COEFS {
proj-4.4.9\src\pj_strerrno.c(61): static char note[50];
proj-4.4.9\src\proj.c(26): static PJ
proj-4.4.9\src\proj.c(28): static projUV
proj-4.4.9\src\proj.c(30): static int
proj-4.4.9\src\proj.c(43): static char
proj-4.4.9\src\proj.c(49): static struct FACTORS
proj-4.4.9\src\proj.c(51): static double
proj-4.4.9\src\rtodms.c(13): static double
proj-4.4.9\src\rtodms.c(17): static char
proj-4.4.9\src\rtodms.c(19): static int
proj-4.4.9\src\pj_datums.c(73):C_NAMESPACE struct PJ_DATUMS pj_datums[]
= { proj-4.4.9\src\pj_datums.c(98):C_NAMESPACE struct PJ_PRIME_MERIDIANS
pj_prime_meridians[] = { proj-4.4.9\src\pj_ellps.c(8):C_NAMESPACE struct
PJ_ELLPS proj-4.4.9\src\pj_errno.c(8):C_NAMESPACE int pj_errno = 0;
proj-4.4.9\src\PJ_goode.c(10): C_NAMESPACE PJ
proj-4.4.9\src\pj_units.c(10):C_NAMESPACE struct PJ_UNITS
I wonder if mapscript can safely be used in a real multithreaded
environment regardless of the performance problems. Are there any
further experiences about this issue? Which architecture can be used?
I cannot step beyond the "one process handles one request at a time"
model.
Best Regards,
Tamas
2006/8/8, Steve Lime <Steve.Lime at dnr.state.mn.us>:
> Tamas: What happened to the talk of thread specific globals that was
> started on mapserver-dev awhile back? Now's the time to move on such
> things...
>
> Steve
>
> >>> Tamas Szekeres <szekerest at GMAIL.COM> 8/7/2006 3:22:16 PM >>>
> Hi,
>
> Many of the mapserver components use globally allocated data so
> multiple threads within the executable (inside libmap.dll in this
> case) share the same data.
> Moreover the .dll is loaded into the host application (eg.
> aspnet_wp.exe or w3wp.exe) and the global variable will be shared
> among the threads within the overall worker process.
> The process may host multiple appdomains so the variables is shared
> among multiple web applications in this case. Lastly, the ASP
> application model implies the various requests are handled by multiple
> threads taken out of a pool of threads.
>
> It seems terrible compared to the architecture of a windows
> application, isn't it?
>
> If mapserver is compiled with thread support the global data in some
> of the components are protected by locks to provide synchronized
> access by multiple threads. The access will take place sequentially so
> the threads will wait for each other. For example if you are creating
> mapObj from a .map file multiple requests will wait for each other
> when parsing the file. This kind of thread interaction may be slightly
> decreased if the protected critical sections could be made smaller.
>
> The issue mentioned previously makes some of us frustrated.
> Maintaining locks kills the performance of the application, it would
> be preferable to avoid the global data at all when possible.
> Unfortunately to achieve this level much of the code should be
> reconsidered ( a better sort of collaboration should be made between
> the developers ).
>
> I consider this problem is critical and will manifest itself more and
> more obviously. I wonder the other mapscript languages are living
> happily without it ;-)
>
> Getting back to your question this time i would prefer establishing
> the mapping functionality in the middleware if possible. The drawing
> request should be queued and a pool of multiple processes should be
> spawned to handle the requests. The result may be dispatched back to
> the client using HTTP. The MONO XSP implementation may also help to
> establish the required functionality.
>
> Best Regards,
>
>
> Tamas Szekeres
>
>
>
>
> 2006/8/7, Zhonghai Wang <zhonghaiw at gmail.com>:
> >
> > Hi All,
> >
> > I've developed a windows application with the C# MapScript, and this
> > application works well with all the common mapping functions. I am
> very
> > happy with the performance of MapServer. So I am now trying to
> migrate the
> > windows application to a web application, here I have some doubts
> with the
> > migration:
> >
> > 1. for the windows application I use a singelton class to return the
> mapobj
> > for all the mapping API -- I've programmed a mapping class for
> mapping
> > functionalities, and this works well to store all the changes of a
> mapobj
> > during the mapping process. But if I migrate the mapping class to
> web
> > application, this singelton class should not be used, since there
> > may
> be
> > hundred users, who are accessing MapServer at the same time. I am
> just
> > wondering how to solve this problem in MapServer?
> >
> > In MapXtreme, this problem is solved with the use of MapXBroker,
> which is a
> > shared pool of MapX objects, does MapServer has any similar
> component?
> >
> > 2. Thread-Safety, I know this is not a new question, but it's
> important. In
> > the .NET application, how to solve this problem? use thread lock? (I
> am
> > programming with VB.NET)
> >
> > thanks for any information about this topic.
> >
> >
> > zhonghai
>
More information about the mapserver-dev
mailing list