Multiple user problem -- C# MapScript Web Application (Is
mapserver thread safe?)
Tamas Szekeres
szekerest at GMAIL.COM
Tue Aug 8 12:36:28 EDT 2006
Umberto,
I have some questions:
How the java application threading models look like? How the requests are
handled and the threads are spawned. Maybe there are some differences
between the java and the .NET approach.
How can you test against the correctness? I mean how did you make sure the
threads are correctly modifying a common variable and if not, the incorrect
usage is surely manifested as an error. As a weak example if one thread
makes a projection resulting in an error another may overwrite or clear the
error code.
Which of the mapserver components have been covered by the tests and under
what circumstances? How can you make sure that all of the code sections are
tested surely?
I think folks should be informed about how to correctly use mapscript and
which components are to be avoided. Moreover we should open a "thread-safe
components" section in the thread safety faq.
Best Regards,
Tamas
-----Original Message-----
From: Umberto Nicoletti [mailto:umberto.nicoletti at gmail.com]
Sent: Tuesday, August 08, 2006 5:42 PM
To: Tamas Szekeres
Cc: MAPSERVER-DEV at lists.umn.edu
Subject: Re: [UMN_MAPSERVER-DEV] Multiple user problem -- C# MapScript Web
Application (Is mapserver thread safe?)
Tamas,
I have a few applications built with Java mapscript (shape and raster,
so not much really) and they seem to work very well. Years ago (when
mapscript was in its infancy) I performed massive load testing and at
that time I did not experience problems at all.
More recently I have been running the thread test in mapscript/java
against both the standard test map and a map built against oracle
without any problem. Performance was not taken into account, only
correctness.
Umberto
On 8/8/06, Tamas Szekeres <szekerest at gmail.com> wrote:
> Steve,
>
> As digging into the problem increasingly i am getting more and more
> disappointed. By now i am not sure whether mapscript is safe for
> multithreading at all (even if thread support is switched on).
> For example the most essential gd library uses a number of static
> variables without using any protection when accessing them. The proj
> library also has a number of static variables an we have protection
> only against pj_init in the mapserver code. For example i am
> considering pj_transform is unsafe as well (as using static pj_errno
> for example).
>
> The problems existing in the libraries cause the same issues as if
> they were in the mapserver code directly. Each of the dll is loaded
> into the process "globally" so any static variable becomes processwide
> global variable.
> Has anyone reviewed the libraries with this regard?
>
> I have collected the potential problems of the two libraries mentioned
> before. Some of the variables might be const static not causing such
> problems, apologies for those.
>
> In the gd library:
>
> gd-2.0.33\gd_gif_in.c(12):static int verbose_set = 0;
> gd-2.0.33\gd_gif_in.c(13):static int verbose;
> gd-2.0.33\gd_gif_in.c(68):} Gif89 = { -1, -1, -1, 0 };
> gd-2.0.33\gd_gif_in.c(244): static unsigned char buf[256];
> gd-2.0.33\gd_gif_in.c(308): static unsigned char buf[280];
> gd-2.0.33\gd_gif_in.c(309): static int curbit,
> lastbit, done, last_byte;
> gd-2.0.33\gd_gif_in.c(360): static int fresh = FALSE;
> gd-2.0.33\gd_gif_in.c(362): static int code_size,
set_code_size;
> gd-2.0.33\gd_gif_in.c(363): static int max_code, max_code_size;
> gd-2.0.33\gd_gif_in.c(364): static int firstcode, oldcode;
> gd-2.0.33\gd_gif_in.c(365): static int clear_code, end_code;
> gd-2.0.33\gd_gif_in.c(366): static int table[2][(1<<
MAX_LWZ_BITS)];
> gd-2.0.33\gd_gif_in.c(367): static int stack[STACK_SIZE], *sp;
> gd-2.0.33\gd_png.c(50):static jmpbuf_wrapper gdPngJmpbufStruct;
> gd-2.0.33\gdft.c(31):static int fontConfigFlag = 0;
> gd-2.0.33\gdft.c(777):static gdCache_head_t *fontCache;
> gd-2.0.33\gdft.c(778):static FT_Library library;
> gd-2.0.33\gdkanji.c(120): static int whatcode = ASCII;
> gd-2.0.33\gdkanji.c(499): static unsigned char tmp[BUFSIZ];
> gd-2.0.33\gdkanji.c(595): static unsigned char tmp_dest[BUFSIZ];
>
> In the proj library:
>
> proj-4.4.9\src\biveval.c(7): static projUV
> proj-4.4.9\src\cs2cs.c(69):static projPJ fromProj, toProj;
> proj-4.4.9\src\cs2cs.c(71):static int
> proj-4.4.9\src\cs2cs.c(76): static char
> proj-4.4.9\src\cs2cs.c(83):static struct FACTORS facs;
> proj-4.4.9\src\cs2cs.c(84):static double (*informat)(const char *,
> proj-4.4.9\src\geod.c(15): static int
> proj-4.4.9\src\geod.c(20): static char
> proj-4.4.9\src\geod.c(129):static char *pargv[MAX_PARGS];
> proj-4.4.9\src\geod.c(130):static int pargc = 0;
> proj-4.4.9\src\geod.c(135): static int eargc = 0, c;
> proj-4.4.9\src\geod_for.c(7): static double
> proj-4.4.9\src\geod_for.c(9): static int
> proj-4.4.9\src\nad2nad.c(16): static int
> proj-4.4.9\src\nad2nad.c(19): static char
> proj-4.4.9\src\nad2nad.c(54):} input = {
> proj-4.4.9\src\nad2nad.c(58):}, output = {
> proj-4.4.9\src\pj_apply_gridshift.c(79): static int debug_count = 0;
> proj-4.4.9\src\pj_gridinfo.c(66):static int byte_order_test = 1;
> proj-4.4.9\src\pj_gridlist.c(50):static PJ_GRIDINFO *grid_list = NULL;
> proj-4.4.9\src\pj_gridlist.c(54):static int last_nadgrids_max =
0;
> proj-4.4.9\src\pj_gridlist.c(55):static int last_nadgrids_count
= 0;
> proj-4.4.9\src\pj_gridlist.c(56):static PJ_GRIDINFO **last_nadgrids_list =
NULL;
> proj-4.4.9\src\pj_gridlist.c(57):static char *last_nadgrids =
NULL;
> proj-4.4.9\src\pj_init.c(71):static paralist *start;
> proj-4.4.9\src\PJ_mod_ster.c(102): static COMPLEX /* Miller Oblated
> Stereographic */
> proj-4.4.9\src\PJ_mod_ster.c(116): static COMPLEX /* Lee Oblated
> Stereographic */
> proj-4.4.9\src\PJ_mod_ster.c(130): static COMPLEX /* 48 United States
*/
> proj-4.4.9\src\PJ_mod_ster.c(147): static COMPLEX
> proj-4.4.9\src\PJ_mod_ster.c(177): static COMPLEX
> proj-4.4.9\src\PJ_nzmg.c(48): static COMPLEX
> proj-4.4.9\src\PJ_nzmg.c(56): static double
> proj-4.4.9\src\pj_open_lib.c(51):static int path_count = 0;
> proj-4.4.9\src\pj_open_lib.c(52):static char **search_path = NULL;
> proj-4.4.9\src\pj_open_lib.c(53):static char * proj_lib_name =
> proj-4.4.9\src\PJ_robin.c(10):static struct COEFS {
> proj-4.4.9\src\pj_strerrno.c(61): static char note[50];
> proj-4.4.9\src\proj.c(26): static PJ
> proj-4.4.9\src\proj.c(28): static projUV
> proj-4.4.9\src\proj.c(30): static int
> proj-4.4.9\src\proj.c(43): static char
> proj-4.4.9\src\proj.c(49): static struct FACTORS
> proj-4.4.9\src\proj.c(51): static double
> proj-4.4.9\src\rtodms.c(13): static double
> proj-4.4.9\src\rtodms.c(17): static char
> proj-4.4.9\src\rtodms.c(19): static int
> proj-4.4.9\src\pj_datums.c(73):C_NAMESPACE struct PJ_DATUMS pj_datums[] =
{
> proj-4.4.9\src\pj_datums.c(98):C_NAMESPACE struct PJ_PRIME_MERIDIANS
> pj_prime_meridians[] = {
> proj-4.4.9\src\pj_ellps.c(8):C_NAMESPACE struct PJ_ELLPS
> proj-4.4.9\src\pj_errno.c(8):C_NAMESPACE int pj_errno = 0;
> proj-4.4.9\src\PJ_goode.c(10): C_NAMESPACE PJ
> proj-4.4.9\src\pj_units.c(10):C_NAMESPACE struct PJ_UNITS
>
>
> I wonder if mapscript can safely be used in a real multithreaded
> environment regardless of the performance problems. Are there any
> further experiences about this issue? Which architecture can be used?
> I cannot step beyond the "one process handles one request at a time"
> model.
>
>
> Best Regards,
>
> Tamas
>
>
> 2006/8/8, Steve Lime <Steve.Lime at dnr.state.mn.us>:
> > Tamas: What happened to the talk of thread specific globals that
> > was started on mapserver-dev awhile back? Now's the time to move
> > on such things...
> >
> > Steve
> >
> > >>> Tamas Szekeres <szekerest at GMAIL.COM> 8/7/2006 3:22:16 PM >>>
> > Hi,
> >
> > Many of the mapserver components use globally allocated data so
> > multiple threads within the executable (inside libmap.dll in this
> > case) share the same data.
> > Moreover the .dll is loaded into the host application (eg.
> > aspnet_wp.exe or w3wp.exe) and the global variable will be shared
> > among the threads within the overall worker process.
> > The process may host multiple appdomains so the variables is shared
> > among multiple web applications in this case. Lastly, the ASP
> > application model implies the various requests are handled by multiple
> > threads taken out of a pool of threads.
> >
> > It seems terrible compared to the architecture of a windows
> > application, isn't it?
> >
> > If mapserver is compiled with thread support the global data in some
> > of the components are protected by locks to provide synchronized
> > access by multiple threads. The access will take place sequentially so
> > the threads will wait for each
> > other. For example if you are creating mapObj from a .map file
> > multiple requests will wait for each other when parsing the file. This
> > kind of thread interaction may be slightly decreased if the protected
> > critical sections could be made smaller.
> >
> > The issue mentioned previously makes some of us frustrated.
> > Maintaining locks kills the performance of the application, it would
> > be preferable to avoid the global data at all when possible.
> > Unfortunately to achieve this level much of the code should be
> > reconsidered ( a better sort of collaboration should be made between
> > the developers ).
> >
> > I consider this problem is critical and will manifest itself more and
> > more obviously. I wonder the other mapscript languages are living
> > happily without it ;-)
> >
> > Getting back to your question this time i would prefer establishing
> > the mapping functionality in the middleware if possible. The drawing
> > request should be queued and a pool of multiple processes should be
> > spawned to handle the requests. The result may be dispatched back to
> > the client using HTTP. The MONO XSP implementation may also help to
> > establish the required functionality.
> >
> > Best Regards,
> >
> >
> > Tamas Szekeres
> >
> >
> >
> >
> > 2006/8/7, Zhonghai Wang <zhonghaiw at gmail.com>:
> > >
> > > Hi All,
> > >
> > > I've developed a windows application with the C# MapScript, and this
> > > application works well with all the common mapping functions. I am
> > very
> > > happy with the performance of MapServer. So I am now trying to
> > migrate the
> > > windows application to a web application, here I have some doubts
> > with the
> > > migration:
> > >
> > > 1. for the windows application I use a singelton class to return the
> > mapobj
> > > for all the mapping API -- I've programmed a mapping class for
> > mapping
> > > functionalities, and this works well to store all the changes of a
> > mapobj
> > > during the mapping process. But if I migrate the mapping class to
> > web
> > > application, this singelton class should not be used, since there may
> > be
> > > hundred users, who are accessing MapServer at the same time. I am
> > just
> > > wondering how to solve this problem in MapServer?
> > >
> > > In MapXtreme, this problem is solved with the use of MapXBroker,
> > which is a
> > > shared pool of MapX objects, does MapServer has any similar
> > component?
> > >
> > > 2. Thread-Safety, I know this is not a new question, but it's
> > important. In
> > > the .NET application, how to solve this problem? use thread lock? (I
> > am
> > > programming with VB.NET)
> > >
> > > thanks for any information about this topic.
> > >
> > >
> > > zhonghai
> >
>
--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.7/411 - Release Date: 2006.08.07.
--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.7/411 - Release Date: 2006.08.07.
More information about the mapserver-dev
mailing list