Parallelizing calls to msDrawLayer()
dfuhry at CS.KENT.EDU
Sun Oct 14 15:05:55 EDT 2007
Ed McNierney wrote:
> Dave -
> "If the files are located on separate disks, of course, it's game over
> in favor of the threaded model."
> Unfortunately, the world isn't always that simple. I make some pretty
> large images (up to a few hundred megabytes), and if I have to create a
> raster buffer per layer for each request, it's possible I'm going to
> start swapping (I'm serving many simultaneous requests) when I'm not
> swapping now. That's a different kind of "game over". Yes, if my
> parallelism allows my process to finish faster then I'll see a
> counteracting reduction in the number of simultaneous requests I need to
> handle. It's hard to predict which will win in a general sense.
Oh yes, completely agreed. I see how a parallel model there would
send the system to swap hell, as allocation exceeded MemTotal, and
composite time skyrocketed. I had not considered this case.
> Our map-serving systems are complex and not at all homogeneous;
> MapServer is used in a very wide variety of environments. I have found,
> from this list, that a surprising number of users are on "low end"
> hardware (e.g. data on one local IDE disk).
> I've been trying to pay attention to software optimization for a very
> long time, across a variety of applications. Some aspects are easy,
> others are not. The most dangerous thing, I have found, is to think one
> knows the answer. When that happens, one tends to stop paying attention
> and looking hard for even better answers. If you're interested in
> working on parallel layer rendering, that's great and there are
> certainly scenarios that would benefit from it. Your original post
> asked if any major drawbacks were evident, and those of us who've been
> using and optimizing MapServer for a long time have tried to offer our
> input and experience.
Very much appreciated! Apologies if I have seemed trite or
dismissive, it is not my intention.
> - Ed
More information about the mapserver-dev