Proposed RFC on COLORRAMP Support (Bug 13 05)

Stephen Woodbridge woodbri at SWOODBRIDGE.COM
Tue Sep 27 18:26:32 EDT 2005 

Ned,

I totally agree with you and in fact have already invested a lot of 
effort into being able to generate breaks. I'm am hardly at the level of 
sophistication you are at but I have read extensively on the subject and 
the problems of making pretty but misleading maps.

Ned Harding wrote:
> Stephen Woodbridge Wrote:
>  >I can hardly argue with your impressive capabilities, but I have to 
> believe
>  > that you have a fair amount of infrastructure backing up your 
> thematic maps
>  > that does not exist in mapserver today. I am not suggesting that we 
> build
>  > it all into mapserver either.
> 
> My worries are about just how complicated it is to get breaks right and 
> weather that is best placed in MapServer.  The problem is that you can 
> never produce breaks with just the data you are sowing on a map, you 
> have to look at the whole dataset.  A good example of this is population 
> density for Manhattan.  If you are only looking at Manhattan you might 
> make breaks from low to high, but looking at the whole country it is 
> obvious that they should all be high.  In order to do this well, it 
> almost certainly has to be a pre-process.

Right that is why I have not added zooming to my little demo application 
yet. The "whole dataset" is obviously a matter of context, but I 
understand you point and agree with it.

> For example, on our web site http://www.demographicsnow.com (which 
> doesn't use MapServer yet), we have over 7000 thematic variables at 12+ 
> levels of geography.  Every time we change data we have to run a 
> preprocess on the entire data set to calculate breaks for all the 
> variables at all the levels of geography.  Currently it takes 4-5 hours.
> 
> I guess this is a sensitive spot for me, because I have seen so many 
> maps produced with ESRI or MapInfo where people don't know how to 
> calculate breaks so the results are highly misleading.

It is like any tool, people can use it, abuse it or be masters at it, 
but we shouldn't avoid making tools available because someone might 
abuse it out of ignorance or otherwise.

> That said, I have no problem about putting code in MapServer to 
> calculate breaks, but it should take into account the entire layer and 
> not just the current view and I wonder if that would be too slow for a 
> typical map. 

I do have a problem with putting it in mapserver, is is not appropriate. 
But given that you have generated the breaks, it is appropriate to pass 
them to mapserver so it can display the data correctly. Hey, then you 
COULD use mapserver for DemographicsNow :)

> Personally I believe that the best way to generate breaks 
> is to use a logarithmic StdDev and have natural breaks for -2.5, -1.5, 
> -.5, .5, 1.5 and 2.5 * StdDev + Mean.  The breaks don't have an even # 
> of items in them, but they convey the information in a very intuitive 
> fashion.  If anyone wants help implementing such a thing, I would be 
> glad to help with advice or even some code.

I would be very interested in this code or the algorithm for it. Is this 
what you are using on DemographicsNow?

> Maybe the best answer would be to create a separate API and system of 
> helping people set up good breaks with MapServer.
> 
> Sorry if this comes off like a rant, I have fought this battle in other 
> contexts before...

No, I just hear someone that is passionate about something they have a 
lot of experience with.

-Steve W.

> Ned.
> 
> -----Original Message-----
> From: UMN MapServer Developers List [mailto:MAPSERVER-DEV at LISTS.UMN.EDU] 
> On Behalf Of Stephen Woodbridge
> Sent: Tuesday, September 27, 2005 12:46 PM
> To: MAPSERVER-DEV at LISTS.UMN.EDU
> Subject: Re: [UMN_MAPSERVER-DEV] Proposed RFC on COLORRAMP Support (Bug 
> 13 05)
> 
> Bill Binko wrote:
>  > On Tue, 27 Sep 2005, Ned Harding wrote:
>  >
>  >
>  >>Theming demographics is one of the most important things my company
>  >>does with a map, but I'm not sure why the need for RAMPBREAKS.  If you
>  >>are going to calculate your own ranges externally (which we do), it
>  >>seams like at best that is a syntax improvement on the current class
>  >>system, but it doesn't add any new functionality.  We haven't had any
>  >>problems displaying custom breaks in the existing system, since the 
> class let you do almost anything.
>  >>
>  >>It seams to me the big advantage of color ramps is continuous coloring
>  >>(which there is no way currently to do), for which the scale would be
>  >>linear or logarithmic.  Not that I have a problem with the RAMPBREAKS
>  >>idea, but it does seams like feature creep.  I'd rather see something
>  >>get done than wait for a better solution for something we can already
>  >>do well enough.  It would be too bad if every RFC that comes up ends
>  >>up getting expanded to the point that people are afraid to propose 
> them because they get too big.
>  >>
>  >
>  >
>  > Gentlemen,
>  >
>  > I agree that this feels like a bit of feature creep.  I can see why it
>  > would be useful, but I believe there are two ways to accomplish this
>  > with only the COLORRAMP changes currently in the RFC.
>  >
>  > One approach would simply be to manually create several CLASSes using
>  > the existing EXPRESSION keyword and the new COLORRAMP feature.  So, if
>  > you wanted a stop level at '10', you might have this:
>  >
>  > LAYER
>  >  ....
>  >  CLASSITEM somefield
>  >  CLASS
>  >     EXPRESSION ([somefield] < 10)
>  >     COLORRANGE
>  >         MINVALUE 0
>  >       MAXVALUE 10
>  >       MINCOLOR 255 0 0 #red
>  >       MAXCOLOR 255 255 0 #yellow
>  >       RANGEITEM 'somefield'
>  >       METHOD 'linear'
>  >     END
>  >   END
>  >  CLASS
>  >     EXPRESSION ([somefield] >= 10)
>  >     COLORRANGE
>  >         MINVALUE 10
>  >         MAXVALUE 20
>  >         MINCOLOR 255 255 0 #yellow
>  >       MAXCOLOR 0 255 0 #green
>  >         RANGEITEM 'somefield'
>  >         METHOD 'linear'
>  >     END
>  >   END
>  > ...
>  >
>  > Another solution would be to use Mapscript to essentially do the same
>  > thing programatically.  I am sure one of the Mapscript gurus could put
>  > together a function (buildColorRampClasses for example) that
>  > essentially would allow this (in PHP):
>  >
>  > $theLayer = getTheLayerSomehow();
>  > $stops = array(0 => '255 0 0', 10 => '255 255 0', 20 => '0 255 0');
>  > buildColorRampClasses($theLayer, $stops, 'LINEAR', 'somefield');
>  >
>  > I believe the Mapscript class I described in the RFC would make this
>  > fairly simple.
>  >
>  > A final thought would be to create another METHOD that performed this. 
>  > While I'm not necessarily opposed to that, a part of me worries about
>  > any method that would require additional keywords to do its work.
>  >
>  > Stephen, I hope this addresses your concerns.  If not, I'm willing to
>  > take another stab, or we can try it in a separate RFC.
>  >
>  > Thanks for the feedback,
>  > Bill
> 
> Ned,
> 
> I can hardly argue with your impressive capabilities, but I have to 
> believe that you have a fair amount of infrastructure backing up your 
> thematic maps that does not exist in mapserver today. I am not 
> suggesting that we build it all into mapserver either.
> 
> Bill,
> 
> Thank you for the suggestion.
> 
> As a programmer I can munge the data to fit the tools, but what I am 
> trying to make sure that we have is the basics. And that other can use 
> it also.
> 
> I can normalize my class values to integers, I do that today, but you 
> loose all the information about the values for mapsever legends. For 
> example, look at:
> 
> http://imaptools.com/~woodbri/thematic/thematic.cgi?name=ThemeMap&cmd=Map&debug=&location=MA&method=kmeans&numclasses=10&demographic=DP40063 
> <http://imaptools.com/~woodbri/thematic/thematic.cgi?name=ThemeMap&cmd=Map&debug=&location=MA&method=kmeans&numclasses=10&demographic=DP40063>
> 
> by using the breaks you have real values for your legends instead of 
> ordinal number (0,1,2,3...). Notice I created these legends in html.
> 
> I would like to avoid using mapscript to dynamically generate some N 
> number of classes. It should not be required to do that for thematic 
> maps. I should be able to generate a thematic map with a simple LAYER 
> definition using PostGIS and get good legends. I can encapsulate all or 
> most of the logic needed in Perl or postgres functions.
> 
> Here is the mapfile for the link above:
> http://imaptools.com/~woodbri/thematic/thematic.map
> 
> In fact if you look at the SQL you will notice a call to imt_classify() 
> which is converting the breaks into integer values based on the breaks 
> that are computed elsewhere so that the integers can be used to classify 
> the map.
> 
> Summary, I guess I could work with normalizing the data and using a ramp 
> where I can set the min and max value as I can do that today, except 
> %name% substitution does not work in the current code. I do think it is 
> not so good that we are loosing the real values for the legends and 
> would like to see support for that.
> 
> -Steve W.
>

More information about the mapserver-dev mailing list