<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2658.24">
<TITLE>RE: [UMN_MAPSERVER-DEV] Proposed RFC on COLORRAMP Support (Bug 13 05)</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>Stephen Woodbridge Wrote:</FONT>
<BR><FONT SIZE=2>>I can hardly argue with your impressive capabilities, but I have to believe</FONT>
<BR><FONT SIZE=2>> that you have a fair amount of infrastructure backing up your thematic maps</FONT>
<BR><FONT SIZE=2>> that does not exist in mapserver today. I am not suggesting that we build </FONT>
<BR><FONT SIZE=2>> it all into mapserver either. </FONT>
</P>
<P><FONT SIZE=2>My worries are about just how complicated it is to get breaks right and weather that is best placed in MapServer. The problem is that you can never produce breaks with just the data you are sowing on a map, you have to look at the whole dataset. A good example of this is population density for Manhattan. If you are only looking at Manhattan you might make breaks from low to high, but looking at the whole country it is obvious that they should all be high. In order to do this well, it almost certainly has to be a pre-process.</FONT></P>
<P><FONT SIZE=2>For example, on our web site <A HREF="http://www.demographicsnow.com" TARGET="_blank">http://www.demographicsnow.com</A> (which doesn't use MapServer yet), we have over 7000 thematic variables at 12+ levels of geography. Every time we change data we have to run a preprocess on the entire data set to calculate breaks for all the variables at all the levels of geography. Currently it takes 4-5 hours.</FONT></P>
<P><FONT SIZE=2>I guess this is a sensitive spot for me, because I have seen so many maps produced with ESRI or MapInfo where people don't know how to calculate breaks so the results are highly misleading.</FONT></P>
<P><FONT SIZE=2>That said, I have no problem about putting code in MapServer to calculate breaks, but it should take into account the entire layer and not just the current view and I wonder if that would be too slow for a typical map. Personally I believe that the best way to generate breaks is to use a logarithmic StdDev and have natural breaks for -2.5, -1.5, -.5, .5, 1.5 and 2.5 * StdDev + Mean. The breaks don't have an even # of items in them, but they convey the information in a very intuitive fashion. If anyone wants help implementing such a thing, I would be glad to help with advice or even some code.</FONT></P>
<P><FONT SIZE=2>Maybe the best answer would be to create a separate API and system of helping people set up good breaks with MapServer.</FONT>
</P>
<P><FONT SIZE=2>Sorry if this comes off like a rant, I have fought this battle in other contexts before...</FONT>
</P>
<P><FONT SIZE=2>Ned.</FONT>
</P>
<P><FONT SIZE=2>-----Original Message-----</FONT>
<BR><FONT SIZE=2>From: UMN MapServer Developers List [<A HREF="mailto:MAPSERVER-DEV@LISTS.UMN.EDU">mailto:MAPSERVER-DEV@LISTS.UMN.EDU</A>] On Behalf Of Stephen Woodbridge</FONT>
<BR><FONT SIZE=2>Sent: Tuesday, September 27, 2005 12:46 PM</FONT>
<BR><FONT SIZE=2>To: MAPSERVER-DEV@LISTS.UMN.EDU</FONT>
<BR><FONT SIZE=2>Subject: Re: [UMN_MAPSERVER-DEV] Proposed RFC on COLORRAMP Support (Bug 13 05)</FONT>
</P>
<P><FONT SIZE=2>Bill Binko wrote:</FONT>
<BR><FONT SIZE=2>> On Tue, 27 Sep 2005, Ned Harding wrote:</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>>>Theming demographics is one of the most important things my company </FONT>
<BR><FONT SIZE=2>>>does with a map, but I'm not sure why the need for RAMPBREAKS. If you </FONT>
<BR><FONT SIZE=2>>>are going to calculate your own ranges externally (which we do), it </FONT>
<BR><FONT SIZE=2>>>seams like at best that is a syntax improvement on the current class </FONT>
<BR><FONT SIZE=2>>>system, but it doesn't add any new functionality. We haven't had any </FONT>
<BR><FONT SIZE=2>>>problems displaying custom breaks in the existing system, since the class let you do almost anything.</FONT>
<BR><FONT SIZE=2>>></FONT>
<BR><FONT SIZE=2>>>It seams to me the big advantage of color ramps is continuous coloring </FONT>
<BR><FONT SIZE=2>>>(which there is no way currently to do), for which the scale would be </FONT>
<BR><FONT SIZE=2>>>linear or logarithmic. Not that I have a problem with the RAMPBREAKS </FONT>
<BR><FONT SIZE=2>>>idea, but it does seams like feature creep. I'd rather see something </FONT>
<BR><FONT SIZE=2>>>get done than wait for a better solution for something we can already </FONT>
<BR><FONT SIZE=2>>>do well enough. It would be too bad if every RFC that comes up ends </FONT>
<BR><FONT SIZE=2>>>up getting expanded to the point that people are afraid to propose them because they get too big.</FONT>
<BR><FONT SIZE=2>>></FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Gentlemen,</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I agree that this feels like a bit of feature creep. I can see why it </FONT>
<BR><FONT SIZE=2>> would be useful, but I believe there are two ways to accomplish this </FONT>
<BR><FONT SIZE=2>> with only the COLORRAMP changes currently in the RFC.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> One approach would simply be to manually create several CLASSes using </FONT>
<BR><FONT SIZE=2>> the existing EXPRESSION keyword and the new COLORRAMP feature. So, if </FONT>
<BR><FONT SIZE=2>> you wanted a stop level at '10', you might have this:</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> LAYER</FONT>
<BR><FONT SIZE=2>> ....</FONT>
<BR><FONT SIZE=2>> CLASSITEM somefield</FONT>
<BR><FONT SIZE=2>> CLASS</FONT>
<BR><FONT SIZE=2>> EXPRESSION ([somefield] < 10)</FONT>
<BR><FONT SIZE=2>> COLORRANGE</FONT>
<BR><FONT SIZE=2>> MINVALUE 0</FONT>
<BR><FONT SIZE=2>> MAXVALUE 10</FONT>
<BR><FONT SIZE=2>> MINCOLOR 255 0 0 #red</FONT>
<BR><FONT SIZE=2>> MAXCOLOR 255 255 0 #yellow</FONT>
<BR><FONT SIZE=2>> RANGEITEM 'somefield'</FONT>
<BR><FONT SIZE=2>> METHOD 'linear'</FONT>
<BR><FONT SIZE=2>> END</FONT>
<BR><FONT SIZE=2>> END</FONT>
<BR><FONT SIZE=2>> CLASS</FONT>
<BR><FONT SIZE=2>> EXPRESSION ([somefield] >= 10)</FONT>
<BR><FONT SIZE=2>> COLORRANGE</FONT>
<BR><FONT SIZE=2>> MINVALUE 10</FONT>
<BR><FONT SIZE=2>> MAXVALUE 20</FONT>
<BR><FONT SIZE=2>> MINCOLOR 255 255 0 #yellow</FONT>
<BR><FONT SIZE=2>> MAXCOLOR 0 255 0 #green</FONT>
<BR><FONT SIZE=2>> RANGEITEM 'somefield'</FONT>
<BR><FONT SIZE=2>> METHOD 'linear'</FONT>
<BR><FONT SIZE=2>> END</FONT>
<BR><FONT SIZE=2>> END</FONT>
<BR><FONT SIZE=2>> ...</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Another solution would be to use Mapscript to essentially do the same </FONT>
<BR><FONT SIZE=2>> thing programatically. I am sure one of the Mapscript gurus could put </FONT>
<BR><FONT SIZE=2>> together a function (buildColorRampClasses for example) that </FONT>
<BR><FONT SIZE=2>> essentially would allow this (in PHP):</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> $theLayer = getTheLayerSomehow();</FONT>
<BR><FONT SIZE=2>> $stops = array(0 => '255 0 0', 10 => '255 255 0', 20 => '0 255 0'); </FONT>
<BR><FONT SIZE=2>> buildColorRampClasses($theLayer, $stops, 'LINEAR', 'somefield');</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I believe the Mapscript class I described in the RFC would make this </FONT>
<BR><FONT SIZE=2>> fairly simple.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> A final thought would be to create another METHOD that performed this. </FONT>
<BR><FONT SIZE=2>> While I'm not necessarily opposed to that, a part of me worries about </FONT>
<BR><FONT SIZE=2>> any method that would require additional keywords to do its work.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Stephen, I hope this addresses your concerns. If not, I'm willing to </FONT>
<BR><FONT SIZE=2>> take another stab, or we can try it in a separate RFC.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Thanks for the feedback,</FONT>
<BR><FONT SIZE=2>> Bill</FONT>
</P>
<P><FONT SIZE=2>Ned,</FONT>
</P>
<P><FONT SIZE=2>I can hardly argue with your impressive capabilities, but I have to believe that you have a fair amount of infrastructure backing up your thematic maps that does not exist in mapserver today. I am not suggesting that we build it all into mapserver either.</FONT></P>
<P><FONT SIZE=2>Bill,</FONT>
</P>
<P><FONT SIZE=2>Thank you for the suggestion.</FONT>
</P>
<P><FONT SIZE=2>As a programmer I can munge the data to fit the tools, but what I am trying to make sure that we have is the basics. And that other can use it also.</FONT></P>
<P><FONT SIZE=2>I can normalize my class values to integers, I do that today, but you loose all the information about the values for mapsever legends. For example, look at:</FONT></P>
<P><FONT SIZE=2><A HREF="http://imaptools.com/~woodbri/thematic/thematic.cgi?name=ThemeMap&cmd=Map&debug=&location=MA&method=kmeans&numclasses=10&demographic=DP40063" TARGET="_blank">http://imaptools.com/~woodbri/thematic/thematic.cgi?name=ThemeMap&cmd=Map&debug=&location=MA&method=kmeans&numclasses=10&demographic=DP40063</A></FONT></P>
<P><FONT SIZE=2>by using the breaks you have real values for your legends instead of ordinal number (0,1,2,3...). Notice I created these legends in html.</FONT></P>
<P><FONT SIZE=2>I would like to avoid using mapscript to dynamically generate some N number of classes. It should not be required to do that for thematic maps. I should be able to generate a thematic map with a simple LAYER definition using PostGIS and get good legends. I can encapsulate all or most of the logic needed in Perl or postgres functions.</FONT></P>
<P><FONT SIZE=2>Here is the mapfile for the link above:</FONT>
<BR><FONT SIZE=2><A HREF="http://imaptools.com/~woodbri/thematic/thematic.map" TARGET="_blank">http://imaptools.com/~woodbri/thematic/thematic.map</A></FONT>
</P>
<P><FONT SIZE=2>In fact if you look at the SQL you will notice a call to imt_classify() which is converting the breaks into integer values based on the breaks that are computed elsewhere so that the integers can be used to classify the map.</FONT></P>
<P><FONT SIZE=2>Summary, I guess I could work with normalizing the data and using a ramp where I can set the min and max value as I can do that today, except %name% substitution does not work in the current code. I do think it is not so good that we are loosing the real values for the legends and would like to see support for that.</FONT></P>
<P><FONT SIZE=2>-Steve W.</FONT>
</P>
</BODY>
</HTML>