[Mapserver-users] generation of class colors and optimizing large shapefile access

[Charlton Purvis]

Boy I'm showing my naiveté here, but I'm going to go for broke.

I'm dealing w/ topo/bat data whose shapefile approaches 1GB in size.  I have been through all sorts of conversions using ESRI software to get it from its original format to shapefile, and I thought I'd carry it further into a TIFF.  But I'm not going to be happy w/ a TIFF.  Not at this stage of the game, since it's topo/bat data that is the core of our product.

I've been through the wiki and the archives, and I'm not convinced my lines of thinking are going the right direction.

A few things:

* ArcMap allows me to manipulate my layers as categories and assign colors and ramps to each value or value range.  No surprise there.  Thinking MapServer now . . . let's say I were completely crazy and wanted to break my shapefile into 100 classes w/ a different color per class.  Is there any way to pull the RGB values from something ArcMap-esque w/o doing it manually to populate my .map file?  It's not a simple ramp that I use.  I'm not above coding anything, but I can't imagine that this hasn't been encountered before.

* My shapefile is of NC and SC coasts.  Assuming I can handle the classes issue satisfactorily, how about performance issues?  What is the correct way to approach this problem w/ my large shapefile?  For example, the initial image is of the entire Carolinas and their coasts.  Even if it is a 640x480 image, there is no reason to have MapServer crunch the shapefile to try to render pixels that don't even appear on the screen.  And along the same vain, when I'm zoomed in to a SC street detail (streets and population to be added later), I don't need to have MapServer consumed w/ NC-related data.  I have read examples of setting the scale to keep roads from displaying until you're zoomed in to a satisfactory level, but I'm not clear on how this could apply to me since I'm dealing w/ a homogenous set of land and water at this point.  Is the vector tile index the way to go?  (http://mapserver.gis.umn.edu/cgi-bin/wiki.pl?VectorTileIndex)  And that could muck up my classes and colors from my .map file, right?  How about different shapefiles for different levels of zooming?  Sounds painful.

* Finally, on a hopefully related note, would PostGIS create a clearer picture for me?  I'm a RDBMS fellow, not GIS by birth, but it seems reasonable to assume that slapping the large datafile into a PostGIS database would lend itself to more efficient access than as a flatfile.  Why?  I'm not quite sure, since I've read about shapefile indexing on flatfiles.  But I can't imagine breaking up a shapefile that's in the database just to have it render faster to accommodate issues in my second bullet above.

I'll admit that the other half of our outfit is ESRI-centric.  I've asked them how these issues are resolved w/i their suite of applications.  I'm eager to learn.

As always, thanks for your amazingly free and intuitive advice.  I will work to return the favor(s).

Charlton

 
 
 
Charlton Purvis
(803) 777-4453 : voice
(803) 777-8833 : fax
cpurvis at sc.edu
 
Advanced Solutions Group
Department of Physics and Astronomy
University of South Carolina
Columbia, SC 29208