map performance
boice tomlin
boice at RUNSKIP.COM
Wed Dec 6 11:57:28 PST 2006
Hi Steve,
This is great. I will pursue this method.
After generating the dbf file the first time I could then update it
only as need as users update records. PHP has DBF file functions
built in. Those single operations would take no time and the ratio
of updates to the dbf file to map views would be large.
A few questions;
why make a copy of the shp/shx files? I'm guessing your using those
to create a dbf file. That would be new for me. What tools would I
use for that? Could I use MapScript?
I'm guessing some of these questions are now starting to get out of
the scope of the original discussion. If you could just point me in
the direction of some documentation that would be good.
thanks,
boice
On Dec 6, 2006, at 2:20 PM, Steve Lime wrote:
> What if you created a shapefile like I describe ahead of time
> (could use
> MapScript to do it)? Algorithm might look like (for say counties):
>
> 1) make a copy of the base shapefile (shp/shx) files using system cp
> command (or whatever) since your not screwing with the geometries
> 2) create a new DBF table based on the contents in the base shapefile
> and add a column for each variable you want to map (number, no
> decimals)
> 3) for each variable you want to map:
> a) do your mysql query
> b) loop through the results and find the right DBF record (based on
> FIPS or whatever) and set the appropriate column value to, say,
> 1,2,3,4
> (where those are your status values)
>
> It might take a couple of minutes to build the data but it can be done
> offline.
>
> I've done this with species occurance data in Minnesota townships. For
> each township I know if a species is present or not and can create
> range
> maps very
> fast. You're doing something similar but have more states than simply
> yes/no. Same idea though.
>
> In my case the mapfile is super simple since the layer definition is
> the same for each variable, you just change the variable being
> mapped at
> runtime.
>
> Steve
>
>>>> boice tomlin <boice at RUNSKIP.COM> 12/6/2006 12:23:41 PM >>>
>
> The data comes from a MySQL database. The primary function of the
> site
> is to collect the data from users. It is a PHP/MqSQL application.
>
> The map is rendered each time for only a specific layer such as
> orthos.
>
> So I go to the database(MySQL) and grab all distinct county ids that
> have a status of complete for that layer.
>
> I build the regex string from that array of county ids and then use
> phpmapscript to set the expression for the county layer.
>
> I understand the data structure you explained below. But I don't
> understand how you would implement it. Do you propose connecting to a
> data source directly from the mapfile?
>
> At this point the only way I know how to color a county green on the
> map is by using a regular expression. But I'm open to any method.
> Especially if its faster.
>
> thanks,
>
> boice
>
>
>
>
>
>
>
> On Dec 6, 2006, at 12:39 PM, Steve Lime wrote:
> Ok, I looked back through another email and saw the output. I think
> the
> bottleneck is in the large regular expressions. You'd be much better
> off
> with a data structure like I outlined below. How do you determine, for
> example, which counties are do be drawn a certain way. That is, how do
> you build the regex? Using attributes in the shapefile? From another
> source?
>
>
> Steve
>
>
> boice tomlin <boice at RUNSKIP.COM> 12/6/2006 10:55:52 AM >>>
>
>
> Hi Steve,
>
>
> Assuming I was using static class items. Do you think my method of
> using a regular expression to specify which geographies to display is
> a
> good method?
>
>
> Thanks for the tip on saving the map file. Very cool. It looks as I
> expected or at least hoped. Basically my original map file with a few
> extra classes and regular expressions for the layers I display.
>
>
> Someone else suggested a method using postgis that would pull in shape
> data with less polygons for certain extents.
>
>
> Couldn't I use a version of the shape files I have now with less
> detail/polygons when rendering a map at larger extents?
>
>
> I actually want to explore every method I can for optimization. My
> goal is a fairly descriptive national view in 1 tenth of a second. Is
> that too lofty?
>
>
> thanks,
>
>
> boice
>
>
>
>
>
>
> On Dec 5, 2006, at 6:32 PM, Steve Lime wrote:
> Might be interesting to see what the resulting mapfile (after your
> dynamic work) looks like (use $map->save(...)).
>
>
>
>
> We'd really need to know more about the data, how you're doing
> classifications and such to comment more. There may be lots of ways
> to make things go faster. From the looks of it for each geography
> (city, county, tribe, state, nation) you have a bunch of variables
> that
> indicate
> if a data theme is available. One could organize that data like so
> (for
> example state level data):
>
>
>
>
> State Hydrography Watersheds ...
> MN 0 1
> WI 3 2
> IA 3 0
> FL 1 3
>
>
>
>
> Where 0 means no responce, 1 complete and so on. So in that case your
> class definitions would always be the same you'd
> be simply changing the variable you're mapping on (e.g. CLASSITEM),
> and
> you wouldn't need dynamic classes (or even MapScript for that
> matter). Just thinking out loud...
>
>
>
>
> Steve
>
>
>
>
> boice tomlin <boice at RUNSKIP.COM> 12/4/2006 1:24:58 PM >>>
>
>
>
>
> Hello users,
>
>
>
>
> The map located here;
>
>
>
>
> http://gisinventory.net/status_maps.html
>
>
>
>
> takes a while to load. At least at the national view where there is a
> lot of area to render.
>
>
>
>
> I am looking for alternative ways to generate the map that will
> significantly improve performance.
>
>
>
>
> Currently I am using php and looping through data and turning on
> layers
> as I go. The PHP part is lightning fast. But after I get the map
> ready
> mapserver takes several seconds to generate it.
>
>
>
>
> I'm curious about alternative ways to handle this problem and wonder
> if
> anyone had comments on any of them.
>
>
>
>
> 1) modifying the shape files in some way so the layer information is
> in
> those files so that all mapserver has to do is load those files and
> not
> depend on the map files.
> 2) using a db such as postgres with postgis so that layer information
> is available all in one compact source.
>
>
>
>
> Right now I have to generate a bunch of dynamic classes in PHP using
> the general method below.
>
>
>
>
> $lyr = $this->ramona_map->getLayerByName("state_yes");
> $cla = $lyr->getClass(0);
> $cla->setExpression("/".$expression."/");
> $lyr->set("status", MS_ON);
>
>
>
>
> I have to do this several hundred times to represent all of the data.
> The time it takes to generate the map seem proportionate to the amount
> of layers I make visible. And again this is on the mapserver side and
> not PHP. PHP does its part of the operation in thousandths of a
> second.
>
>
>
>
> anyone's thoughts are greatly appreciated.
>
>
>
>
> -boice tomlin
>
>
>
>
>
>
>
>
>
>
>
>
> ////////////////////////////
> Run Skip
> http://runskip.com/
>
>
>
>
>
>
>
>
> boice tomlin
>
>
>
>
>
>
>
>
> boice at runskip.com
>
>
>
>
>
>
>
>
> 503-528-6204
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ////////////////////////////
> Run Skip
> http://runskip.com/
>
>
>
>
> boice tomlin
>
>
>
>
> boice at runskip.com
>
>
>
>
> 503-528-6204
>
>
>
>
>
>
>
>
>
>
> ////////////////////////////
> Run Skip
> http://runskip.com/
>
>
> boice tomlin
>
>
> boice at runskip.com
>
>
> 503-528-6204
>
>
>
////////////////////////////
Run Skip
http://runskip.com/
boice tomlin
boice at runskip.com
503-528-6204
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20061206/6447b76d/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20061206/6447b76d/attachment.sig>
More information about the MapServer-users
mailing list