Contours again.

[Ed McNierney]

Bob -

Thanks for the info - here are some thoughts:

1. Is there a particular reason you need to merge all those tiles into a single shapefile?  Since you're not merging the polylines inside that shapefile, there isn't much benefit (and a bit of harm) in merging them.  If you keep each file as a separate tile, you can use a TILEINDEX to combine all the tiles into a single logical layer - as if they were a single shapefile.  You can then use shptree on each of those individual tiles to create a spatial index for it.  The combination bascially provides a two-level index that can be very efficient.

2. The contour interval has an effect, but so does the granularity of the data.  If your source data is at a higher resolution than the output image then you can often end up drawing multiple polyline segments all of which live inside a single output pixel!  This is where generalizing the data to a level appropriate to the output can help.  If your data has a much higher resolution than the output image, it will simply slow things down - 100 polylines in one pixel looks exactly the same as only 1 polyline in that same pixel.  There's no improvement in quality.

3. What is the physical image size (in pixels) and the geographic extent (in feet) of your test image?

     - Ed

----- Original Message -----
From: Bob Basques <bob.basques at CI.STPAUL.MN.US>
To: MAPSERVER-USERS at LISTS.UMN.EDU
Sent: Fri, 15 Oct 2004 13:56:30 -0400
Subject: Re: [UMN_MAPSERVER-USERS] Contours again.


> Ed McNierney wrote:
> 
> >Frank -
> >
> >Thanks for chiming in!  You're right, of course; Bob made the comment ea=
> >rlier:
> >
> >"I actually started out with something like that in the AutoCAD world.
> >Each tile (1/2 sq. mile) was brought together into a single coverage. I
> >didn't do any joins on the linework, I just let them be seperate
> >entities. I figured that was better than joining them anyway."
> >
> >from which I concluded that these were a lot of small, independent objec=
> >ts.  But I'm now not sure that's really the case, so, Bob, can you confi=
> >rm the arrangement of your data=3F
> >
> >
> Yes, the source data was all organized into 1/2 x 1/2 mile tiles (2000 x
> 2000 ft).   When assembled into a single SHP file they were NOT joined
> together.  They are still polylines though.  I'm starting to wonder now
> though about exploding them into their respective line segments.  The
> data size will double though if I do that with all the extra end points
> to take manage.
> 
> Even though the contours are chopped up by 1/2 mile increments, they are
> still quite large polylines, with hundreds of vertices each.   I could
> chop them up further, down to 500 foot grid tile perhaps.   The big
> concern for me was labelling.  I didn't want to end up with seperate
> segments that were labelled to often.   I could get away with a single
> label per object with the tiling method of aggregation.  Seperating
> every segment out seems like going backwards for some reason.  :c)
> 
> I don't recall now if I mentioned this or not, but the contours are at
> one foot increments, quite dense and 560+ megs for the *.SHP file.  The
> qix file was 4+ megs.  As a matter of fact, this is the densest dataset
> that I need to publish ( so far )  I have the DEM next that was used to
> generate the Contours.  :c)
> 
> bobb
> 
> >I ran into a similar situation a few years ago with US street map data, =
> >which was originally organized as one shapefile per layer type per state=
> >.  I was surprised at how noticeable the improvement was when I switched=
> > to one shapefile per layer type per COUNTY, as it helped prune large ob=
> >jects (interstates, rivers).
> >
> >     - Ed
> >
> >----- Original Message -----
> >From: Frank Warmerdam <warmerdam at pobox.com>
> >To: Ed McNierney <ed at TOPOZONE.COM>
> >Cc: MAPSERVER-USERS at LISTS.UMN.EDU
> >Sent: Fri, 15 Oct 2004 12:45:34 -0400
> >Subject: Re: [UMN=5FMAPSERVER-USERS] Contours again.
> >
> >
> >
> >
> >>Ed McNierney wrote:
> >>
> >>
> >>>Bob -
> >>>=20
> >>>If things got 25% faster, the index is being used - you don't need t=
> >>>
> >>>
> >o do
> >
> >
> >>anything other than create it.  Do you have a feel for what level of b=
> >>
> >>
> >andwidth
> >
> >
> >>you're getting from the NAS server=3F  Roughly what percentage of the =
> >>
> >>
> >vectors in
> >
> >
> >>the shapefile are being used to draw your test image=3F
> >>
> >>
> >>>=20
> >>>Don't mess around with different values yet - just use the default. =
> >>>
> >>>
> > It sounds
> >
> >
> >>like you're suffering from either excessively-detailed data (needs
> >>simplification for the scales you're using), a test that draws a large
> >>percentage of the vectors in the file, and/or poor throughput to the f=
> >>
> >>
> >ile
> >
> >
> >>server.  A simple test would be to copy the shapefile to the local dis=
> >>
> >>
> >k just to
> >
> >
> >>see if there's a difference.
> >>=20
> >>=20
> >>Guys,
> >>=20
> >>I haven't followed this discussion very closely, so I may have missed
> >>something.  But one issue I have encountered with contours is that the=
> >>
> >>
> >y
> >
> >
> >>often end up being very large polygons.  For instance, a "zero elevati=
> >>
> >>
> >on"
> >
> >
> >>coastal contour might well be continental sized.  Really big "rings" g=
> >>
> >>
> >et
> >
> >
> >>pushed way up the the spatial index tree and will have to be read from
> >>disk and rendered even if the current map view is just somewhere insid=
> >>
> >>
> >e
> >
> >
> >>the contour ring such that the contour won't actually be visible, but
> >>the bounding rectangles of the contour and the view area will overlap.
> >>=20
> >>If this is Bob's problem, he will really be much better off to split t=
> >>
> >>
> >he
> >
> >
> >>contours into shortish line segments instead of keeping as massive sin=
> >>
> >>
> >gle
> >
> >
> >>geometries as this would allow the spatial index to localize things
> >>much better.
> >>=20
> >>I didn't speak up about this before because it wasn't clear if this ha=
> >>
> >>
> >s
> >
> >
> >>already been done.
> >>=20
> >>Best regards,
> >>--=20
> >>---------------------------------------+------------------------------=
> >>
> >>
> >--------
> >
> >
> >>I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at po=
> >>
> >>
> >box.com
> >
> >
> >>light and sound - activate the windows | http://pobox.com/~warmerdam
> >>and watch the world go round - Rush    | Geospatial Programmer for Ren=
> >>
> >>
> >t
> >
> >
> >>=20
> >>=20
> >>
> >>
> >
> >
> >
>