Contours again.

Fri Oct 15 15:15:17 EDT 2004

Ed McNierney wrote:

>Bob -
>
>Thanks for the info - here are some thoughts:
>
>1. Is there a particular reason you need to merge all those tiles into a=
> single shapefile=3F  Since you're not merging the polylines inside that=
> shapefile, there isn't much benefit (and a bit of harm) in merging them=
>.  If you keep each file as a separate tile, you can use a TILEINDEX to =
>combine all the tiles into a single logical layer - as if they were a si=
>ngle shapefile.  You can then use shptree on each of those individual ti=
>les to create a spatial index for it.  The combination bascially provide=
>s a two-level index that can be very efficient.
>
>
Actaully that was my first thought.  I do this already with Aerial Photo
layers.  This was really an experiment to guage performance.  It's still
impressive (to me) that the request comes back as quick as it does,
considering the data amounts.

>2. The contour interval has an effect, but so does the granularity of th=
>e data.  If your source data is at a higher resolution than the output i=
>mage then you can often end up drawing multiple polyline segments all of=
> which live inside a single output pixel!
>
That's not an issue at the test viewings.  I'm also planning on
generating multiple resolutions of the contours data for just this
reason.  Doing a Contour map of the whole City for example.

> This is where generalizing th=
>e data to a level appropriate to the output can help.
>
Understood.  I do this now for the aerial photo's too. where I have
multiple tile sets for the same LAYER at different source resaolutions
that are used for different viewing scales.   I've not tried this same
technique with the SHP file though.  Are there any pitfalls to look out for?

> If your data has =
>a much higher resolution than the output image, it will simply slow thin=
>gs down - 100 polylines in one pixel looks exactly the same as only 1 po=
>lyline in that same pixel.  There's no improvement in quality.
>
>3. What is the physical image size (in pixels) and the geographic extent=
> (in feet) of your test image=3F
>
>
1600 x 1300 feet (extents) and 800 x 650 pixels.   Right in the correct
neighborhood for the source data.  I'm also labelling the lines, but I
don't think that's a significant overhead overall.

The biggest issue seems to be the fact that the Layer resided on a
network mounted device.  (This is the same for the Aerial photo datasets
as well.)  The smaller I can make the initial loading the better I
think.  So the TileIndex seems to be the way to go, in my case anyways.

bobb

>     - Ed
>
>----- Original Message -----
>From: Bob Basques <bob.basques at CI.STPAUL.MN.US>
>To: MAPSERVER-USERS at LISTS.UMN.EDU
>Sent: Fri, 15 Oct 2004 13:56:30 -0400
>Subject: Re: [UMN=5FMAPSERVER-USERS] Contours again.
>
>
>
>
>>Ed McNierney wrote:
>>=20
>>
>>
>>>Frank -
>>>
>>>Thanks for chiming in!  You're right, of course; Bob made the comment=
>>>
>>>
> ea=3D
>
>
>>>rlier:
>>>
>>>"I actually started out with something like that in the AutoCAD world=
>>>
>>>
>.
>
>
>>>Each tile (1/2 sq. mile) was brought together into a single coverage.=
>>>
>>>
> I
>
>
>>>didn't do any joins on the linework, I just let them be seperate
>>>entities. I figured that was better than joining them anyway."
>>>
>>>
>>>
>>>from which I concluded that these were a lot of small, independent ob=
>>
>>
>jec=3D
>
>
>>>ts.  But I'm now not sure that's really the case, so, Bob, can you co=
>>>
>>>
>nfi=3D
>
>
>>>rm the arrangement of your data=3D3F
>>>
>>>
>>>
>>>
>>Yes, the source data was all organized into 1/2 x 1/2 mile tiles (2000=
>>
>>
> x
>
>
>>2000 ft).   When assembled into a single SHP file they were NOT joined
>>together.  They are still polylines though.  I'm starting to wonder no=
>>
>>
>w
>
>
>>though about exploding them into their respective line segments.  The
>>data size will double though if I do that with all the extra end point=
>>
>>
>s
>
>
>>to take manage.
>>=20
>>Even though the contours are chopped up by 1/2 mile increments, they a=
>>
>>
>re
>
>
>>still quite large polylines, with hundreds of vertices each.   I could
>>chop them up further, down to 500 foot grid tile perhaps.   The big
>>concern for me was labelling.  I didn't want to end up with seperate
>>segments that were labelled to often.   I could get away with a single
>>label per object with the tiling method of aggregation.  Seperating
>>every segment out seems like going backwards for some reason.  :c)
>>=20
>>I don't recall now if I mentioned this or not, but the contours are at
>>one foot increments, quite dense and 560+ megs for the *.SHP file.  Th=
>>
>>
>e
>
>
>>qix file was 4+ megs.  As a matter of fact, this is the densest datase=
>>
>>
>t
>
>
>>that I need to publish ( so far )  I have the DEM next that was used t=
>>
>>
>o
>
>
>>generate the Contours.  :c)
>>=20
>>bobb
>>=20
>>
>>
>>>I ran into a similar situation a few years ago with US street map dat=
>>>
>>>
>a, =3D
>
>
>>>which was originally organized as one shapefile per layer type per st=
>>>
>>>
>ate=3D
>
>
>>>.  I was surprised at how noticeable the improvement was when I switc=
>>>
>>>
>hed=3D
>
>
>>>to one shapefile per layer type per COUNTY, as it helped prune large=
>>>
>>>
> ob=3D
>
>
>>>jects (interstates, rivers).
>>>
>>>    - Ed
>>>
>>>----- Original Message -----
>>>From: Frank Warmerdam <warmerdam at pobox.com>
>>>To: Ed McNierney <ed at TOPOZONE.COM>
>>>Cc: MAPSERVER-USERS at LISTS.UMN.EDU
>>>Sent: Fri, 15 Oct 2004 12:45:34 -0400
>>>Subject: Re: [UMN=3D5FMAPSERVER-USERS] Contours again.
>>>
>>>
>>>
>>>
>>>
>>>
>>>>Ed McNierney wrote:
>>>>
>>>>
>>>>
>>>>
>>>>>Bob -
>>>>>=3D20
>>>>>If things got 25% faster, the index is being used - you don't need =
>>>>>
>>>>>
>t=3D
>
>
>>>>>
>>>>>
>>>o do
>>>
>>>
>>>
>>>
>>>>anything other than create it.  Do you have a feel for what level of=
>>>>
>>>>
> b=3D
>
>
>>>>
>>>>
>>>andwidth
>>>
>>>
>>>
>>>
>>>>you're getting from the NAS server=3D3F  Roughly what percentage of =
>>>>
>>>>
>the =3D
>
>
>>>>
>>>>
>>>vectors in
>>>
>>>
>>>
>>>
>>>>the shapefile are being used to draw your test image=3D3F
>>>>
>>>>
>>>>
>>>>
>>>>>=3D20
>>>>>Don't mess around with different values yet - just use the default.=
>>>>>
>>>>>
> =3D
>
>
>>>>>
>>>>>
>>>It sounds
>>>
>>>
>>>
>>>
>>>>like you're suffering from either excessively-detailed data (needs
>>>>simplification for the scales you're using), a test that draws a lar=
>>>>
>>>>
>ge
>
>
>>>>percentage of the vectors in the file, and/or poor throughput to the=
>>>>
>>>>
> f=3D
>
>
>>>>
>>>>
>>>ile
>>>
>>>
>>>
>>>
>>>>server.  A simple test would be to copy the shapefile to the local d=
>>>>
>>>>
>is=3D
>
>
>>>>
>>>>
>>>k just to
>>>
>>>
>>>
>>>
>>>>see if there's a difference.
>>>>=3D20
>>>>=3D20
>>>>Guys,
>>>>=3D20
>>>>I haven't followed this discussion very closely, so I may have misse=
>>>>
>>>>
>d
>
>
>>>>something.  But one issue I have encountered with contours is that t=
>>>>
>>>>
>he=3D
>
>
>>>>
>>>>
>>>y
>>>
>>>
>>>
>>>
>>>>often end up being very large polygons.  For instance, a "zero eleva=
>>>>
>>>>
>ti=3D
>
>
>>>>
>>>>
>>>on"
>>>
>>>
>>>
>>>
>>>>coastal contour might well be continental sized.  Really big "rings"=
>>>>
>>>>
> g=3D
>
>
>>>>
>>>>
>>>et
>>>
>>>
>>>
>>>
>>>>pushed way up the the spatial index tree and will have to be read fr=
>>>>
>>>>
>om
>
>
>>>>disk and rendered even if the current map view is just somewhere ins=
>>>>
>>>>
>id=3D
>
>
>>>>
>>>>
>>>e
>>>
>>>
>>>
>>>
>>>>the contour ring such that the contour won't actually be visible, bu=
>>>>
>>>>
>t
>
>
>>>>the bounding rectangles of the contour and the view area will overla=
>>>>
>>>>
>p.
>
>
>>>>=3D20
>>>>If this is Bob's problem, he will really be much better off to split=
>>>>
>>>>
> t=3D
>
>
>>>>
>>>>
>>>he
>>>
>>>
>>>
>>>
>>>>contours into shortish line segments instead of keeping as massive s=
>>>>
>>>>
>in=3D
>
>
>>>>
>>>>
>>>gle
>>>
>>>
>>>
>>>
>>>>geometries as this would allow the spatial index to localize things
>>>>much better.
>>>>=3D20
>>>>I didn't speak up about this before because it wasn't clear if this =
>>>>
>>>>
>ha=3D
>
>
>>>>
>>>>
>>>s
>>>
>>>
>>>
>>>
>>>>already been done.
>>>>=3D20
>>>>Best regards,
>>>>--=3D20
>>>>---------------------------------------+----------------------------=
>>>>
>>>>
>--=3D
>
>
>>>>
>>>>
>>>--------
>>>
>>>
>>>
>>>
>>>>I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam@=
>>>>
>>>>
>po=3D
>
>
>>>>
>>>>
>>>box.com
>>>
>>>
>>>
>>>
>>>>light and sound - activate the windows | http://pobox.com/~warmerdam
>>>>and watch the world go round - Rush    | Geospatial Programmer for R=
>>>>
>>>>
>en=3D
>
>
>>>>
>>>>
>>>t
>>>
>>>
>>>
>>>
>>>>=3D20
>>>>=3D20
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>=20
>>
>>
>
>
>