Contours again.

[Bob Basques]

Ed McNierney wrote:

>Bob -
>
>After you run shptree you will see a new file with a .qix extension.  You should then rerun your tests and see a difference.
>
>
It shaved off about 30 sec for the search.  Is there a way to verify
whether the new index is even being used ?  I also tried with some
different tree values (24, 30) and they didn't seem to make a difference
either way., this is one reason I would like to verify if the index is
even being used.  Does it need to be specified anywhere in a MAPfile or ???

bobb

>     - Ed
>
>----- Original Message -----
>From: Bob Basques <bob.basques at ci.stpaul.mn.us>
>To: Ed McNierney <ed at TOPOZONE.COM>
>Sent: Thu, 14 Oct 2004 11:05:02 -0400
>Subject: Re: Contours again.
>
>
>
>
>>Ed McNierney wrote:
>>
>>
>>
>>>Bob -
>>>
>>>I'm not entirely sure from your reply that you understood that Armin's f=
>>>irst suggestion is to simply use shptree to create a spatial index for y=
>>>our single shapefile.  In the directory where your shapefile is stored, =
>>>simply use the command
>>>
>>>   shptree shapefile.shp
>>>
>>>
>>>
>>>
>>I tried this and nothing is produced, the prompt sits there (crewating
>>index of new  LSB format) for a while then just returns.
>>
>>I'm running on REDHAT Fedora Core.
>>
>>Is there some special syntax to look out for?
>>
>>Some notes:  I have the data on a NAS mounted on the WebServer.  I'm
>>running the SHPTREE command on the Server into the home directory of the
>>login.  Don't know if this might be causing write problems or not.
>>
>>
>>
>>>and a file with the same base name and the extension .qix will be create=
>>>d.  Then you can run your test again - creating the index will just take=
>>>a few moments and you don't have to do anything with your data organiza=
>>>tion.
>>>
>>>I'm assuming, by the way, that your test request is actually displaying =
>>>only a small portion of the entire shapefile.
>>>
>>>
>>>
>>Yup, I plan on making a couple of generalized layers for the wider area
>>views.
>>
>>
>>
>>> If you're trying to draw =
>>>the entire 550 MB shapefile in a single image, you have different proble=
>>>ms <g>.
>>>
>>>
>>>
>>>
>>Nope.  Just trying to see if this is going to be a performance problem
>>or not right now.
>>
>>bobb
>>
>>
>>
>>>    - Ed
>>>
>>>Ed McNierney
>>>TopoZone.com
>>>
>>>----- Original Message -----
>>>From: Bob Basques <bob.basques at CI.STPAUL.MN.US>
>>>To: MAPSERVER-USERS at LISTS.UMN.EDU
>>>Sent: Wed, 13 Oct 2004 17:56:00 -0400
>>>Subject: Re: [UMN=5FMAPSERVER-USERS] Contours again.
>>>
>>>
>>>
>>>
>>>
>>>
>>>>Armin Burger wrote:
>>>>=20
>>>>
>>>>
>>>>
>>>>
>>>>>Bob,
>>>>>
>>>>>you could also try just creating a spatial index for the shapefile us=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ing s=3D
>>>
>>>
>>>
>>>
>>>>>hptree. 2 minutes looks like there is no existing spatial index and m=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ore o=3D
>>>
>>>
>>>
>>>
>>>>>r less all features of the shapefile are read all the time.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>Yes I think this is correct.
>>>>=20
>>>>
>>>>
>>>>
>>>>
>>>>>Maybe this ind=3D
>>>>>ex already speeds up things sufficiently.
>>>>>
>>>>>Instead of splitting up the contour lines into seperate shapefiles th=
>>>>>
>>>>>
>>>>>
>>>>>
>>>at ha=3D
>>>
>>>
>>>
>>>
>>>>>ve then to be put together with a shapeindex, it might be an alternat=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ive t=3D
>>>
>>>
>>>
>>>
>>>>>o intersect the contour lines with a regular grid. E.g. like the shee=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ts fo=3D
>>>
>>>
>>>
>>>
>>>>>r topographic maps or a rectangular one with arbitrary extent you cre=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ate o=3D
>>>
>>>
>>>
>>>
>>>>>n your own. That way very long lines with huge amount of vertices are=
>>>>>
>>>>>
>>>>>
>>>>>
>>>spli=3D
>>>
>>>
>>>
>>>
>>>>>t up into smaller chunks.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>I actually started out with something like that in the AutoCAD world.
>>>>Each tile (1/2 sq. mile) was brought together into a single coverage. =
>>>>
>>>>
>>>>
>>>>
>>>I
>>>
>>>
>>>
>>>
>>>>didn't do any joins on the linework, I just let them be seperate
>>>>entities.  I figured that was better than joining them anyway.
>>>>=20
>>>>
>>>>
>>>>
>>>>
>>>>>The time to read such features is much less than=3D
>>>>>for the large ones, but you still have everything in a single shapef=
>>>>>
>>>>>
>>>>>
>>>>>
>>>ile.
>>>
>>>
>>>
>>>
>>>>>If you want just display the values as integer, it might be the easie=
>>>>>
>>>>>
>>>>>
>>>>>
>>>st wa=3D
>>>
>>>
>>>
>>>
>>>>>y to add a new integer column in the dbf file and copy the values fro=
>>>>>
>>>>>
>>>>>
>>>>>
>>>m the=3D
>>>
>>>
>>>
>>>
>>>>>float column. And then use the new integer column  for the labeling.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>Good idea, although I think I'll just get rid of the old column
>>>>afterwards.  I tried some things with the data writer(AutoCAD Map) but
>>>>nothing seemed to help with the labelling.
>>>>=20
>>>>I think your idea will work though.  I can load the DBF up and add a
>>>>column easy enough.
>>>>=20
>>>>Thanks for the suggestions
>>>>=20
>>>>bobb
>>>>=20
>>>>
>>>>
>>>>
>>>>
>>>>>armin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>All,
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Well I tried out a composite file before asking this question.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>I just ran a 550+meg SHP file for our Contour data.  There is only o=
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>ne
>>>
>>>
>>>
>>>
>>>>>>attribute in the DBF, which is the elevation.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>a Typical request takes around 120 sec to complete.  so I'm going to=
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>try
>>>
>>>
>>>
>>>
>>>>>>and tile the shp file out into smaller version and use a tile index =
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>on
>>>
>>>
>>>
>>>
>>>>>>top of that.  Are there any pitfalls I should watch out for during t=
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>he
>>>
>>>
>>>
>>>
>>>>>>process of tiling a SHP file=3F
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Also, how do I label with a INTEGER vs a REAL for the elevations.  I=
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>'m
>>>
>>>
>>>
>>>
>>>>>>getting  14 decimal places in the labels.  :c)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>bobb
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>=20
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>
>
>
>