how to speed up MapServer performance?
Zhonghai Wang
zhonghaiw at GMAIL.COM
Tue May 16 10:42:07 EDT 2006
Hi All,
I am trying to make the application run with ESRI data, I've transformed the
MapInfo format data into ESRI format, and then:
1. perform "shptree" batch command, to get the "*.qix" file. This really
speeds up the performance, and each request takes about 1.5 second, but the
initializing request takes a little bit longer, about 6 second. (at the
initializing level layer "country boundaries" and layer "forests" should be
visible.)
2. perform the "tile4ms" command, get the tileindex shapfile and DBF for
road networks and each other layer, because the roads are in separate vector
layers, in the mapfile I still set each in a seperate layer object. for
other shapfiles that do not have the same columns and attributes I perform
the command separately for them, and each has a tileindex shapefile. ---
this solution seems not better than the first one, for each request it takes
longer.
questions again ^_^:
1. is it possible to speed up the initializing ?
2. I am trying to put the road network shapefiles into one layer object
after having got some information about tile4ms in the list, but how to
styling them separately, since they are different road classes (in
original data set they are in separate shapefiles)?
*****
Frank Warmerdam said, FASTCGI is not really speed up the applications with
MapInfo or ESRI data, I think he is right, so I am not trying to use it.
I do not know that much about Linux, therefore an application based on Linux
is currently possible for me. Maybe I will try it someday, and I'd like to,
since it can benefit GIS applications.
On 5/16/06, Ed McNierney <ed at topozone.com> wrote:
>
> J.F. -
>
> I was with you all the way until you mentioned operating systems! There's
> no reason to think that Zhonghai's operating system is at all related to his
> problem, and it's dangerous to propagate operating system myths. I run lots
> of MapServer machines on Windows and Linux (mostly Linux) and I cannot find
> any evidence (after 5 years or so) that there's any difference worth
> thinking about on identical hardware.
>
> If you can show me a MapServer test with identical configurations on
> identical hardware showing statistically significant differences in
> performance when running Window vs. Linux, I'd love to see the details.
>
> - Ed
>
> Ed McNierney
> President and Chief Mapmaker
> TopoZone.com / Maps a la carte, Inc.
> 73 Princeton Street, Suite 305
> North Chelmsford, MA 01863
> Phone: +1 (978) 251-4242
> Fax: +1 (978) 251-1396
> ed at topozone.com
>
> ------------------------------
> *From:* UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] *On
> Behalf Of *Doyon, Jean-Francois
> *Sent:* Tuesday, May 16, 2006 9:53 AM
> *To:* MAPSERVER-USERS at LISTS.UMN.EDU
>
> *Subject:* Re: [UMN_MAPSERVER-USERS] how to speed up MapServer
> performance?
>
>
> Zhonghai,
>
> You have the one big data set in there, which is the roads. If you're
> trying to display ALL the data in the one map, all at once, then 3 secs
> isn't all that bad, considering you have 60 megs of data to work through.
>
> You may want to consider generalizing the data for viewing at different
> scales.
>
> The fileformat itself will create a tiny bit of an impact, since it all
> goes through OGR, for "raw" speed, use shapefiles , indexed with .qix ...
>
> The MapServer CGI is about the fastest mapping engine you'll find out
> there, open source or commercial!
>
> Also, what's the hardware like? Notably disk IO?
>
> Also, I'm sorry, but I must ask :) Does it have to be windows? On
> Windows, processes are quite expensive, which includes CGI ... this isn't
> the case on Linux, where processes are cheap, and threads are expensive.
> Also, Linux tends to cache things better in my experience.
>
> If you're using WMS, I think there's a FastCGI interface you could try
> using, though I'm not sure how mature it is.
>
> J.F.
>
> ------------------------------
> *From:* UMN MapServer Users List [mailto:MAPSERVER-USERS at lists.umn.edu] *On
> Behalf Of *Zhonghai Wang
> *Sent:* May 16, 2006 4:09 AM
>
> *To:* MAPSERVER-USERS at lists.umn.edu
> *Subject:* Re: [UMN_MAPSERVER-USERS] how to speed up MapServer
> performance?
>
>
> Hi my friends,
>
> I am really very thankful for all your information, I am a newbie in
> MapServer.
>
> here is the data amount that I am using for the application:
> (all the source data is MapInfo format, and what I calculated is the
> ".MAP" file)
>
> 1. boundaries -- 388 KB
> 2. forests -- 9163 KB
> 3. lakes -- 26 KB
> 4. city parks -- 25 KB
> 5. settlement -- 2279 KB
> 6. rivers -- 1653 KB
> 7. road networks -- 56370 KB
> 8. railway -- 1343 KB
> 9. train station -- 27 KB
> 10 bus station -- 2 KB
> 11. tourist centers -- 9 KB
> 12. theaters -- 6 KB
> 13. shopping center -- 7 KB
> 14. city location -- 9 KB
>
> *****
> and here is how I am symbolizing the layers, mapfile for all these layers:
>
> ***
> LAYER
> NAME "Country"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_COUNTRY.TAB"
> TRANSPARENCY 40
> DEBUG ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> #NAME "Boundaries"
> STYLE
> SYMBOL 0
> COLOR 155 155 155
> OUTLINECOLOR 155 155 155
> BACKGROUNDCOLOR 255 255 255
> END
> END
> END
> LAYER
> NAME "lakes"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_LAKE.TAB"
> DEBUG ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Lake"
> STYLE
> COLOR 102 153 255
> END
> END
> END
> LAYER
> NAME "woodland"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_WOODLAND_G.TAB"
> DEBUG ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "woodland"
> STYLE
> COLOR 0 185 92
> END
> END
> END
> LAYER
> NAME "woodland"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_WOODLAND.TAB"
> MAXSCALE 100000
> #LABELITEM "Street_Label"
> #SYMBOLSCALE 50000
> #LABELMAXSCALE 80000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "woodland"
> STYLE
> COLOR 0 185 92
> END
> END
> END
> LAYER
> NAME "city parks"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_CITY_PARK.TAB"
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Park"
> STYLE
> SYMBOL 0
> COLOR 233 254 205
> END
> END
> END
> LAYER
> NAME "settlement"
> STATUS DEFAULT
> TYPE POLYGON
> #CLASSITEM "Area"
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_BUILT_UP_AREA.TAB"
> MAXSCALE 800000
> #LABELITEM "Street_Label"
> #SYMBOLSCALE 80000
> #LABELMAXSCALE 120000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "settlement"
> #EXPRESSION ([Area]>20000)
> STYLE
> COLOR 207 207 158
> END
> END
> END
> LAYER
> NAME "rivers"
> STATUS DEFAULT
> TYPE POLYGON
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_WATERWYS.TAB"
> MAXSCALE 800000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "river"
> STYLE
> COLOR 102 153 255
> END
> END
> END
> LAYER
> NAME "street50"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_S50.TAB"
> MAXSCALE 40000
> LABELITEM "Street_Label"
> SYMBOLSCALE 5000
> LABELMAXSCALE 20000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Street 50"
> LABEL
> # must be in your FONTSET
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 2
> MAXSIZE 8
> ANGLE AUTO
> POSITION CC
> OFFSET 0 0
> COLOR 0 0 255
> MINFEATURESIZE AUTO
> BUFFER 0
> END
> STYLE
> SYMBOL "point"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> COLOR 255 255 255
> END
> END
> END
> LAYER
> NAME "oneway"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_ONEWAY.TAB"
> MAXSCALE 40000
> SYMBOLSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Oneway"
> STYLE
> SYMBOL "oneway"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> COLOR 255 51 0
> END
> END
> END
> LAYER
> NAME "street40"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_S40.TAB"
> MAXSCALE 100000
> LABELITEM "Street_Label"
> SYMBOLSCALE 15000
> LABELMAXSCALE 30000
> LABELCACHE ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Street 40"
> LABEL
> # must be in the FONTSET
> TYPE TRUETYPE
> FONT "vera"
> SIZE 9
> MINSIZE 2
> MAXSIZE 9
> ANGLE AUTO
> POSITION CC
> OFFSET 0 0
> COLOR 0 0 255
> MINFEATURESIZE 20
> BUFFER 0
> END
> STYLE
> SYMBOL "point"
> SIZE 10
> MINSIZE 3
> MAXSIZE 10
> COLOR 160 160 160
> ANTIALIAS TRUE
> END
> STYLE
> SYMBOL "point"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> COLOR 255 255 255
> ANTIALIAS TRUE
> END
> END
> END
> LAYER
> NAME "street30"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_S30.TAB"
> MAXSCALE 250000
> LABELITEM "Street_Label"
> SYMBOLSCALE 40000
> LABELMAXSCALE 120000
> LABELCACHE ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Street 30"
> LABEL
> # must be in your FONTSET
> TYPE TRUETYPE
> FONT "vera"
> SIZE 9
> MINSIZE 2
> MAXSIZE 9
> ANGLE AUTO
> POSITION CC
> OFFSET 0 0
> COLOR 0 0 255
> MINFEATURESIZE 30
> #MINDISTANCE 10
> BUFFER 0
> END
> STYLE
> SYMBOL "point"
> SIZE 10
> MINSIZE 3
> MAXSIZE 10
> COLOR 160 160 160
> END
> STYLE
> SYMBOL "point"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> COLOR 255 204 51
> END
> END
> END
> LAYER
> NAME "railway"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_RAILRDS.TAB"
> MAXSCALE 8000
> SYMBOLSCALE 4000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "railway"
> SYMBOL 'point'
> COLOR 0 0 0
> SIZE 5
> MINSIZE 1
> MAXSIZE 5
> OVERLAYSYMBOL "dashedline"
> OVERLAYCOLOR 255 255 255
> OVERLAYSIZE 3
> OVERLAYMINSIZE 0
> OVERLAYMAXSIZE 3
> END
> END
> LAYER
> NAME "street20"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_S20.TAB"
> MAXSCALE 900000
> LABELITEM "Street_Label"
> SYMBOLSCALE 100000
> LABELMAXSCALE 430000
> LABELCACHE ON
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Street 20"
> LABEL
> # must be in your FONTSET
> TYPE TRUETYPE
> FONT "vera"
> SIZE 9
> MINSIZE 1
> MAXSIZE 9
> ANGLE AUTO
> POSITION CC
> OFFSET 0 0
> COLOR 0 0 255
> MINFEATURESIZE 50
> #MINDISTANCE 10
> BUFFER 0
> END
> STYLE
> SYMBOL "point"
> SIZE 10
> MINSIZE 3
> MAXSIZE 10
> COLOR 160 160 160
> END
> STYLE
> SYMBOL "point"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> COLOR 247 193 157
> END
> END
> END
> LAYER
> NAME "street10"
> STATUS DEFAULT
> TYPE LINE
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_S10_O.TAB"
> MAXSCALE 1700000
> LABELITEM "Street_Label"
> SYMBOLSCALE 400000
> LABELMAXSCALE 500000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Street 10"
> LABEL
> # must be in your FONTSET
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 1
> MAXSIZE 8
> ANGLE AUTO
> POSITION CC
> OFFSET 0 0
> COLOR 255 0 0
> BACKGROUNDCOLOR 153 204 153
> MINFEATURESIZE AUTO
> MINDISTANCE 200
> BUFFER 1
> END
> STYLE
> SYMBOL "point"
> SIZE 10
> MINSIZE 3
> MAXSIZE 10
> COLOR 255 0 0
> END
> STYLE
> SYMBOL "point"
> SIZE 6
> MINSIZE 1
> MAXSIZE 6
> COLOR 255 255 0
> END
> END
> END
> LAYER
> NAME "train station"
> STATUS OFF
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_TRAIN_STATION.TAB"
> MAXSCALE 10000
> #LABELITEM "Label"
> SYMBOLSCALE 1000
> LABELMAXSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Train station"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 2
> MAXSIZE 8
> ANGLE AUTO
> POSITION UC
> OFFSET 0 0
> END
> STYLE
> SYMBOL "triangle"
> COLOR 153 0 0
> SIZE 8
> MINSIZE 3
> MAXSIZE 8
> END
> END
> END
> LAYER
> NAME "bus station"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_BUS_STATION.TAB"
> MAXSCALE 10000
> LABELITEM "Label"
> SYMBOLSCALE 1000
> LABELMAXSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "bus station"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 2
> MAXSIZE 8
> ANGLE AUTO
> POSITION UC
> OFFSET 0 0
> END
> STYLE
> SYMBOL "triangle"
> COLOR 0 153 0
> SIZE 8
> MINSIZE 3
> MAXSIZE 8
> END
> END
> END
> LAYER
> NAME "LOCATIONS0"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_AND_LOCATIONS0.TAB"
> MAXSCALE 1000000
> LABELITEM "Name"
> SYMBOLSCALE 200000
> LABELMAXSCALE 500000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "LOCATIONS0"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 12
> MINSIZE 2
> MAXSIZE 12
> ANGLE AUTO
> POSITION AUTO
> OFFSET 0 0
> COLOR 0 0 255
> END
> STYLE
> SYMBOL "star"
> COLOR 255 0 0
> SIZE 18
> MINSIZE 8
> MAXSIZE 18
> END
> END
> END
> LAYER
> NAME "LOCATIONS1"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_AND_LOCATIONS1.TAB"
> MAXSCALE 800000
> LABELITEM "Name"
> SYMBOLSCALE 200000
> LABELMAXSCALE 500000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "LOCATIONS1"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 12
> MINSIZE 2
> MAXSIZE 12
> ANGLE AUTO
> POSITION AUTO
> OFFSET 0 0
> COLOR 0 0 255
> END
> STYLE
> SYMBOL "point"
> COLOR 153 153 0
> SIZE 8
> MINSIZE 3
> MAXSIZE 8
> END
> END
> END
> LAYER
> NAME "LOCATIONS2"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_AND_LOCATIONS2.TAB"
> MAXSCALE 800000
> LABELITEM "Name"
> SYMBOLSCALE 200000
> LABELMAXSCALE 500000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "LOCATIONS2"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 10
> MINSIZE 2
> MAXSIZE 10
> ANGLE AUTO
> POSITION AUTO
> OFFSET 0 0
> COLOR 0 0 255
> END
> STYLE
> SYMBOL "point"
> COLOR 153 153 0
> SIZE 6
> MINSIZE 1
> MAXSIZE 6
> END
> END
> END
> LAYER
> NAME "LOCATIONS3"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_AND_LOCATIONS3.TAB"
> MAXSCALE 400000
> LABELITEM "Name"
> SYMBOLSCALE 200000
> LABELMAXSCALE 200000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "LOCATIONS3"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 10
> MINSIZE 2
> MAXSIZE 10
> ANGLE AUTO
> POSITION AUTO
> OFFSET 0 0
> COLOR 0 0 255
> END
> STYLE
> SYMBOL "point"
> COLOR 153 51 153
> SIZE 6
> MINSIZE 1
> MAXSIZE 6
> END
> END
> END
> LAYER
> NAME "Tourist Information"
> STATUS OFF
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_TOURIST_INFORMATION.TAB"
> MAXSCALE 10000
> LABELITEM "Label"
> SYMBOLSCALE 1000
> LABELMAXSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Tourist Information"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 2
> MAXSIZE 8
> ANGLE AUTO
> POSITION UC
> OFFSET 0 0
> END
> STYLE
> SYMBOL "rectangle"
> COLOR 255 51 0
> SIZE 8
> MINSIZE 3
> MAXSIZE 8
> END
> END
> END
> LAYER
> NAME "Shopping Center"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_SHOPPING_CENTER.TAB"
> MAXSCALE 10000
> LABELITEM "Label"
> SYMBOLSCALE 1000
> LABELMAXSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "shopping center"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 10
> MINSIZE 2
> MAXSIZE 10
> ANGLE AUTO
> POSITION UC
> OFFSET 0 0
> END
> STYLE
> SYMBOL "special"
> COLOR 255 51 0
> SIZE 15
> MINSIZE 6
> MAXSIZE 15
> END
> END
> END
> LAYER
> NAME "Theater"
> STATUS DEFAULT
> TYPE POINT
> CONNECTIONTYPE OGR
> CONNECTION "CA_NAV_THEATER.TAB"
> MAXSCALE 10000
> LABELITEM "Label"
> SYMBOLSCALE 1000
> LABELMAXSCALE 5000
> PROJECTION
> "init=epsg:4326"
> END
> CLASS
> NAME "Theater"
> LABEL
> TYPE TRUETYPE
> FONT "vera"
> SIZE 8
> MINSIZE 2
> MAXSIZE 8
> ANGLE AUTO
> POSITION UC
> OFFSET 0 0
> END
> STYLE
> SYMBOL "triangle"
> COLOR 255 51 0
> SIZE 8
> MINSIZE 3
> MAXSIZE 8
> END
> END
> END
> ***********************
>
> is it still possible to speed up my application?
>
> ****
>
> the system parameters of my application:
>
> Windows XP professional
> MS4W 4.8.1
> data source -- MapInfo data
>
> *--only CGI parameters are used,since it is a pure static version with
> HTML,
>
> is it possible to develop a pure static application based on MapServer,
> even in conjunction with PHP/MapScript or Chameleon?
>
>
> thanks for any clues.
>
> zhonghai
>
> 16.05.2006
>
>
> On 5/15/06, Steve Lime <Steve.Lime at dnr.state.mn.us> wrote:
> >
> > We really need more information about the nature of the data. I mean if
> > you've got
> > 20 layers each with a million polygons and are trying to show all of
> > them then 3 seconds
> > might be really fast! Also, the way you are symbolizing features may
> > impact things.
> >
> > In my experience neither 1 or 2 is worth considering...
> >
> > Steve
> >
> > Steve
> >
> > >>> Zhonghai Wang < zhonghaiw at GMAIL.COM> 5/15/2006 9:15:49 AM >>>
> > Hi there,
> >
> > for a special purpose I've developed an static version application based
> > on
> > MapServer CGI, which means I just use pure HTML tages for the GUI, even
> > without JavaScript code snippets to validate forms.
> >
> > I have about 20 map layers to display according to different map scale
> > (ussing the MAXSCALE, SYMBOLSCALE AND LABELMAXSCALE), there are no data
> > classifications within a layer object, the status of all layers are set
> > as
> > default.
> >
> > I've made a test on a standalone MapServer (XP system), every request
> > takes
> > about 3 second. Obvioursly it's not fast enough for a web application,
> > since
> > there may be many users requesting the server at the same time if the
> > application is published on the web.
> >
> > Is MapServer really so slow? can anybody give me some ideas?should I
> > modify
> > the mapfiles?
> >
> > I am wondering whether the following solutions will work:
> > 1. define every layer as a seperate mapfile, and combine them together
> > by
> > request
> > 2. turn off the non-displayed layers off on the fly
> >
> > are these possible to speed up MapServer performance?
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-users/attachments/20060516/08299ed0/attachment.html
More information about the mapserver-users
mailing list