[mapguide-users] Unmanaged SHP File Locations

Tue May 29 12:17:44 EDT 2007

It sounds like your last paragraph is probably what is happening.  If that
is the case, this should be addressed as soon as possible since performance
differences between having a folder based unmanaged data connection and
loading them into the repository can be very significant.  In my test case,
with about 30 shape files in one folder, the unmanaged feature source
performance is so bad that it cannot be used.

To use the unmanaged data source feature in this manner is the most common
scenario I can think of that myself and my clients will encounter - a whole
bunch of files in one folder.  The work around of having to create multiple
MGOS data connections, one per shape file, is not an appealing one.  Perhaps
behind the scenes, if one creates a layer defined from a single class/shp
file from an unmanaged folder based data connection, then MapGuide can build
individual config files and reference those.  

Andy 

-----Original Message-----
From: mapguide-users-bounces at lists.osgeo.org
[mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Robert Fortin
Sent: Tuesday, May 29, 2007 8:54 AM
To: MapGuide Users Mail List
Subject: RE: [mapguide-users] Unmanaged SHP File Locations

For the SHP provider, open depends on the connection parameters used. If
it's a file, it will open only that file;  If it's directory it will open
all the shp file in that directory (no XML involved at this point).
You might be under the impression that even if the layer in MG was created
from a directory, it should only open a single shp file.  

I asssume MG is always using the directory connection even if you selected a
single class for the layer.  Also, it would be providing the entire schema
in the config file for the directory connection even if the layer refers to
a single class/shp file.  The SHP provider would then have to verify the
info in the config file on open even if only one class is used by the layer.
This would slow down connection time compare to a connection with a single
shp file.

RF 

-----Original Message-----
From: mapguide-users-bounces at lists.osgeo.org
[mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Andy Morsell
Sent: Friday, May 25, 2007 9:24 PM
To: 'MapGuide Users Mail List'
Subject: RE: [mapguide-users] Unmanaged SHP File Locations

Zak,
In this case, I loaded the exact some files into the repository
(managed) that I had in the unmanaged directory to see the differences in
performance.
So the performance difference must be attributed to something internal to
MapGuide / FDO.

Pointing at a directory of shapefiles is done at the unmanaged data source
alias configuration level in webconfig.ini.  When you create a layer, that
layer still only references one feature class (one shape
file) from within that directory so there should be no reason for the
provider to do extra work when accessing layers for a map render since it
knows what exact shape file to access.  So there's something else going on
here........ Sure, when you are first defining a layer, it must enumerate
all shape files in a directory so you can choose which feature source to
use, but that's a one time only thing.

Andy 

-----Original Message-----
From: mapguide-users-bounces at lists.osgeo.org
[mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Zak James
Sent: Friday, May 25, 2007 5:39 PM
To: MapGuide Users Mail List
Subject: Re: [mapguide-users] Unmanaged SHP File Locations

There may be efficiencies to be gained in the provider, but there are some
limitations that don't have easy solutions.

There are no attribute indexes for shapefiles so a lookup in a large file is
going to be slow.

Pointing at a directory of shapefiles is like pointing at one big data
source with many feature classes. It may be convenient during development,
but for production you will want to reference specific shapefiles or at
least break down the large directory into smaller sets.
I'm pretty sure managed shapefile feature sources are one per directory.

If an unmanaged shape feature source draws more slowly than a managed one,
it's worth checking that the shx file is present and valid.

zak

On 5/25/07, Andy Morsell <amorsell at spatialgis.com> wrote:
>
>
> First, I love the unmanaged resource concept and will most likely 
> deploy most future MGOS applications this way.
>
> But, similar to Chris' problems, I have discovered that displaying 
> tooltips from unmanaged shape files causes a much more significant 
> performance hit and delay.  In my case, I have a 150 MB parcel polygon

> shape file for a county.  In the unmanaged case, each tooltip AJAX 
> fetch spikes the CPU with mgserver.exe at 100% for a couple of 
> seconds.  Doing the same with the exact same shape file in a managed 
> state (not converted to SDF at load time), the CPU will only hit 25% 
> or
less and responds much more quickly.
>
> Further, I discovered that the more unmanaged data in a map, the 
> slower the tooltip response is for all layers.  For instance, if I add

> another 7 or so layers (like streets, water bodies, etc) to the map, 
> when I float over a parcel polygon it can take up to 10 seconds for a 
> tooltip response during which time mgserver is cranking away.  So, I'm

> not sure what it's doing at that point.  All of these additional 
> layers are all in the same unmanaged data source folder.
>
> I went ahead and loaded all of the shape files into the repository and

> created a duplicate map pointing to the new managed feature sources.
> This map behaves fine with tooltip fetches consuming 25% of a single 
> CPU and very quick response time.  So, this seems to confirm that the 
> more unmanaged feature sources in a map (at least from the same 
> directory), the larger the performance hit and the slower the map 
> responds
to requests.
>
> I have appended this information to the TRAC ticket that Chris started

> earlier today.
>
>
> Andy Morsell, P.E.
> Spatial Integrators, Inc.
> http://www.SpatialGIS.com
>  ________________________________
>  From: mapguide-users-bounces at lists.osgeo.org
> [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Chris 
> Gountanis
> Sent: Friday, May 25, 2007 7:52 AM
> To: 'MapGuide Users Mail List'
> Subject: [mapguide-users] Unmanaged SHP File Locations
> Importance: High
>
>
>
>
>
>
> Why do unmanaged FeatureSources respond slower as more files get added

> to the directory location? We have a FeatureSource called SHPDefualt 
> for example. The zooms take longer as well as other functions. It 
> seems when you query a Feature Source is does not hit the DBF you want

> directly it seems to fiddle with everything in the unmanaged 
> directory. We love the concept of a folder that contains all SHP 
> files. One location is great for batch processes and great for editing

> new layers. It is just simple for customers to understand. For sure 
> easier to back up and recreate. Will this always be a performance
killer?
>
>
>
>
>
> Is this a bug? Will this be fixed soon?
>
>
>
>
>
> This really takes the complication out of knowing what the heck is 
> going on in the repository. Here is XML Example:
>
> <?xml version="1.0" encoding="utf-8" ?>
>
> <FeatureSource
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> xsi:noNamespaceSchemaLocation="FeatureSource-1.0.0.xsd">
>
> <Provider>OSGeo.SHP</Provider>
>
> <Parameter>
>
> <Name>DefaultFileLocation</Name>
>
> <Value>C:\SHPFILES</Value>
>
> </Parameter>
>
> </FeatureSource>
>
>
>
>
>
>
>
>
> Chris
>
>
> _______________________________________________
> mapguide-users mailing list
> mapguide-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-users
>
>

--
Zak James
Applications and Software Development
DM Solutions Group Inc.
http://www.dmsolutions.ca
_______________________________________________
mapguide-users mailing list
mapguide-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-users

_______________________________________________
mapguide-users mailing list
mapguide-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-users

_______________________________________________
mapguide-users mailing list
mapguide-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-users