[mapguide-users] RE: SDF load problems
Andy Morsell
amorsell at spatialgis.com
Sat Sep 30 16:05:08 EDT 2006
There is actually a white paper PDF about this topic on Autodesk's web site
now. There is link from the MGOS wiki here:
http://wiki.osgeo.org/index.php/MapGuide_Open_User_Technical_FAQ%27s
Andy Morsell, P.E.
Spatial Integrators, Inc.
http://www.SpatialGIS.com
-----Original Message-----
From: Jim O'Leary [mailto:joleary.public at gmail.com]
Sent: Saturday, September 30, 2006 12:43 PM
To: users at mapguide.osgeo.org
Subject: [mapguide-users] RE: SDF load problems
Did you attach a file with this posting? If so, it didn't make it. Could you
post the XML? Thanks.
Traian Stanev wrote:
>
>
> There is a way around copying the data files to the repository. You
> just need to manually publish the feature source to the MapGuide
repository.
> Here is how:
>
> * Navigate a browser to the html index of the mapguide. For me this is
> http://localhost:8008/mapguide/mapagent/index.html -- you should see
> something like the attached SetResource.png.
>
> * In the upper left frame, click Resource Service
>
> * In the lower left frame click SetResource. We will call the
> SetResource API to publish our custom feature source that points to an
> SDF or SHP somewhere on disk.
>
> * If you want to publish an SDF file, open the attached
> FeatureSourceContent_SDF.txt, locate the string %MG_DATA_FILE_PATH%
> and replace it by the location of your file on the server, e.g. D:\my
> data\sdf files\boundaries.sdf. Save the txt file.
>
> * Back to the html page, in the "Content:" field, give the location of
> the text file you just modified.
>
> * In the "ResourceID:" field, you must specify a valid resource ID. In
> general those look very much like the tree you see in the left pane of
> MG Studio with the string Library:// prepended to them. In our case
> the ID has to end with ".FeatureSource" since we are defining a
> feature source. An example of a valid ID is :
> Library://Test/boundaries.FeatureSource
>
> * Leave the Header field blank and click on Submit.
>
> If you refresh your Studio, you will now see the feature source you
> just published. You can now create a new layer in Studio and point the
> layer to this feature source. Everything else should work as before.
>
> Note that this works for SDF 3.0+ only, not for 2.0. For SHP files,
> the only difference is that you need to replace the
> %MG_DATA_FILE_PATH% by the directory where your SHP/DBF files are, and
> not by the full file path. I also verified that deleting the feature
> source from Studio will NOT delete the SDF file that we manually
> pointed to. In general you can use the SetResource page to publish
> other data to the repository, like maps and layer definitions. You can
> also download resources using the GetResourceContent page, which is
> how I got the attached sample feature sources.
>
> Traian
>
>
>
>
> _____
>
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 8:17 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
>
> Yes, I have done exactly that in the past to see where my files were
> copied to in the repository. I guess my big problem is that there is
> no way around copying our data from the place where it is maintained
> to the repository in order for it to be accessible as a resource.
> Allowing a resource to point to a different physical location would
> make data management quite a bit easier for us and we wouldn't have to
> worry about reversing out the actual physical path to that resource
> and then incorporating that into our batch routines. This would also
> totally eliminate the need to load these resources at all via HTTP or
> as packages.
>
> Andy
> _____
>
> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
> Sent: Wednesday, March 22, 2006 4:07 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
>
>
> Regarding "This would be OK if there was an easy way to know where in
> the repository that feature source is created when it is initially
> defined.", do this experiment:
>
>
>
> 1) Name your source data something unique (ex: myUniqueFile.sdf)
>
> 2) Load it using Studio.
>
> 3) On the server, recursively search the Repository folder under
> the server installation for the data file (ex: myUniqueFile.sdf).
>
>
>
> As long as you don't change the feature source wrapping the data file,
> the location of the resource data file on the server won't change. You
> can directly update the data file there on the server in the future.
>
>
>
> (DISCLAIMER: This hack bypasses the API. If the resource repository
> implementation changes, this might break the hack. Be warned. )
>
>
>
> Also, to get a feel for the repository API, try using the html forms
> from in your browser
> (http://<server_name>/mapguide/mapagent/index.html). It's an easy way
> get a look into the resources being created with the Studio load
> procedures.
>
>
>
> Ed
>
>
>
>
>
> _____
>
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 6:43 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
>
>
> I haven't tried it. I'm sure it would work fine, but it seems like an
> awful lot of work to take a Shape file and copy it from one network
> location to another (the repository)......... And what about those 1
> GB Shape files? It's a lot of package and data shuffling to keep your
> repository data in synch with the source data. I would like to be
> able to point the repository location to reference the source data in
> my chosen network location. Sort of like MapGuide 6.5 UDL files, but
> you could still retain the managed data aspect.
>
>
>
> In the case of converting your source to SDF+, some of us would rather
> use FME scripts to update the SDF files in the repository. This would
> be OK if there was an easy way to know where in the repository that
> feature source is created when it is initially defined.
>
> Andy
>
> _____
>
> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
> Sent: Wednesday, March 22, 2006 3:29 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
> Regarding "Many of us will want to update our data through external
> processes", has anyone tried to write some code which does the
> following?
>
>
>
> 1) Use the Studio Load API to create the package locally from the
> latest source data.
>
> 2) Copy the resulting package file into the Package directory on
> the server.
>
> 3) Use the MGOS Admin service API to apply this updated package.
>
>
>
> Would this meet most of your needs to update the source flat file data
> wrapped by FeatureSource resources?
>
>
>
> Thanks,
>
> Ed
>
>
>
> _____
>
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 6:03 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
>
>
> Sorry, I should have clarified that my rant pertains to flat files only.
> Database connections will be direct to their source.
>
>
>
> Andy
>
> _____
>
> From: Butler, Juan P [mailto:jpbutler at co.pinellas.fl.us]
> Sent: Wednesday, March 22, 2006 2:46 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
> Wow. So you're saying that if and when a provider for Oracle Spatial
> comes along, we won't be able to define the connection to the database
> and have the data come directly from that connection? It basically
> dumps the data (that point in time) into a repository that resides in on
> the MG server?
>
>
>
> Can someone from Autodesk comment on this?
>
>
>
> Juan Butler - Senior Programmer/Analyst GIS Applications Pinellas
> County Information Systems jpbutler at pinellascounty.org
>
>
>
>
>
> _____
>
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 5:37 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
>
>
> This is where using a managed data structure is a detriment, in my
> opinion. Many of us will want to update our data through external
> processes and there is no easy way to just copy files into the
> repository or point to an existing network location. I think this is
> going to become a much bigger problem as people put these systems into
> production and realize that they have to "load" their data into
> repositories. It's also going to be a large problem for keeping data
> up to date with ongoing system maintenance. Scheduled process are
> going to be a necessity to ensure that the data in the repository is
> in synch with the source data. And, in the case of large source
> datasets, the entire dataset will have to be reloaded as there is no
> incremental update method.
>
>
>
> I believe (but have not yet done this) that the Studio API is going to
> have to be used to load these resources, not the server API.
>
>
>
> Andy Morsell, P.E.
> Spatial Integrators, Inc.
> http://www.SpatialGIS.com
>
> _____
>
> From: Joel Carranza [mailto:jec at gatekeeper.com]
> Sent: Wednesday, March 22, 2006 2:17 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
> Ahhh. I see, I misunderstood the use of "Packages" and didn't think too
> look there. Thank you. It is working fine now.
>
>
>
> I have noticed that imported SDFs are processed (into SDF v3, I
> believe). Is it currently possible to import SDF data sources into the
> repository without using Studio at all, just using the server API? Is
> it possible to create your own Packages using the server API? I would
> be interested in doing this for our migration tools.
>
>
>
> Joel Carranza
> Gatekeeper Systems
> 626-449-3070 x114
>
> _____
>
> From: Jason Birch [mailto:Jason.Birch at nanaimo.ca]
> Sent: Wednesday, March 22, 2006 1:42 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
>
> It looks like it's too big. You'll probably have to "Package" the
> data and upload it to the server manually, then bring it in via the
> site Admin web app.
>
>
>
> Jason
>
>
>
> _____
>
> From: Joel Carranza [mailto:jec at gatekeeper.com]
> Sent: Wednesday, March 22, 2006 13:39
> To: users at mapguide.osgeo.org
> Subject: [mapguide-users] SDF load problems
>
> I am trying to load a 225 MB SDF file using MapGuide studio. I have
> tried any number of different options and they all seem to fail. The
> process crunches for a while, and then says "Communicating with server"
> and the progress bars fills to 100% . Eventually, an error dialog is
> popped that says "Unable to complete request to site. Connection lost".
>
> The apache logs showed the following. None of the other logs reported
> anything of interest that I could see.
>
> [Wed Mar 22 13:33:54 2006] [warn] FastCGI: (dynamic) server
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> restarted (pid 5284)
>
> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: scheduled the restart of
> the last (dynamic) server "C:/Program
> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> process: reached dynamicMaxClassProcs (10)
>
> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: (dynamic) server
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> restarted (pid 6008)
>
> [Wed Mar 22 13:34:21 2006] [warn] FastCGI: (dynamic) server
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 5540) terminated with exit with status '0'
>
> [Wed Mar 22 13:34:38 2006] [warn] FastCGI: (dynamic) server
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 6008) termination signaled
>
> [Wed Mar 22 13:34:39 2006] [warn] FastCGI: (dynamic) server
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 6008) terminated with exit with status '0'
>
> Anyone have any ideas about what is going on?
>
> Joel Carranza
> Gatekeeper Systems
> 626-449-3070 x114
>
>
>
>
>
>
--
View this message in context:
http://www.nabble.com/SDF-load-problems-tf1326833.html#a6582422
Sent from the OSGeo MapGuide mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at mapguide.osgeo.org
For additional commands, e-mail: users-help at mapguide.osgeo.org
More information about the Mapguide-users
mailing list