[mapguide-users] RE: SDF load problems

Andy Morsell amorsell at spatialgis.com
Sat Sep 30 16:05:08 EDT 2006


There is actually a white paper PDF about this topic on Autodesk's web site
now.  There is link from the MGOS wiki here:

http://wiki.osgeo.org/index.php/MapGuide_Open_User_Technical_FAQ%27s 


Andy Morsell, P.E.
Spatial Integrators, Inc.
http://www.SpatialGIS.com

-----Original Message-----
From: Jim O'Leary [mailto:joleary.public at gmail.com] 
Sent: Saturday, September 30, 2006 12:43 PM
To: users at mapguide.osgeo.org
Subject: [mapguide-users] RE: SDF load problems


Did you attach a file with this posting? If so, it didn't make it. Could you
post the XML? Thanks.



Traian Stanev wrote:
> 
>  
> There is a way around copying the data files to the repository. You 
> just need to manually publish the feature source to the MapGuide
repository.
> Here is how:
>  
> * Navigate a browser to the html index of the mapguide. For me this is 
> http://localhost:8008/mapguide/mapagent/index.html -- you should see 
> something like the attached SetResource.png.
>  
> * In the upper left frame, click Resource Service
>  
> * In the lower left frame click SetResource. We will call the 
> SetResource API to publish our custom feature source that points to an 
> SDF or SHP somewhere on disk.
>  
> * If you want to publish an SDF file, open the attached 
> FeatureSourceContent_SDF.txt, locate the string %MG_DATA_FILE_PATH% 
> and replace it by the location of your file on the server, e.g. D:\my 
> data\sdf files\boundaries.sdf. Save the txt file.
>  
> * Back to the html page, in the "Content:" field, give the location of 
> the text file you just modified.
>  
> * In the "ResourceID:" field, you must specify a valid resource ID. In 
> general those look very much like the tree you see in the left pane of 
> MG Studio with the string Library:// prepended to them. In our case 
> the ID has to end with ".FeatureSource" since we are defining a 
> feature source. An example of a valid ID is :
> Library://Test/boundaries.FeatureSource
>  
> * Leave the Header field blank and click on Submit.
>  
> If you refresh your Studio, you will now see the feature source you 
> just published. You can now create a new layer in Studio and point the 
> layer to this feature source. Everything else should work as before.
>  
> Note that this works for SDF 3.0+ only, not for 2.0. For SHP files, 
> the only difference is that you need to replace the 
> %MG_DATA_FILE_PATH% by the directory where your SHP/DBF files are, and 
> not by the full file path. I also verified that deleting the feature 
> source from Studio will NOT delete the SDF file that we manually 
> pointed to. In general you can use the SetResource page to publish 
> other data to the repository, like maps and layer definitions. You can 
> also download resources using the GetResourceContent page, which is 
> how I got the attached sample feature sources.
>  
> Traian
>  
>  
> 
> 
>   _____
> 
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 8:17 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> 
> Yes, I have done exactly that in the past to see where my files were 
> copied to in the repository.  I guess my big problem is that there is 
> no way around copying our data from the place where it is maintained 
> to the repository in order for it to be accessible as a resource.  
> Allowing a resource to point to a different physical location would 
> make data management quite a bit easier for us and we wouldn't have to 
> worry about reversing out the actual physical path to that resource 
> and then incorporating that into our batch routines.  This would also 
> totally eliminate the need to load these resources at all via HTTP or 
> as packages.
> 
> Andy 
>   _____
> 
> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
> Sent: Wednesday, March 22, 2006 4:07 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> 
> 
> Regarding "This would be OK if there was an easy way to know where in 
> the repository that feature source is created when it is initially 
> defined.", do this experiment:
> 
>  
> 
> 1)       Name your source data something unique (ex: myUniqueFile.sdf)
> 
> 2)       Load it using Studio.
> 
> 3)       On the server, recursively search the Repository folder under
> the server installation for the data file (ex: myUniqueFile.sdf).
> 
>  
> 
> As long as you don't change the feature source wrapping the data file, 
> the location of the resource data file on the server won't change. You 
> can directly update the data file there on the server in the future.
> 
>  
> 
> (DISCLAIMER: This hack bypasses the API. If the resource repository 
> implementation changes, this might break the hack. Be warned. )
> 
>  
> 
> Also, to get a feel for the repository API, try using the html forms 
> from in your browser 
> (http://<server_name>/mapguide/mapagent/index.html). It's an easy way 
> get a look into the resources being created with the Studio load 
> procedures.
> 
>  
> 
> Ed
> 
>  
> 
>  
> 
>   _____
> 
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 6:43 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
>  
> 
> I haven't tried it.  I'm sure it would work fine, but it seems like an 
> awful lot of work to take a Shape file and copy it from one network 
> location to another (the repository)......... And what about those 1 
> GB Shape files?  It's a lot of package and data shuffling to keep your 
> repository data in synch with the source data.  I would like to be 
> able to point the repository location to reference the source data in 
> my chosen network location.  Sort of like MapGuide 6.5 UDL files, but 
> you could still retain the managed data aspect.
> 
>  
> 
> In the case of converting your source to SDF+, some of us would rather 
> use FME scripts to update the SDF files in the repository.  This would 
> be OK if there was an easy way to know where in the repository that 
> feature source is created when it is initially defined.
> 
> Andy
> 
>   _____
> 
> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
> Sent: Wednesday, March 22, 2006 3:29 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> Regarding "Many of us will want to update our data through external 
> processes", has anyone tried to write some code which does the 
> following?
> 
>  
> 
> 1)       Use the Studio Load API to create the package locally from the
> latest source data.
> 
> 2)       Copy the resulting package file into the Package directory on
> the server.
> 
> 3)       Use the MGOS Admin service API to apply this updated package.
> 
>  
> 
> Would this meet most of your needs to update the source flat file data 
> wrapped by FeatureSource resources?
> 
>  
> 
> Thanks,
> 
> Ed
> 
>  
> 
>   _____
> 
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 6:03 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
>  
> 
> Sorry, I should have clarified that my rant pertains to flat files only.
> Database connections will be direct to their source.
> 
>  
> 
> Andy
> 
>   _____
> 
> From: Butler, Juan P [mailto:jpbutler at co.pinellas.fl.us]
> Sent: Wednesday, March 22, 2006 2:46 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> Wow.  So you're saying that if and when a provider for Oracle Spatial 
> comes along, we won't be able to define the connection to the database
> and have the data come directly from that connection?   It basically
> dumps the data (that point in time) into a repository that resides in on
> the MG server?   
> 
>  
> 
> Can someone from Autodesk comment on this?   
> 
>  
> 
> Juan Butler - Senior Programmer/Analyst GIS Applications Pinellas 
> County Information Systems jpbutler at pinellascounty.org
> 
>  
> 
>  
> 
>   _____
> 
> From: Andy Morsell [mailto:amorsell at spatialgis.com]
> Sent: Wednesday, March 22, 2006 5:37 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
>  
> 
> This is where using a managed data structure is a detriment, in my 
> opinion.  Many of us will want to update our data through external 
> processes and there is no easy way to just copy files into the 
> repository or point to an existing network location.  I think this is 
> going to become a much bigger problem as people put these systems into 
> production and realize that they have to "load" their data into 
> repositories.  It's also going to be a large problem for keeping data 
> up to date with ongoing system maintenance.  Scheduled process are 
> going to be a necessity to ensure that the data in the repository is 
> in synch with the source data.  And, in the case of large source 
> datasets, the entire dataset will have to be reloaded as there is no 
> incremental update method.
> 
>  
> 
> I believe (but have not yet done this) that the Studio API is going to 
> have to be used to load these resources, not the server API.
> 
>  
> 
> Andy Morsell, P.E. 
> Spatial Integrators, Inc. 
> http://www.SpatialGIS.com
> 
>   _____
> 
> From: Joel Carranza [mailto:jec at gatekeeper.com]
> Sent: Wednesday, March 22, 2006 2:17 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> Ahhh. I see, I misunderstood the use of "Packages" and didn't think too
> look there.   Thank you. It is working fine now. 
> 
>  
> 
> I have noticed that imported SDFs are processed (into SDF v3, I 
> believe). Is it currently possible to import SDF data sources into the 
> repository without using Studio at all, just using the server API? Is 
> it possible to create your own Packages using the server API?  I would 
> be interested in doing this for our migration tools.
> 
>  
> 
> Joel Carranza
> Gatekeeper Systems
> 626-449-3070 x114
> 
>   _____
> 
> From: Jason Birch [mailto:Jason.Birch at nanaimo.ca]
> Sent: Wednesday, March 22, 2006 1:42 PM
> To: users at mapguide.osgeo.org
> Subject: RE: [mapguide-users] SDF load problems
> 
> It looks like it's too big.  You'll probably have to "Package" the 
> data and upload it to the server manually, then bring it in via the 
> site Admin web app.
> 
>  
> 
> Jason
> 
>  
> 
>   _____
> 
> From: Joel Carranza [mailto:jec at gatekeeper.com]
> Sent: Wednesday, March 22, 2006 13:39
> To: users at mapguide.osgeo.org
> Subject: [mapguide-users] SDF load problems
> 
> I am trying to load a 225 MB SDF file using MapGuide studio. I have 
> tried any number of different options and they all seem to fail. The 
> process crunches for a while, and then says "Communicating with server"
> and the progress bars fills to 100% .  Eventually, an error dialog is 
> popped that says "Unable to complete request to site. Connection lost".
> 
> The apache logs showed the following. None of the other logs reported 
> anything of interest that I could see.
> 
> [Wed Mar 22 13:33:54 2006] [warn] FastCGI: (dynamic) server 
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> restarted (pid 5284)
> 
> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: scheduled the restart of 
> the last (dynamic) server "C:/Program 
> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> process: reached dynamicMaxClassProcs (10)
> 
> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: (dynamic) server 
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> restarted (pid 6008)
> 
> [Wed Mar 22 13:34:21 2006] [warn] FastCGI: (dynamic) server 
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 5540) terminated with exit with status '0'
> 
> [Wed Mar 22 13:34:38 2006] [warn] FastCGI: (dynamic) server 
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 6008) termination signaled
> 
> [Wed Mar 22 13:34:39 2006] [warn] FastCGI: (dynamic) server 
> "C:/Program
Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
> (pid 6008) terminated with exit with status '0'
> 
> Anyone have any ideas about what is going on? 
> 
> Joel Carranza
> Gatekeeper Systems
> 626-449-3070 x114
> 
>  
> 
> 
>  
> 

--
View this message in context:
http://www.nabble.com/SDF-load-problems-tf1326833.html#a6582422
Sent from the OSGeo MapGuide mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at mapguide.osgeo.org
For additional commands, e-mail: users-help at mapguide.osgeo.org







More information about the Mapguide-users mailing list