SDF load problems
Jim O'Leary
joleary.public at gmail.com
Sat Sep 30 16:16:18 EDT 2006
I found a simple one that does the job:
<?xml version="1.0" encoding="utf-8"?>
<FeatureSource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xsi:noNamespaceSchemaLocation="FeatureSource-1.0.0.xsd">
<Provider>OSGeo.SHP.3.0</Provider>
<Parameter>
<Name>DefaultFileLocation</Name>
<Value>C:\Program
Files\MapGuideOpenSource\Server\Repositories\Library\myData\fiss_coho_clipped.shp</Value>
</Parameter>
</FeatureSource>
Jim O wrote:
>
> Did you attach a file with this posting? If so, it didn't make it. Could
> you post the XML? Thanks.
>
>
>
> Traian Stanev wrote:
>>
>>
>> There is a way around copying the data files to the repository. You just
>> need to manually publish the feature source to the MapGuide repository.
>> Here is how:
>>
>> * Navigate a browser to the html index of the mapguide. For me this is
>> http://localhost:8008/mapguide/mapagent/index.html -- you should see
>> something like the attached SetResource.png.
>>
>> * In the upper left frame, click Resource Service
>>
>> * In the lower left frame click SetResource. We will call the
>> SetResource API to publish our custom feature source that points to an
>> SDF or SHP somewhere on disk.
>>
>> * If you want to publish an SDF file, open the attached
>> FeatureSourceContent_SDF.txt, locate the string %MG_DATA_FILE_PATH% and
>> replace it by the location of your file on the server, e.g. D:\my
>> data\sdf files\boundaries.sdf. Save the txt file.
>>
>> * Back to the html page, in the "Content:" field, give the location of
>> the text file you just modified.
>>
>> * In the "ResourceID:" field, you must specify a valid resource ID. In
>> general those look very much like the tree you see in the left pane of
>> MG Studio with the string Library:// prepended to them. In our case the
>> ID has to end with ".FeatureSource" since we are defining a feature
>> source. An example of a valid ID is :
>> Library://Test/boundaries.FeatureSource
>>
>> * Leave the Header field blank and click on Submit.
>>
>> If you refresh your Studio, you will now see the feature source you just
>> published. You can now create a new layer in Studio and point the layer
>> to this feature source. Everything else should work as before.
>>
>> Note that this works for SDF 3.0+ only, not for 2.0. For SHP files, the
>> only difference is that you need to replace the %MG_DATA_FILE_PATH% by
>> the directory where your SHP/DBF files are, and not by the full file
>> path. I also verified that deleting the feature source from Studio will
>> NOT delete the SDF file that we manually pointed to. In general you can
>> use the SetResource page to publish other data to the repository, like
>> maps and layer definitions. You can also download resources using the
>> GetResourceContent page, which is how I got the attached sample feature
>> sources.
>>
>> Traian
>>
>>
>>
>>
>> _____
>>
>> From: Andy Morsell [mailto:amorsell at spatialgis.com]
>> Sent: Wednesday, March 22, 2006 8:17 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>>
>> Yes, I have done exactly that in the past to see where my files were
>> copied to in the repository. I guess my big problem is that there is no
>> way around copying our data from the place where it is maintained to the
>> repository in order for it to be accessible as a resource. Allowing a
>> resource to point to a different physical location would make data
>> management quite a bit easier for us and we wouldn't have to worry about
>> reversing out the actual physical path to that resource and then
>> incorporating that into our batch routines. This would also totally
>> eliminate the need to load these resources at all via HTTP or as
>> packages.
>>
>> Andy
>> _____
>>
>> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
>> Sent: Wednesday, March 22, 2006 4:07 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>>
>>
>> Regarding "This would be OK if there was an easy way to know where in
>> the repository that feature source is created when it is initially
>> defined.", do this experiment:
>>
>>
>>
>> 1) Name your source data something unique (ex: myUniqueFile.sdf)
>>
>> 2) Load it using Studio.
>>
>> 3) On the server, recursively search the Repository folder under
>> the server installation for the data file (ex: myUniqueFile.sdf).
>>
>>
>>
>> As long as you don't change the feature source wrapping the data file,
>> the location of the resource data file on the server won't change. You
>> can directly update the data file there on the server in the future.
>>
>>
>>
>> (DISCLAIMER: This hack bypasses the API. If the resource repository
>> implementation changes, this might break the hack. Be warned. )
>>
>>
>>
>> Also, to get a feel for the repository API, try using the html forms
>> from in your browser
>> (http://<server_name>/mapguide/mapagent/index.html). It's an easy way
>> get a look into the resources being created with the Studio load
>> procedures.
>>
>>
>>
>> Ed
>>
>>
>>
>>
>>
>> _____
>>
>> From: Andy Morsell [mailto:amorsell at spatialgis.com]
>> Sent: Wednesday, March 22, 2006 6:43 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>>
>>
>> I haven't tried it. I'm sure it would work fine, but it seems like an
>> awful lot of work to take a Shape file and copy it from one network
>> location to another (the repository)......... And what about those 1 GB
>> Shape files? It's a lot of package and data shuffling to keep your
>> repository data in synch with the source data. I would like to be able
>> to point the repository location to reference the source data in my
>> chosen network location. Sort of like MapGuide 6.5 UDL files, but you
>> could still retain the managed data aspect.
>>
>>
>>
>> In the case of converting your source to SDF+, some of us would rather
>> use FME scripts to update the SDF files in the repository. This would
>> be OK if there was an easy way to know where in the repository that
>> feature source is created when it is initially defined.
>>
>> Andy
>>
>> _____
>>
>> From: Edward Zimmerman [mailto:edward.zimmerman at autodesk.com]
>> Sent: Wednesday, March 22, 2006 3:29 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>> Regarding "Many of us will want to update our data through external
>> processes", has anyone tried to write some code which does the
>> following?
>>
>>
>>
>> 1) Use the Studio Load API to create the package locally from the
>> latest source data.
>>
>> 2) Copy the resulting package file into the Package directory on
>> the server.
>>
>> 3) Use the MGOS Admin service API to apply this updated package.
>>
>>
>>
>> Would this meet most of your needs to update the source flat file data
>> wrapped by FeatureSource resources?
>>
>>
>>
>> Thanks,
>>
>> Ed
>>
>>
>>
>> _____
>>
>> From: Andy Morsell [mailto:amorsell at spatialgis.com]
>> Sent: Wednesday, March 22, 2006 6:03 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>>
>>
>> Sorry, I should have clarified that my rant pertains to flat files only.
>> Database connections will be direct to their source.
>>
>>
>>
>> Andy
>>
>> _____
>>
>> From: Butler, Juan P [mailto:jpbutler at co.pinellas.fl.us]
>> Sent: Wednesday, March 22, 2006 2:46 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>> Wow. So you're saying that if and when a provider for Oracle Spatial
>> comes along, we won't be able to define the connection to the database
>> and have the data come directly from that connection? It basically
>> dumps the data (that point in time) into a repository that resides in on
>> the MG server?
>>
>>
>>
>> Can someone from Autodesk comment on this?
>>
>>
>>
>> Juan Butler - Senior Programmer/Analyst
>> GIS Applications
>> Pinellas County Information Systems
>> jpbutler at pinellascounty.org
>>
>>
>>
>>
>>
>> _____
>>
>> From: Andy Morsell [mailto:amorsell at spatialgis.com]
>> Sent: Wednesday, March 22, 2006 5:37 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>>
>>
>> This is where using a managed data structure is a detriment, in my
>> opinion. Many of us will want to update our data through external
>> processes and there is no easy way to just copy files into the
>> repository or point to an existing network location. I think this is
>> going to become a much bigger problem as people put these systems into
>> production and realize that they have to "load" their data into
>> repositories. It's also going to be a large problem for keeping data up
>> to date with ongoing system maintenance. Scheduled process are going to
>> be a necessity to ensure that the data in the repository is in synch
>> with the source data. And, in the case of large source datasets, the
>> entire dataset will have to be reloaded as there is no incremental
>> update method.
>>
>>
>>
>> I believe (but have not yet done this) that the Studio API is going to
>> have to be used to load these resources, not the server API.
>>
>>
>>
>> Andy Morsell, P.E.
>> Spatial Integrators, Inc.
>> http://www.SpatialGIS.com
>>
>> _____
>>
>> From: Joel Carranza [mailto:jec at gatekeeper.com]
>> Sent: Wednesday, March 22, 2006 2:17 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>> Ahhh. I see, I misunderstood the use of "Packages" and didn't think too
>> look there. Thank you. It is working fine now.
>>
>>
>>
>> I have noticed that imported SDFs are processed (into SDF v3, I
>> believe). Is it currently possible to import SDF data sources into the
>> repository without using Studio at all, just using the server API? Is it
>> possible to create your own Packages using the server API? I would be
>> interested in doing this for our migration tools.
>>
>>
>>
>> Joel Carranza
>> Gatekeeper Systems
>> 626-449-3070 x114
>>
>> _____
>>
>> From: Jason Birch [mailto:Jason.Birch at nanaimo.ca]
>> Sent: Wednesday, March 22, 2006 1:42 PM
>> To: users at mapguide.osgeo.org
>> Subject: RE: [mapguide-users] SDF load problems
>>
>> It looks like it's too big. You'll probably have to "Package" the data
>> and upload it to the server manually, then bring it in via the site
>> Admin web app.
>>
>>
>>
>> Jason
>>
>>
>>
>> _____
>>
>> From: Joel Carranza [mailto:jec at gatekeeper.com]
>> Sent: Wednesday, March 22, 2006 13:39
>> To: users at mapguide.osgeo.org
>> Subject: [mapguide-users] SDF load problems
>>
>> I am trying to load a 225 MB SDF file using MapGuide studio. I have
>> tried any number of different options and they all seem to fail. The
>> process crunches for a while, and then says "Communicating with server"
>> and the progress bars fills to 100% . Eventually, an error dialog is
>> popped that says "Unable to complete request to site. Connection lost".
>>
>> The apache logs showed the following. None of the other logs reported
>> anything of interest that I could see.
>>
>> [Wed Mar 22 13:33:54 2006] [warn] FastCGI: (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> restarted (pid 5284)
>>
>> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: scheduled the restart of the
>> last (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> process: reached dynamicMaxClassProcs (10)
>>
>> [Wed Mar 22 13:34:00 2006] [warn] FastCGI: (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> restarted (pid 6008)
>>
>> [Wed Mar 22 13:34:21 2006] [warn] FastCGI: (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> (pid 5540) terminated with exit with status '0'
>>
>> [Wed Mar 22 13:34:38 2006] [warn] FastCGI: (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> (pid 6008) termination signaled
>>
>> [Wed Mar 22 13:34:39 2006] [warn] FastCGI: (dynamic) server "C:/Program
>> Files/MapGuideOpenSource/WebServerExtensions/www/mapagent/MapAgent.exe"
>> (pid 6008) terminated with exit with status '0'
>>
>> Anyone have any ideas about what is going on?
>>
>> Joel Carranza
>> Gatekeeper Systems
>> 626-449-3070 x114
>>
>>
>>
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/SDF-load-problems-tf1326833.html#a6582705
Sent from the OSGeo MapGuide mailing list archive at Nabble.com.
More information about the Mapguide-users
mailing list