[Live-demo] Provide .fix files for all Shapefiles
Cameron Shorter
cameron.shorter at gmail.com
Wed Aug 11 07:18:42 EDT 2010
Please go ahead and apply this patch.
Please also run the install_gisdata.sh script, and confirm that it works
as expected, and that the other applications like UDig work.
On 11/08/10 19:24, Stefan A. Tzeggai wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi
>
> Jody wrote:
> JG> That is great you got a command line tool going (indeed that was very
> JG> quick). Updating the script (perhaps the atlas styler script) to
> JG> create the .fix files is the best option today.
>
>
> Accoring to bin/main.sh the bin/install_gisdata.sh is executed after
> bin/install_atlasstyler.sh, so the following patch is open for comments.
>
>
>
> Index: install_gisdata.sh
> ===================================================================
> - --- install_gisdata.sh (revision 4603)
> +++ install_gisdata.sh (working copy)
> @@ -36,6 +36,13 @@
> exit 1
> fi
>
> +# check required tools are installed
> +if [ ! -x "`which atlasstyler`" ] ; then
> + echo "ERROR: atlasstyler is required as a tool to create .fix files
> for all shapefiles, please install it with bin/install_atlasstyler.sh
> and try again"
> + exit 1
> +fi
> +
> +
> # create tmp folders
> mkdir "$TMP"
> cd "$TMP"
> @@ -84,6 +91,13 @@
> unzip "$file" -d "$DATA_FOLDER/natural_earth"
> done
>
> +
> +# Add Geotools .fix files to all Shapefiles. Normally Geotools
> application would create these
> +# files when opeing the Shapefile, but since the data-dir is read-only,
> we do it here.
> +# This REQUIRES that install_atlasstyler.sh has been executed before
> (which is checked above)
> +find "$DATA_FOLDER/natural_earth" -iname "*.shp" -exec atlasstyler
> "addFix={}" \;
> +
> +
> chmod a+r "$DATA_FOLDER/natural_earth" ## read the data dir
> chmod 444 $DATA_FOLDER/natural_earth/* ## and all files in it
> chmod -R +X "$DATA_FOLDER/natural_earth" ## but keep x on directories
>
>
>
> Am 11.08.2010 01:47, schrieb Jody Garnett:
>
>> That is great you got a command line tool going (indeed that was very
>> quick). Updating the script (perhaps the atlas styler script) to
>> create the .fix files is the best option today.
>>
>> Small correction: It is especially annoying that this .fix index is
>> not even a spatial index - it is used to preserve row order when
>> editing. As a result the index files are very small.
>>
>> Can I also ask if there is a "qix" spatial index?
>>
>> (we will be getting that patch into GeoTools so the solution to
>> generate a fix file should be viewed as temporary and only needed to
>> get our live dvd release deadlines).
>>
>> Jody
>>
>> On Wed, Aug 11, 2010 at 2:38 AM, Stefan A. Tzeggai
>> <tzeggai at wikisquare.de> wrote:
>> Hi
>>
>> 1. good news: I debugged AtlasStyler and Geopublisher today a OSGeoLive
>> system and fixed all the problems reported.
>> AS and GP now have no more problems when opening a Shapefile without a
>> .fix spatial index from a read-only drive - it doesn't try to create the
>> index anymore.
>>
>> 2. The problem with read-only and .fix lies in Geotools, so it affects
>> uDig also.
>>
>> 3. I created a patch for Geotools
>> (http://jira.codehaus.org/browse/GEOT-3245) but it is not yet reviewed.
>>
>> Options are:
>>
>> A: Get the patch into Geotools and make a new release of Geotools and uDig
>>
>> Advantage: No big changes on the OSGeo side.
>>
>> Disadvantage: Lots of work for Jody. uDig, Geopublisher, AtlasStyler and
>> will not profit from a spatial index when working with the demo
>> shapefile data.
>>
>>
>> B: Create .fix indexes during build after the Shapefiles are unzipped
>> and while the drive is still read-write.
>>
>> .fix are Geotools-specific, so ogr doesn't create these files. We need a
>> command line tool to add .fix indexes if we want it to be done in
>> 'bin/loadgisdata.sh'. Such a tool doesn't exists yet... So i added a
>> command line option to AtlasStyler 1.5 that does exactly that.
>>
>> So for option B, we have to do only one thing: Add a line to
>> 'bin/install_atlasStyler' that will add a spatial index to all
>> shapefiles in the $DATA_FOLDER:
>>
>>>>> find $DATA_FOLDER -iname "*.shp" -exec atlasstyler "addFix={}" \;
>>>>>
>> This REQUIRES that 'bin/loadgisdata.sh' is run before
>> install_AtlasStyler.sh is run.
>>
>> Actually install_Gisdata is setting the data read-only already. So in
>> the install_AtlasStyler.sh i will have to change the permissions twice..
>> something not so nice.
>>
>> Advantage: No quick re-releasing of Geotools and uDig just for one patch
>> two days after the last release.
>>
>> Disadvantage: Some unorthodox modifications on the install scripts.
>>
>>
>>
>> A or B ? Waiting for feedback.
>>
>> Greetings,
>> Steve
>>
>>
>> Am 10.08.2010 13:33, schrieb Cameron Shorter:
>>
>>>>> The naturalearth data shapefiles are loaded by bin/loadgisdata.sh
>>>>>
>>>>> It is loaded from the naturalearth website directly.
>>>>>
>>>>> I'm open to you adding .fix files if you think it is easy enough to
>>>>> achieve.
>>>>>
>>>>> On 10/08/10 21:10, Stefan A. Tzeggai wrote:
>>>>> Hi
>>>>>
>>>>> I have installed the rc5 to my HDD and started investigating the
>>>>> problems reported for AtlasStyler and Geopublisher. This is what happens:
>>>>>
>>>>> The natural earth shapes don't have .fix spatial indexes. AtlasStyler
>>>>> tries to create them on-the-fly and gets a permission exception.
>>>>>
>>>>> I will write a fix, so that that will not be a problem anymore, but
>>>>> maybe we should anyways add .fix files to all Shapefiles on the DVD.
>>>>>
>>>>> .fix files are supported by Geotools, so uDig, Geoserver,
>>>>> Geopublisher and AtlasStyler (and probably more) would profit
>>>>> performance-wise.
>>>>>
>>>>> The data dir is not in the svn.. Tell me where it is, and i can provide
>>>>> .fix files for all shapefiles.
>>>>>
>>>>> Greetings,
>>>>> Steve
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- wiki² - Softwareentwicklung
>>>>> Stefan A. Tzeggai, geb. Krüger
>>>>> Straßburger Weg 26
>>>>> 53113 Bonn
>>>>>
>>>>> email tzeggai at wikisquare.de
>>>>> phone 0228 24 000 528
>>>>> mobile 0176 40 38 9559
>>>>> webpage wikisquare.de
>>>>> twitter http://twitter.com/geopublishing
>>>>> skype alfonx
>>>>>
>>>>> reclaim your net - http://tor.eff.org
>>>>> enforce privacy - http://www.pgpi.org
>>>>> pgp key id: 51B576FD - http://pgp.mit.edu
>>>>>
>>>>> Please note that according to the German law on data retention,
>>>>> information on every electronic information exchange with me is
>>>>> retained for a period of six months.
>>>>>
>> _______________________________________________
>> Live-demo mailing list
>> Live-demo at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/live-demo
>> http://wiki.osgeo.org/wiki/Live_GIS_Disc
>>
>>
>>>
> - --
> wiki² - Softwareentwicklung
> Stefan A. Tzeggai, geb. Krüger
> Straßburger Weg 26
> 53113 Bonn
>
> email tzeggai at wikisquare.de
> phone 0228 24 000 528
> mobile 0176 40 38 9559
> webpage wikisquare.de
> twitter http://twitter.com/geopublishing
> skype alfonx
>
> reclaim your net - http://tor.eff.org
> enforce privacy - http://www.pgpi.org
> pgp key id: 51B576FD - http://pgp.mit.edu
>
> Please note that according to the German law on data retention,
> information on every electronic information exchange with me is
> retained for a period of six months.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkxibFMACgkQdFDQR1G1dv2U/gCgl/P8IwiQVa/oOcDljFpLqy4e
> stYAnieZB74X1knG1nviCGf4UHapayZm
> =tsma
> -----END PGP SIGNATURE-----
> _______________________________________________
> Live-demo mailing list
> Live-demo at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/live-demo
> http://wiki.osgeo.org/wiki/Live_GIS_Disc
>
--
Cameron Shorter
Geospatial Director
Tel: +61 (0)2 8570 5050
Mob: +61 (0)419 142 254
Think Globally, Fix Locally
Geospatial Solutions enhanced with Open Standards and Open Source
http://www.lisasoft.com
More information about the Live-demo
mailing list