file size issue

[Julien-Samuel Lacroix]

Hi,

I think what you are trying to do will be much more efficient if you 
"tile" your shapefile instead of putting all your data in MySQL. The 
tiling separate your big shape into multiple smaller one. When it's time 
to draw them, mapserver will only read the one it needs. Check both:

http://mapserver.gis.umn.edu/data2/wilma/mapserver-users/0303/msg00885.html

http://mapserver.gis.umn.edu/cgi-bin/wiki.pl?Tile4ms

You can also get tips for data optimization from here:
http://ms.gis.umn.edu/new_users/index_html#data-optimization

Julien

On 06/16/2005 09:41 AM, Jerl Simpson wrote:
> Hello,
> 
> I have a particular issue with file size limits.  I have a layer (2
> actually) that contains roads classified as "Local Roads" throughout
> the entire contiguous 48 US states.  I had to split this up into 2
> files due to filesize limits.  My apache server keeps throwing errors
> that say "[notice] child pid 12855 exit signal File size limit
> exceeded (25)".  So I would like to add these to a MySQL table and
> pull the data from there.
> 
> I am running into two problems.
> 
> 1.  I used one of my very small shape files to setup an SQL connection
> to grab the data.  When I modified my .map file I now get an internal
> server error.  Which simply means some data was spit out before HTTP
> headers terminated.  I have a log file and DEBUG ON set in my map file
> and nothing gets placed in the file.  Is there any documentation that
> tells one how to build the queries?  I can't seem to find them.
> 
> 2.  The shape file I have for Local Roads is 1.2Gb.  And the perl
> Shapelib module reads the entire shape file into an array.  Well, I
> don't  have 1.2Gb of memory on my machine available so when I run
> "shp2mysql.pl LocalRoad" the program terminates with the error
> "Terminated".  I looked at the documentation and I don't see a way to
> just grab records out of the file.  I tried get_record() but that
> threw error messages.
> 
> I have a tiled this file into 100 smaller files the largest of which
> is 63Mb.  I could write a script to run to shp2mysql.pl on each file. 
> I would rather have a program to operate on the original file though,
> as I'm afraid I'll end up with 100 seperate tables with the same data.
>  Or I would need to modify the script to foce them all into the same
> table.
> 
> I appreciate any help,
> 
> jerl
> 
> 

-- 
------------------------------------------------------------
Julien-Samuel Lacroix            jlacroix at dmsolutions.ca
DM Solutions Group               http://www.dmsolutions.ca/
------------------------------------------------------------