[mapserver-users] Best way to import 4.5TB of imagery?

Mon Jun 10 09:57:46 PDT 2013

Hi Stephen,
Thanks, for the reply.  I previously got 4 sample images from the USDA, and
was able to get them to work just fine.  There was no processing required.
The sample images I got were all from Utah, and they are NAD83, UTM zone 12.
I added the 4 sample images to a shape file using gdaltindex.   I used UPSG
26912 and mapserver served them up very quickly for such large files.  

Now I have this entire data set, and it stretches from UTM zone 10, to UTM
zone 19.  The data is divided into directories by two letter state
abbreviations, and under that into subdirectories.  I'm just wondering how
to add this to my mapfile.  Do I need a different entry for each UTM Zone?
How is it possible to get a single layer entry that includes multiple
projections?  This is looking like a huge job and I just want to know the
best strategy for getting this done.
Thanks,
James

-----Original Message-----
From: mapserver-users-bounces at lists.osgeo.org
[mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of Stephen
Woodbridge
Sent: Friday, June 07, 2013 8:41 AM
To: mapserver-users at lists.osgeo.org
Subject: Re: [mapserver-users] Best way to import 4.5TB of imagery?

On 6/7/2013 10:31 AM, James_in_Utah wrote:
> Hi,
> We just got 3 hard drive, loaded with 4.5TB of NAIP imagery for all of 
> CONUS.  I think there's a total of about 400,000 jpgs.  The data is in 
> directories, by states.  Under each state, there are subfolders, 
> probably reference by longitude.  Other than going through folder by 
> folder, adding each image to a shape file using gdaltindex, what's the 
> best strategy for loading a couple of hundred thousand files up to our 
> server and making the imagery available via our mapserver?  Should I 
> maintain the current directory structure when I copy the imagery to 
> the server, or just dump all of it into a single directory?  Do I want 
> to stay with 1 shape file, or break it up by state?  We eventually 
> want a contiguous layer for all of CONUS to be served up to our users.

James,

Since imagery data is served via gdal, you might want to also ask this
question on the gdal list.

There are issues with jpg related to the fact that if you only want a small
part of the image you still have to uncompress the whole image. So part of
the answer might be that you need to pre-process all the imagery into
something like a jpg compress tiled geotif or something else.

You also need to consider what projection your imagery is in and what
projection you want to display it in. Because if you need to preprocess the
data, that would also be a good time to reproject it.

Anyway the gdal list can probably ask additional questions to help sort all
that out.

-Steve W

_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6912 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20130610/96a3b5f1/attachment.bin>