[mapserver-users] Best way to do a batch reprojection (onwindows)

Wed Aug 7 15:20:23 PDT 2013

Thanks for the responses.  I kicked off a batch Convert in Global Mapper,
converting the 3482 files in the Alabama directory.  At least this way I get
some visual feedback.  It says it will be done on Friday and 1330.  It's
currently Wednesday at 1615.  That's not bad.  The data is on three USB
drives, and we have 3 licenses for GM, so I should be able to get all the
data projected in a week or two once I work out a good procedure.
James

-----Original Message-----
From: mapserver-users-bounces at lists.osgeo.org
[mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of andy
Sent: Wednesday, August 07, 2013 2:45 PM
To: mapserver-users at lists.osgeo.org
Subject: Re: [mapserver-users] Best way to do a batch reprojection
(onwindows)

On 8/7/2013 3:35 PM, James_in_Utah wrote:
> Hi,
> I have the NAIP imagery for CONUS, and it's rather large.  About 4.2TBs,
of
> JP2 files.  It's projected in 10 UTM zones across the country.  If I'm
going
> to offer a single layer with Mapserver, I think I have to reproject all of
> the files into a single projection, say WGS84.  I'm trying to figure out a
> good way to do that.  GlobalMapper seems to have this function.  I loaded
up
> all the data from Alabama, and told it to convert the projection.  It's
been
> working at it for about 8 hours and I see no sign of progress.  I tried
> creating a batch file to call GdalWarp, but for some reason that's getting
> an error.  Here's the line from my batch:
>
> for %%f in (*.jp2) do gdalwarp -t_srs EPSG:4326 %%f c:\out\%%f
>
> The error is saying 4326 can't be found in the GSC.CSV, even though it is
> plainly in there.  Even if I get this working, this wiill still be very
> tedious.  The data is divide up by states, and under the state directory,
> there are dozens of subdirectories.  I would need some sort of script to
> walk through the directories.
>
> If anyone has a suggestion on how to efficiently reproject this large
amount
> of data it would be greatly appreciated!
> Thanks,
> James
>

I use GlobalMapper.  Never had 4TB though.  I usually convert to .ecw, 
which I'd guess would be smaller than jpeg.  Its not fast, and it has 
refresh problems, but it works fine.  I've had jobs run 12 hours or so, 
I think 30GB'ish is about the most I've done.

Also, if you use the map catalog thing, it'll use less memory, and might 
be quicker.  (File -> create new map catalog).  The catalog will let it 
selective load chunks of data when its needed.  This is useful when you 
have lots and lots of little files.

-Andy

_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6912 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20130807/40963fdf/attachment.bin>