Problem trying to serve large .tif dataset

[Glen Thompson]

Hi,

I am having difficulties with mapserver 4.6 generating output with a
large .tif dataset.  I'm attempting to setup a large dataset to be served
up via WMS.

I have a .tif dataset of approximately 2000 .tif files totalling around
160k pixels by 375k pixels.  Each tif image is 5000 x 5000 pixels.

My workflow is as follows:

create overviews for each .tif image.
for each image in dataset
   addtiffo "file.tif" 2 4 8 16 32 64

Now, I have used gdaltindex to create a shapefile containing all of
the .tif files for quickly identfying which tiles need to be accessed for
a given bbox.

gdaltindex shapefile.shp "path_to_tiffs/*.tif"

So far, so good.

I'm having good luck using getMap requests to retrieve subsets of the
entire bbox.  A request for a bbox that is the extent for 1 tile works
great, as do larger bbox's.  Based on the amount of data read by the
mapserver.exe process, I can tell that it's using the overviews properly
and not reading the entire source .tif file(s).

When I request the entire bbox for the dataset, the mapserver.exe process
runs for about 8 minutes and then seems to crash or error out or
something.  I don't know nor can't find any logfiles to see what is
happening.

Has anyone had any success dealing with datasets of this size?  If so, how
have you implemented them?

Any help/advise would be greatly appreciated.

Thank you,
Glen Thompson