using Mapserver for on-the-fly raster data?

[Jeff de La Beaujardiere]

I have a set of huge raw binary datafiles containing output from
someone's numerical simulation (6GB/file, 4files/day). Each file
contains values for a variety of atmospheric quantities, each at 20
timesteps, and some at multiple elevations. I would like to offer
access to at least some of the quantities via WMS protocol using
Mapserver.

The brute force way works: I can pre-generate PNG or GeoTIFF images of
selected quantities, and then generate a mapfile that includes all the
images. The obvious disadvantage is that I will have many files laying
around whether they ever get used or not.

I would like a more elegant approach, wherein Mapserver's WMS service
metadata can advertise what quantities are available, but the data are
extracted from the 6GB model file only upon request. Is it possible to
do something like this? I see a few mentions of database connections
in the Mapserver documentation but must admit that I am somewhat
mystified.

Thanks for any help,
Jeff DLB