[Landsat-pds] Bringing GLS Scenes into s3://landsat-pds

[Sundwall, Jed]

We’re planning to add Landsat GLS data into s3://landsat-pds, and I’d like to validate our approach on organizing the files with the group, because organizing GLS data is slightly more complicated than L8 data.

# Directory Structure
The main complication of GLS data is that it’s divided into 5 “epochs” GLS1975, GLS1990, GLS2000, GLS2005, and GLS2010. There are variations in file names across epochs due to differences in sensors used, which are described thoroughly in the “GLS Collection” section of this page http://glcf.umd.edu/library/guide/namingconvention.shtml

My question is this: should we try to organize GLS data from all epochs according to the path/row structure that we’ve used for L8, or should we organize data from each epoch in its own directory, and then follow the path/row structure?

That is, do we list available scenes in a directory like this?

landsat-pds/gls/path/row/

or one like this?

landsat-pds/gls/epoch/path/row/

There’s something nice about making it easy to see all available data for a path/row within one directory, but I’m not sure if that will be the most useful organization for users of the data.

# Compression, tiling, and overlays

Does it make sense to apply the same in-file compression and internal tiling to GLS scenes? Should we also create overlays for the scenes?

# Creating the scripts

What scripts would we need for this and is there a best way for us to incorporate them into the ingestor?

Right now, we have l8_aws_config.py, l8_lib.py, l8_process_run.py, l8_process_scene.py, and l8_queue_tar.py scripts. Would we need the equivalent scripts for GLS data or is there a way we could rely on the existing scripts for most of the work, but apply slightly different rules for GLS scenes. However we go about it, it’d be nice to make this extensible so we could do the same thing for L7, L5, etc.

Any response to this would be very appreciated. As soon as we have a good plan, we can start bringing in the data!

Thanks, Jed.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/landsat-pds/attachments/20150416/24005022/attachment.html>