<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Team: thank you all for working with us on this project. I just want to point out that we recently acquired our 10,000th scene. Looking forward to acquiring many many more, but just wanted to celebrate such a nice round number as we get started.
<div class=""><br class="">
</div>
<div class="">Thank you Frank for getting this going, and thank you Amit, for offering to help.</div>
<div class=""><br class="">
</div>
<div class="">Have a great weekend everyone, Jed.<br class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jan 23, 2015, at 1:03 PM, Amit Kapadia <<a href="mailto:amit@mapbox.com" class="">amit@mapbox.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hi Frank,
<div class=""><br class="">
</div>
<div class="">Thanks for all the hard work!</div>
<div class=""><br class="">
</div>
<div class="">We do have most of December and January scenes available on S3. They're in a private bucket, so we'll have to figure out the best way to transfer these data. One option is that I can run a copy from our bucket into the public bucket, and you can
take it from there. I'm open to other ideas. Our scenes expire after 50 days, so the December stash dwindles a little bit each day.</div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class="">Amit</div>
<div class=""><br class="">
</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Thu, Jan 22, 2015 at 1:23 PM, Chris Holmes <span dir="ltr" class="">
<<a href="mailto:cholmes@planet.com" target="_blank" class="">cholmes@planet.com</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr" class="">Nice work Frank!
<div class=""><br class="">
</div>
<div class="">Markdown in the github repo sounds ideal to me.</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">
<div class="">
<div class="h5">On Thu, Jan 22, 2015 at 12:20 PM, Frank Warmerdam <span dir="ltr" class="">
<<a href="mailto:warmerdam@pobox.com" target="_blank" class="">warmerdam@pobox.com</a>></span> wrote:<br class="">
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="">
<div class="h5">
<div dir="ltr" class="">Folks,
<div class=""><br class="">
</div>
<div class="">OK, I now have the pull from usgs and queue in S3 job running every two hours (for two hours) on our job system. At the beginning of each job it processes the previous run's queued scenes in parallel (usually 200-400), then builds index files,
then pulls scenes from usgs until the job times out (2 hour limit). </div>
<div class=""><br class="">
</div>
<div class="">So we should see the catalog fill out automatically, though the slowness and 503's of the USGS download service are pretty frustrating to me. Incidentally we try to run up to 6 of the puller jobs. They also now do 4 retries with exponential
backoff on 503's. </div>
<div class=""><br class="">
</div>
<div class="">If MapBox has the scene .gz files still sitting around for some of January, I'd be interested in us side-loading missing ones into the input queue. If Amit or something is interested in that we could do a little push on it in person now that we
(Planet Labs) are just down the street.</div>
<div class=""><br class="">
</div>
<div class="">Also, I added a small job (tree_index_maker.py) to build very simple index files so we can more easily walk the tree in the web browser. If anyone knows a way of auto-indexing instead of rebuilding these every two hours, I'd be interested in
hearing.</div>
<div class=""><br class="">
</div>
<div class="">See:</div>
<div class=""><br class="">
</div>
<div class=""> <a href="https://s3-us-west-2.amazonaws.com/landsat-pds/L8/index.html" target="_blank" class="">
https://s3-us-west-2.amazonaws.com/landsat-pds/L8/index.html</a></div>
<div class=""><br class="">
</div>
<div class="">Also, thanks to Charlie for the improved thumbnail generation PR, it has been merged and deployed though old ones aren't regenerated automatically.</div>
<div class=""><br class="">
</div>
<div class="">Lastly, I'd like us to setup a wiki or markup based web site for our effort when I can start documenting things and we can describe the effort. Should I just start writing markdown docs in the git repo for the ingestor? Or what?</div>
<div class=""><br class="">
</div>
<div class="">Best regards,</div>
<span class=""><font color="#888888" class="">
<div class="">-- <br class="">
<div class="">---------------------------------------+--------------------------------------<br class="">
I set the clouds in motion - turn up | Frank Warmerdam, <a href="mailto:warmerdam@pobox.com" target="_blank" class="">
warmerdam@pobox.com</a><br class="">
light and sound - activate the windows | <a href="http://pobox.com/~warmerdam" target="_blank" class="">
http://pobox.com/~warmerdam</a><br class="">
and watch the world go round - Rush | Geospatial Software Developer<br class="">
</div>
</div>
</font></span></div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
Landsat-pds mailing list<br class="">
<a href="mailto:Landsat-pds@lists.osgeo.org" target="_blank" class="">Landsat-pds@lists.osgeo.org</a><br class="">
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds" target="_blank" class="">http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
</div>
<br class="">
_______________________________________________<br class="">
Landsat-pds mailing list<br class="">
<a href="mailto:Landsat-pds@lists.osgeo.org" class="">Landsat-pds@lists.osgeo.org</a><br class="">
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds" target="_blank" class="">http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
</div>
_______________________________________________<br class="">
Landsat-pds mailing list<br class="">
<a href="mailto:Landsat-pds@lists.osgeo.org" class="">Landsat-pds@lists.osgeo.org</a><br class="">
http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</body>
</html>