<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
I’ve uploaded the file to <a href="https://s3-us-west-2.amazonaws.com/awsopendata/test/testfile2.json" class="">https://s3-us-west-2.amazonaws.com/awsopendata/test/testfile2.json</a>
<div class=""><br class="">
</div>
<div class="">The mime type is “application/json” which should be appropriate.</div>
<div class=""><br class="">
</div>
<div class="">I’m not sure where this would fit within the landsat_ingestor scripts, but setting the mime type with boto is as simple as:</div>
<div class=""><br class="">
</div>
<div class="">
<div class=""> if s3_path.endswith(‘.json'):</div>
<div class=""> key.content_type = ‘application/json'</div>
</div>
<div class=""><br class="">
</div>
<div class="">Re: converting the numbers, Jeff, could you share the script you’ve written to create the json files? Perhaps someone will be able to see a simple way to make the conversion.</div>
<div class=""><br class="">
</div>
<div class="">Jed.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 26, 2015, at 10:55 AM, Frank Warmerdam <<a href="mailto:warmerdam@pobox.com" class="">warmerdam@pobox.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
Jeff,
<div class=""><br class="">
</div>
<div class="">From an attachment I (don't think) I can know if mime types are getting properly attached in S3. If you can point me to an object in S3 I can try and check. I see the AWS S3 browser shows the mimetype under the "Metadata" properties section
if you have access to that for our bucket. </div>
<div class=""><br class="">
</div>
<div class="">I'll second Charlie's suggestion to preserve numeric types for numeric fields. I don't know this gets obscured by mtlutils. </div>
<div class=""><br class="">
</div>
<div class="">Best regards,</div>
<div class="">Frank</div>
</div>
<div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br class="">
<div class="gmail_quote">On Thu, Feb 26, 2015 at 7:38 AM, Layton, Jeff<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:jefflay@amazon.com" target="_blank" class="">jefflay@amazon.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class="">
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">
<div lang="EN-US" link="blue" vlink="purple" class="">
<div class="">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Frank,<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">I think I’ve added the mimetype correctly.Can you look at the attached file and let me know if I did?<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Thanks!<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Jeff<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><b class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">From:</span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="Apple-converted-space"> </span><a href="mailto:fwarmerdam@gmail.com" target="_blank" class="">fwarmerdam@gmail.com</a><span class="Apple-converted-space"> </span>[mailto:<a href="mailto:fwarmerdam@gmail.com" target="_blank" class="">fwarmerdam@gmail.com</a>]<span class="Apple-converted-space"> </span><b class="">On
Behalf Of<span class="Apple-converted-space"> </span></b>Frank Warmerdam<br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Wednesday, February 25, 2015 5:20 PM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Layton, Jeff<br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:landsat-pds@lists.osgeo.org" target="_blank" class="">landsat-pds@lists.osgeo.org</a><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [Landsat-pds] Converting MTL.txt to JSON<u class=""></u><u class=""></u></span></p>
<div class="">
<div class="h5">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<div class="">
<p class="MsoNormal">Jeff,<u class=""></u><u class=""></u></p>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">This looks good to me. I'd suggest we drop these in the same directory as the MTL txt files, and just name them .json. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">eg.<u class=""></u><u class=""></u></p>
</div>
<div class="">
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B4.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B10.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B5.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B2.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B9.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B1.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B7.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B8.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B11.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B3.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_BQA.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_B6.TIF<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_MTL.txt<u class=""></u><u class=""></u></p>
</div>
</div>
<div class="">
<p class="MsoNormal">LC80150252015005LGN00_MTL.json<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">It might be nice to have the mimetype set properly for json too.<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">With regard to putting them all in one place, the downside I see to that is they aren't as likely to be noticed by folks who have maneuvered to a scene. The scene_list.gz file already has the links to each scene directory for folks who
need a directory-in-a-file.<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">Best regards,<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">Frank<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<div class="">
<p class="MsoNormal">On Wed, Feb 25, 2015 at 1:13 PM, Layton, Jeff <<a href="mailto:jefflay@amazon.com" target="_blank" class="">jefflay@amazon.com</a>> wrote:<u class=""></u><u class=""></u></p>
<blockquote style="border-style: none none none solid; border-left-color: rgb(204, 204, 204); border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;" class="">
<div class="">
<div class="">
<p class="MsoNormal">Hello everyone,<u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
<p class="MsoNormal">I just joined the mailing list and I’m working with Jed from AWS (I work for AWS as well). The first project I did was write a simple Python tool to convert the MTL.txt files to JSON. I’m attaching a sample MTL file and the resulting JSON.
Let me know what you think!<u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
<p class="MsoNormal">Thanks!<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><br class="">
Jeff<u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
<p class="MsoNormal">Dr. Jeff Layton<u class=""></u><u class=""></u></p>
<p class="MsoNormal">Solutions Architect – HPC<u class=""></u><u class=""></u></p>
<p class="MsoNormal">Amazon Web Services<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><a href="mailto:jefflay@amazon.com" target="_blank" class=""><span style="color: rgb(5, 99, 193);" class="">jefflay@amazon.com</span></a><u class=""></u><u class=""></u></p>
<p class="MsoNormal"><a href="tel:678-310-3685" target="_blank" class="">678-310-3685</a><u class=""></u><u class=""></u></p>
<p class="MsoNormal"> <u class=""></u><u class=""></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><br class="">
_______________________________________________<br class="">
Landsat-pds mailing list<br class="">
<a href="mailto:Landsat-pds@lists.osgeo.org" target="_blank" class="">Landsat-pds@lists.osgeo.org</a><br class="">
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds" target="_blank" class="">http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds</a><u class=""></u><u class=""></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br class="">
<br clear="all" class="">
<u class=""></u><u class=""></u></p>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<p class="MsoNormal">--<span class="Apple-converted-space"> </span><u class=""></u><u class=""></u></p>
<div class="">
<p class="MsoNormal">---------------------------------------+--------------------------------------<br class="">
I set the clouds in motion - turn up | Frank Warmerdam,<span class="Apple-converted-space"> </span><a href="mailto:warmerdam@pobox.com" target="_blank" class="">warmerdam@pobox.com</a><br class="">
light and sound - activate the windows |<span class="Apple-converted-space"> </span><a href="http://pobox.com/~warmerdam" target="_blank" class="">http://pobox.com/~warmerdam</a><br class="">
and watch the world go round - Rush | Geospatial Software Developer<u class=""></u><u class=""></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<br clear="all" class="">
<div class=""><br class="">
</div>
--<span class="Apple-converted-space"> </span><br class="">
<div class="gmail_signature">---------------------------------------+--------------------------------------<br class="">
I set the clouds in motion - turn up | Frank Warmerdam,<span class="Apple-converted-space"> </span><a href="mailto:warmerdam@pobox.com" target="_blank" class="">warmerdam@pobox.com</a><br class="">
light and sound - activate the windows |<span class="Apple-converted-space"> </span><a href="http://pobox.com/~warmerdam" target="_blank" class="">http://pobox.com/~warmerdam</a><br class="">
and watch the world go round - Rush | Geospatial Software Developer<br class="">
</div>
</div>
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Landsat-pds
mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="mailto:Landsat-pds@lists.osgeo.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Landsat-pds@lists.osgeo.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds</a></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>