[Landsat-pds] Converting MTL.txt to JSON

[Layton, Jeff]

Give me a few minutes – I’m working on a conversion function now. I’ll send it out when it “mostly” works ☺

Jeff


From: Sundwall, Jed
Sent: Thursday, February 26, 2015 11:25 AM
To: Frank Warmerdam
Cc: Layton, Jeff; landsat-pds at lists.osgeo.org
Subject: Re: [Landsat-pds] Converting MTL.txt to JSON

I’ve uploaded the file to https://s3-us-west-2.amazonaws.com/awsopendata/test/testfile2.json

The mime type is “application/json” which should be appropriate.

I’m not sure where this would fit within the landsat_ingestor scripts, but setting the mime type with boto is as simple as:

 if s3_path.endswith(‘.json'):
         key.content_type = ‘application/json'

Re: converting the numbers, Jeff, could you share the script you’ve written to create the json files? Perhaps someone will be able to see a simple way to make the conversion.

Jed.



On Feb 26, 2015, at 10:55 AM, Frank Warmerdam <warmerdam at pobox.com<mailto:warmerdam at pobox.com>> wrote:

Jeff,

From an attachment I (don't think) I can know if mime types are getting properly attached in S3.  If you can point me to an object in S3 I can try and check.  I see the AWS S3 browser shows the mimetype under the "Metadata" properties section if you have access to that for our bucket.

I'll second Charlie's suggestion to preserve numeric types for numeric fields.  I don't know this gets obscured by mtlutils.

Best regards,
Frank

On Thu, Feb 26, 2015 at 7:38 AM, Layton, Jeff <jefflay at amazon.com<mailto:jefflay at amazon.com>> wrote:
Frank,

I think I’ve added the mimetype correctly.Can you look at the attached file and let me know if I did?

Thanks!

Jeff



From: fwarmerdam at gmail.com<mailto:fwarmerdam at gmail.com> [mailto:fwarmerdam at gmail.com<mailto:fwarmerdam at gmail.com>] On Behalf Of Frank Warmerdam
Sent: Wednesday, February 25, 2015 5:20 PM
To: Layton, Jeff
Cc: landsat-pds at lists.osgeo.org<mailto:landsat-pds at lists.osgeo.org>
Subject: Re: [Landsat-pds] Converting MTL.txt to JSON

Jeff,

This looks good to me.  I'd suggest we drop these in the same directory as the MTL txt files, and just name them .json.

eg.
LC80150252015005LGN00_B4.TIF
LC80150252015005LGN00_B10.TIF
LC80150252015005LGN00_B5.TIF
LC80150252015005LGN00_B2.TIF
LC80150252015005LGN00_B9.TIF
LC80150252015005LGN00_B1.TIF
LC80150252015005LGN00_B7.TIF
LC80150252015005LGN00_B8.TIF
LC80150252015005LGN00_B11.TIF
LC80150252015005LGN00_B3.TIF
LC80150252015005LGN00_BQA.TIF
LC80150252015005LGN00_B6.TIF
LC80150252015005LGN00_MTL.txt
LC80150252015005LGN00_MTL.json

It might be nice to have the mimetype set properly for json too.

With regard to putting them all in one place, the downside I see to that is they aren't as likely to be noticed by folks who have maneuvered to a scene.  The scene_list.gz file already has the links to each scene directory for folks who need a directory-in-a-file.

Best regards,
Frank


On Wed, Feb 25, 2015 at 1:13 PM, Layton, Jeff <jefflay at amazon.com<mailto:jefflay at amazon.com>> wrote:
Hello everyone,

I just joined the mailing list and I’m working with Jed from AWS (I work for AWS as well). The first project I did was write a simple Python tool to convert the MTL.txt files to JSON. I’m attaching a sample MTL file and the resulting JSON. Let me know what you think!

Thanks!

Jeff



Dr. Jeff Layton
Solutions Architect – HPC
Amazon Web Services
jefflay at amazon.com<mailto:jefflay at amazon.com>
678-310-3685<tel:678-310-3685>


_______________________________________________
Landsat-pds mailing list
Landsat-pds at lists.osgeo.org<mailto:Landsat-pds at lists.osgeo.org>
http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds



--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com<mailto:warmerdam at pobox.com>
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer



--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com<mailto:warmerdam at pobox.com>
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
_______________________________________________
Landsat-pds mailing list
Landsat-pds at lists.osgeo.org<mailto:Landsat-pds at lists.osgeo.org>
http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/landsat-pds/attachments/20150226/7d9b57c9/attachment-0001.html>