[Landsat-pds] Converting MTL.txt to JSON

Thu Feb 26 09:34:47 PST 2015

Jeff,

I'm not sure what scene that was generated for, but I don't see any values
that are strings. For instance, I expect the PRODUCT_METADATA group to have
an item called DATA_TYPE with a string value (ie. "L1T").

Also, the whole METADATA_FILE_INFO section appears to be missing.

I am suspecting you lost everything that failed to translate to numbers?

Also, I see now that you have an content-type set *within* the json.  I'm
not sure what the utility of that is as opposed to setting it on the S3
object as Jed was discussing.

Best regards,
Frank

On Thu, Feb 26, 2015 at 9:20 AM, Layton, Jeff <jefflay at amazon.com> wrote:

>  Everyone,
>
>
>
> Try the attached JSON file. It should have everything numeric that can be
> numerical J  Let me know if you find any errors (my cursory examination
> didn’t find any). If it passes muster, I’ll fire off the code.
>
>
>
> Thanks!
>
>
>
> Jeff
>
>
>
>
>
>
>
> *From:* Sundwall, Jed
> *Sent:* Thursday, February 26, 2015 11:25 AM
> *To:* Frank Warmerdam
> *Cc:* Layton, Jeff; landsat-pds at lists.osgeo.org
>
> *Subject:* Re: [Landsat-pds] Converting MTL.txt to JSON
>
>
>
> I’ve uploaded the file to
> https://s3-us-west-2.amazonaws.com/awsopendata/test/testfile2.json
>
>
>
> The mime type is “application/json” which should be appropriate.
>
>
>
> I’m not sure where this would fit within the landsat_ingestor scripts, but
> setting the mime type with boto is as simple as:
>
>
>
>  if s3_path.endswith(‘.json'):
>
>          key.content_type = ‘application/json'
>
>
>
> Re: converting the numbers, Jeff, could you share the script you’ve
> written to create the json files? Perhaps someone will be able to see a
> simple way to make the conversion.
>
>
>
> Jed.
>
>
>
>
>
>
>
>  On Feb 26, 2015, at 10:55 AM, Frank Warmerdam <warmerdam at pobox.com>
> wrote:
>
>
>
> Jeff,
>
>
>
> From an attachment I (don't think) I can know if mime types are getting
> properly attached in S3.  If you can point me to an object in S3 I can try
> and check.  I see the AWS S3 browser shows the mimetype under the
> "Metadata" properties section if you have access to that for our bucket.
>
>
>
> I'll second Charlie's suggestion to preserve numeric types for numeric
> fields.  I don't know this gets obscured by mtlutils.
>
>
>
> Best regards,
>
> Frank
>
>
>
> On Thu, Feb 26, 2015 at 7:38 AM, Layton, Jeff <jefflay at amazon.com> wrote:
>
>  Frank,
>
>
>
> I think I’ve added the mimetype correctly.Can you look at the attached
> file and let me know if I did?
>
>
>
> Thanks!
>
>
>
> Jeff
>
>
>
>
>
>
>
> *From:* fwarmerdam at gmail.com [mailto:fwarmerdam at gmail.com] *On Behalf Of *Frank
> Warmerdam
> *Sent:* Wednesday, February 25, 2015 5:20 PM
> *To:* Layton, Jeff
> *Cc:* landsat-pds at lists.osgeo.org
> *Subject:* Re: [Landsat-pds] Converting MTL.txt to JSON
>
>
>
> Jeff,
>
>
>
> This looks good to me.  I'd suggest we drop these in the same directory as
> the MTL txt files, and just name them .json.
>
>
>
> eg.
>
> LC80150252015005LGN00_B4.TIF
>
> LC80150252015005LGN00_B10.TIF
>
> LC80150252015005LGN00_B5.TIF
>
> LC80150252015005LGN00_B2.TIF
>
> LC80150252015005LGN00_B9.TIF
>
> LC80150252015005LGN00_B1.TIF
>
> LC80150252015005LGN00_B7.TIF
>
> LC80150252015005LGN00_B8.TIF
>
> LC80150252015005LGN00_B11.TIF
>
> LC80150252015005LGN00_B3.TIF
>
> LC80150252015005LGN00_BQA.TIF
>
> LC80150252015005LGN00_B6.TIF
>
> LC80150252015005LGN00_MTL.txt
>
> LC80150252015005LGN00_MTL.json
>
>
>
> It might be nice to have the mimetype set properly for json too.
>
>
>
> With regard to putting them all in one place, the downside I see to that
> is they aren't as likely to be noticed by folks who have maneuvered to a
> scene.  The scene_list.gz file already has the links to each scene
> directory for folks who need a directory-in-a-file.
>
>
>
> Best regards,
>
> Frank
>
>
>
>
>
> On Wed, Feb 25, 2015 at 1:13 PM, Layton, Jeff <jefflay at amazon.com> wrote:
>
>  Hello everyone,
>
>
>
> I just joined the mailing list and I’m working with Jed from AWS (I work
> for AWS as well). The first project I did was write a simple Python tool to
> convert the MTL.txt files to JSON. I’m attaching a sample MTL file and the
> resulting JSON. Let me know what you think!
>
>
>
> Thanks!
>
>
> Jeff
>
>
>
>
>
>
>
> Dr. Jeff Layton
>
> Solutions Architect – HPC
>
> Amazon Web Services
>
> jefflay at amazon.com
>
> 678-310-3685
>
>
>
>
> _______________________________________________
> Landsat-pds mailing list
> Landsat-pds at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds
>
>
>
>
>
> --
>
>
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Software Developer
>
>
>
>
>
> --
>
>
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Software Developer
>
> _______________________________________________
> Landsat-pds mailing list
> Landsat-pds at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/landsat-pds
>
>
>

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/landsat-pds/attachments/20150226/d57bf749/attachment-0001.html>