From bryce at agdeveloper.com Tue Dec 12 09:44:38 2017 From: bryce at agdeveloper.com (Bryce Johnston) Date: Tue, 12 Dec 2017 09:44:38 -0800 Subject: [Sentinel-pds] AWS Sentinel-2 - S2B product listings Message-ID: <1513100678.963297.1202703816.615DDD86@webmail.messagingengine.com> Hello, I had a question about if S2B products should be showing up when browsing http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/ ? For an example here is a tile from S2B product: http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#tiles/22/K/EC/2017/10/10/0/ , but when browsing http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/2017/10/10/ it only appears to list S2A products. Is there a way to get the S2B entries as well or am I overlooking something? Thanks, -- Bryce Johnston bryce at agdeveloper.com From grega.milcinski at sinergise.com Tue Dec 12 12:51:37 2017 From: grega.milcinski at sinergise.com (Grega Milcinski) Date: Tue, 12 Dec 2017 21:51:37 +0100 Subject: [Sentinel-pds] AWS Sentinel-2 - S2B product listings In-Reply-To: <1513111489.1043869.1202914832.38122389@webmail.messagingengine.com> References: <1513100678.963297.1202703816.615DDD86@webmail.messagingengine.com> <12518065-3804-435F-9A80-BCBE851B3937@sinergise.com> <1513111489.1043869.1202914832.38122389@webmail.messagingengine.com> Message-ID: <6E9F7549-09F7-45E8-AB9F-72E0DC23CC14@sinergise.com> Hi Bryce, MGRS-based searching is not yet supported in OpenSearch. You can however always use S3 listing of your chosen object (e.g. http://sentinel-s2-l1c.s3.amazonaws.com/tiles/22/K/EC/) and find relevant tiles. Alternatively you can download CSV catalogue and query that one: https://scihub.copernicus.eu/news/News00220 Regards Grega > Grega, > > My goal was to crawl for all products on a specific date, get the tile > metadata and persist it to query against. It looks like OpenSearch > accomplishes most of what I was attempting to do, so that is great. > > One quirk I'm wondering about is that I have a list of specific tiles > that I'm interested in that I keep track of based on their MGRS grid > identifier (i.e 22KEC). I was looking at the opensearch.sentinel-hub.com > describe.xml definition and have been trying out some queries. > > Is there currently a way to send a wildcard query term based on a subset > of the full product name to accommodate searching for everything for > that particular tile? > > For example something like: > http://opensearch.sentinel-hub.com/resto/api/collections/Sentinel2/search.json?productIdentifier=*22KEC* > > Or would there be another way to accomplish this? > > Thanks > -- > Bryce Johnston > bryce at agdeveloper.com > > On Tue, Dec 12, 2017, at 10:13 AM, Grega Milcinski wrote: >> Hi Bryce, >> >> S2A and S2B are treated exactly the same at AWS. What you are probably >> noticing is that “products” site >> (http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/) >> is only listing first 1000 entries as it was not really designed to >> browse through “products” but rather through “tiles”. >> >> If you need information from this folder, you can look-up for “path” >> attribute in productInfo.json and then go directly there, e.g.: >> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/2017/10/10/S2B_MSIL1C_20171010T133209_N0205_R081_T22KEC_20171010T133528/ >> >> If you want to “crawl” for data, I suggest you use OpenSearch, check this >> blog post: >> https://aws.amazon.com/blogs/publicsector/complete-sentinel-2-archives-freely-available-to-users/ >> >> Regards >> Grega >> >> >>> Hello, >>> >>> I had a question about if S2B products should be showing up when >>> browsing >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/ >>> ? >>> >>> For an example here is a tile from S2B product: >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#tiles/22/K/EC/2017/10/10/0/ >>> , but when browsing >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/2017/10/10/ >>> it only appears to list S2A products. >>> >>> Is there a way to get the S2B entries as well or am I overlooking >>> something? >>> >>> Thanks, >>> -- >>> Bryce Johnston >>> bryce at agdeveloper.com >>> _______________________________________________ >>> Sentinel-pds mailing list >>> Sentinel-pds at lists.osgeo.org >>> https://lists.osgeo.org/mailman/listinfo/sentinel-pds >> From bryce at agdeveloper.com Tue Dec 12 13:24:40 2017 From: bryce at agdeveloper.com (Bryce Johnston) Date: Tue, 12 Dec 2017 13:24:40 -0800 Subject: [Sentinel-pds] AWS Sentinel-2 - S2B product listings In-Reply-To: <6E9F7549-09F7-45E8-AB9F-72E0DC23CC14@sinergise.com> References: <1513100678.963297.1202703816.615DDD86@webmail.messagingengine.com> <12518065-3804-435F-9A80-BCBE851B3937@sinergise.com> <1513111489.1043869.1202914832.38122389@webmail.messagingengine.com> <6E9F7549-09F7-45E8-AB9F-72E0DC23CC14@sinergise.com> Message-ID: <1513113880.1981904.1202946240.4322DFDC@webmail.messagingengine.com> Grega, Ok thanks. Are the OpenSearch entries and feed built during the ingestion process? I saw this https://github.com/sinergise/Sentinel2ProductIngestor/tree/master/src/main/java/com/sinergise/sentinel/scihub/opensearch but I'm assuming thats not the current code handling ingestion and OpenSearch? If there is a way to help with contributing to MGRS searches in OpenSearch, I'd love to try to help. If not, I can still do what I need from OpenSearch date queries and iterating through to check tiles, so I appreciate it being available. Thanks, -- Bryce Johnston bryce at agdeveloper.com On Tue, Dec 12, 2017, at 12:51 PM, Grega Milcinski wrote: > Hi Bryce, > > MGRS-based searching is not yet supported in OpenSearch. You can however > always use S3 listing of your chosen object (e.g. > http://sentinel-s2-l1c.s3.amazonaws.com/tiles/22/K/EC/) and find relevant > tiles. > > Alternatively you can download CSV catalogue and query that one: > https://scihub.copernicus.eu/news/News00220 > > Regards > Grega > > > Grega, > > > > My goal was to crawl for all products on a specific date, get the tile > > metadata and persist it to query against. It looks like OpenSearch > > accomplishes most of what I was attempting to do, so that is great. > > > > One quirk I'm wondering about is that I have a list of specific tiles > > that I'm interested in that I keep track of based on their MGRS grid > > identifier (i.e 22KEC). I was looking at the opensearch.sentinel-hub.com > > describe.xml definition and have been trying out some queries. > > > > Is there currently a way to send a wildcard query term based on a subset > > of the full product name to accommodate searching for everything for > > that particular tile? > > > > For example something like: > > http://opensearch.sentinel-hub.com/resto/api/collections/Sentinel2/search.json?productIdentifier=*22KEC* > > > > Or would there be another way to accomplish this? > > > > Thanks > > -- > > Bryce Johnston > > bryce at agdeveloper.com > > > > On Tue, Dec 12, 2017, at 10:13 AM, Grega Milcinski wrote: > >> Hi Bryce, > >> > >> S2A and S2B are treated exactly the same at AWS. What you are probably > >> noticing is that “products” site > >> (http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/) > >> is only listing first 1000 entries as it was not really designed to > >> browse through “products” but rather through “tiles”. > >> > >> If you need information from this folder, you can look-up for “path” > >> attribute in productInfo.json and then go directly there, e.g.: > >> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/2017/10/10/S2B_MSIL1C_20171010T133209_N0205_R081_T22KEC_20171010T133528/ > >> > >> If you want to “crawl” for data, I suggest you use OpenSearch, check this > >> blog post: > >> https://aws.amazon.com/blogs/publicsector/complete-sentinel-2-archives-freely-available-to-users/ > >> > >> Regards > >> Grega > >> > >> > >>> Hello, > >>> > >>> I had a question about if S2B products should be showing up when > >>> browsing > >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/ > >>> ? > >>> > >>> For an example here is a tile from S2B product: > >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#tiles/22/K/EC/2017/10/10/0/ > >>> , but when browsing > >>> http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#products/2017/10/10/ > >>> it only appears to list S2A products. > >>> > >>> Is there a way to get the S2B entries as well or am I overlooking > >>> something? > >>> > >>> Thanks, > >>> -- > >>> Bryce Johnston > >>> bryce at agdeveloper.com > >>> _______________________________________________ > >>> Sentinel-pds mailing list > >>> Sentinel-pds at lists.osgeo.org > >>> https://lists.osgeo.org/mailman/listinfo/sentinel-pds > >> > > _______________________________________________ > Sentinel-pds mailing list > Sentinel-pds at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/sentinel-pds