[postgis-users] WKTRaster : gdal2wktraster.py cannot read AIG/Arc/Info Binary Grid
Jorge Arévalo
jorge.arevalo at deimos-space.com
Wed Jul 28 01:44:52 PDT 2010
Hello Peter,
On Wed, Jul 28, 2010 at 10:26 AM, Peter Baumann
<p.baumann at jacobs-university.de> wrote:
> Pierre,
>
> admittedly this was a little inexact, and I certainly will correct it. A
> more accurate phrasing I guess is "does not support retrieval on a raster
> array consisting of a set of tiles, rather tiles have to be treated
> individually."
> From what I understand WKT Raster can tile an incoming image, but cannot
> treat a set of tiles making up a larger coverage as one.
>
> Would you agree to that?
>
> -Peter
>
WKT Raster can't do that just now, but it's one of the specs for beta
0.2.4. Take a look at
http://trac.osgeo.org/postgis/wiki/WKTRaster/SpecificationWorking02#ObjectiveB02d-Beingabletousegroupbytoaccumulatetilestoformanewraster.
Best regards,
Jorge
>
>
> On 07/27/2010 05:52 PM, Pierre Racine wrote:
>
> BTW Peter,
>
>
>
> in your site
> (http://kahlua.eecs.jacobs-university.de/trac/rasdaman/wiki/WhyRasdaman) you
> say: “First and foremost, it (PostGIS WKT Raster) does not support tiling
> (partitioning) of raster arrays.” This statement is false since WKT Raster
> actually does support tiling of large rasters. You can partition any raster
> coverage by importing it as separate tiles or by using the –k option of
> gdal2wktraster.py. Each tile become a separate row in the table and the
> whole table is the coverage. The table is spatially indexed using the
> PostgreSQL GiST index.
>
>
>
> I think it would be just fair if you could rectify this statement.
>
>
>
> Thanks,
>
>
>
> Pierre
>
>
>
> From: Peter Baumann [mailto:p.baumann at jacobs-university.de]
> Sent: 21 juillet 2010 17:28
> To: Pierre Racine
> Cc: PostGIS Users Discussion
> Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> Pierre-
>
> rasdaman per se is domain agnostic - you can just as well maintain human
> brain images (as we did). The petascope layer, which is part of the distro,
> adds geo semantics by providing OGC interfaces.
> PostgreSQL integration is not yet done, we are working on this; same for
> GDAL / MapServer.
>
> -Peter
>
>
> On 07/21/2010 08:29 PM, Pierre Racine wrote:
>
> Peter,
>
>
>
> Are rasters in rasdaman georeferenced? How do I convert my vectorial PostGIS
> points coordinates into rasdaman coordinates? Do I have to do all the math
> myself?
>
>
>
> Pierre
>
>
>
> From: Peter Baumann [mailto:p.baumann at jacobs-university.de]
> Sent: 21 juillet 2010 04:41
> To: Pierre Racine
> Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> Pierre-
>
> let's see. First, I would not store 100k raster points in the relational
> table as its terribly inefficient. We do have input data formats, though,
> which are similar to what a PG dump of such a table would deliver. We import
> this into an array and then can query it. Import via rasql or through an
> import tool which internally generates rasql calls.
>
> Then, for corner points (x0,y0) and (x1,y1) we retrieve as
>
> select png( M[x0:y1,y0:y1] ) from MyMap as M
>
> ...in PNG format via the rasql cmd line utility or the C++ or Java API.
>
> Alternatively, sending this this query over the API gets you the data into
> main memory ready for nested for loops = in the main memory format of your
> client:
>
> select M[x0:y1,y0:y1] from MyMap as M
>
> -Peter
>
>
>
> On 07/20/2010 10:03 PM, Pierre Racine wrote:
>
> So if I already have a table of, say 100 000 points, uploaded in PostGIS,
> how do I use rasdaman to get the raster values of the pixels located under
> the points?
>
>
>
> Pierre
>
>
>
> From: Peter Baumann [mailto:p.baumann at jacobs-university.de]
> Sent: 20 juillet 2010 12:08
> To: PostGIS Users Discussion
> Cc: Pierre Racine
> Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> On 07/20/2010 05:47 PM, Pierre Racine wrote:
>
> Since we have this bug with big rasters,
>
> proven, any-size, ... ;-)
>
> Who is proven? Well, running a dozen-TB seamless mosaic on PostgreSQL for
> many years, having online-demos available since years, etc... you decide, in
> comparison.
>
> -Peter
>
>
>
>
>
> I would suggest you split your raster with something like GDAL
> gdal_retile.py and then import your tiles into a single WKT Raster table
> following the instruction provided in our tutorial. Each tile will be stored
> in a column cell of type RASTER similar to the PostGIS GEOMETRY type.
>
>
>
> As I said previously you can then query the raster in SQL like this:
>
>
>
> SELECT ST_Value(rast, ST_Geomfromtext('Point(-78.1 58.1)', 4326))
>
> FROM srtm_tiled_100x100
>
> WHERE ST_Intersects(rast::geometry, ST_Geomfromtext('Point(-78.1 58.1)',
> 4326)) AND whatever you want.
>
>
>
> Basically WKT Raster is the first true SQL interface with which is is simple
> to do such things… It's proven, any-size, cloud-scalable, and open source J
> Maybe not that prooven. But who is?
>
>
>
> There is also rasdaman but I don’t think you can use SQL. It would be nice
> if you could compare both… I can’t find time for this. Jorge has started
> comparing PostGIS WKT Raster with Oracle Georaster though.
>
>
>
> Pierre
>
>
>
>
>
> From: postgis-users-bounces at postgis.refractions.net
> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
> Sebastian E. Ovide
> Sent: 20 juillet 2010 10:46
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> yes... with png worked... but it was a different png (a smaller one)...
>
> so I've converted the ESRI into a png and tried to import it... and it
> didn't work neither...
>
> so this is the situation:
> A have huge rusters (from 150kx150k).. In Oracle I would just load it (the
> huge raster) in a single row of a GeoRaster table and then Oracle GeoRaster
> would split it in small tiles and store one tile per line of another table
> (Raster Data Table)... then I can run a query similar to this one: SELECT
> getcellvalue(t.rastercolumn,x,y) from GeoRasterTable t where t.id=1; and
> GeoRaster will query automatically the spatial indexes and the Raster Data
> Table and it will find the right tile etc...
>
> I do not know how WKTRaster works.... If I cannot import a such big image,
> of course I can split it in smaller georeferenced tiles... (how?)... but....
> My main question is: after that, how will the table look like ? how can I do
> the same query (where id=1 or where name="UK" etc...) ?
>
>
>
>
> On Tue, Jul 20, 2010 at 2:31 PM, Pierre Racine <Pierre.Racine at sbf.ulaval.ca>
> wrote:
>
> Wait. You first said the png was working. Now it’s not? Did you try
> gdal_translate with the ESRI grid? For sure I haven’t test yet with such big
> rasters. Is this the result of a merge or all your original raster are all
> this size? The point is that with WKT Raster you don’t have to merge your
> raster first into a gigantic raster in order to get it store in a unique
> table like with Oracle Spatial.
>
>
>
> Pierre
>
>
>
> From: postgis-users-bounces at postgis.refractions.net
> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
> Sebastian E. Ovide
> Sent: 20 juillet 2010 05:51
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> Hi Pierre,
>
> Does gdal2wktraster.py have any limitation on the maximum number of
> columnsxrows ?
>
> in my case, my raster is 107759 x 168633...
>
> gdal works well:
>
> sebas at SeansPC:~/rasters$ gdal_translate -of PNG raster/ test.png
> Input file size is 107759, 168633
> 0...10...20...30...40...50...60...70...80...90...100 - done.
>
>
> sebas at SeansPC:~/rasters$ python gdal2wktraster.py -r raster/ -t sebastable
> -o ok.sql
> gdal2wktraster.py:695: DeprecationWarning: 'H' format requires 0 <= number
> <= 65535
> hexwkb += wkblify('H', xsize)
> gdal2wktraster.py:696: DeprecationWarning: 'H' format requires 0 <= number
> <= 65535
> hexwkb += wkblify('H', ysize)
> gdal2wktraster.py:727: DeprecationWarning: integer argument expected, got
> float
> hexwkb += wkblify(pt2fmt(pixtype), nodata)
> Traceback (most recent call last):
> File "gdal2wktraster.py", line 1013, in <module>
> main()
> File "gdal2wktraster.py", line 976, in main
> wkblify_raster(opts, filename, i)
> File "gdal2wktraster.py", line 921, in wkblify_raster
> summary = wkblify_raster_level(options, ds, options.overview_level,
> band_range, infile, i)
> File "gdal2wktraster.py", line 888, in wkblify_raster_level
> hexwkb += wkblify_band(options, band, level, xoff, yoff,
> read_block_size, block_size, infile, b)
> File "gdal2wktraster.py", line 777, in wkblify_band
> target_block_size[0], target_block_size[1])
> File "/usr/lib/python2.6/dist-packages/osgeo/gdal.py", line 895, in
> ReadAsArray
> buf_xsize, buf_ysize, buf_obj )
> File "/usr/lib/python2.6/dist-packages/osgeo/gdal_array.py", line 228, in
> BandReadAsArray
> ar = numpy.empty([buf_ysize,buf_xsize], dtype = typecode)
> MemoryError
>
>
> sebas at SeansPC:~/rasters$ python gdal2wktraster.py -r test.png -t sebastable
> -o ok.sql
> gdal2wktraster.py:695: DeprecationWarning: 'H' format requires 0 <= number
> <= 65535
> hexwkb += wkblify('H', xsize)
> gdal2wktraster.py:696: DeprecationWarning: 'H' format requires 0 <= number
> <= 65535
> hexwkb += wkblify('H', ysize)
> gdal2wktraster.py:727: DeprecationWarning: integer argument expected, got
> float
> hexwkb += wkblify(pt2fmt(pixtype), nodata)
> Traceback (most recent call last):
> File "gdal2wktraster.py", line 1013, in <module>
> main()
> File "gdal2wktraster.py", line 976, in main
> wkblify_raster(opts, filename, i)
> File "gdal2wktraster.py", line 921, in wkblify_raster
> summary = wkblify_raster_level(options, ds, options.overview_level,
> band_range, infile, i)
> File "gdal2wktraster.py", line 888, in wkblify_raster_level
> hexwkb += wkblify_band(options, band, level, xoff, yoff,
> read_block_size, block_size, infile, b)
> File "gdal2wktraster.py", line 777, in wkblify_band
> target_block_size[0], target_block_size[1])
> File "/usr/lib/python2.6/dist-packages/osgeo/gdal.py", line 895, in
> ReadAsArray
> buf_xsize, buf_ysize, buf_obj )
> File "/usr/lib/python2.6/dist-packages/osgeo/gdal_array.py", line 228, in
> BandReadAsArray
> ar = numpy.empty([buf_ysize,buf_xsize], dtype = typecode)
> MemoryError
>
>
>
>
>
>
> On Mon, Jul 19, 2010 at 5:56 PM, Pierre Racine <Pierre.Racine at sbf.ulaval.ca>
> wrote:
>
> Hi Sebastian,
>
>
>
> I can convert ESRI Grid file to .sql without problem using gdal2wktraster.py
> and the same parameters as you. I can do both integer and floating point
> rasters.
>
>
>
> Maybe this is a GDAL problem. Try to convert it using gdal_translate (to
> tiff for example). This would be a better test than just gdalinfo.
>
>
>
> Could you provide us with a file sample?
>
>
>
> Pierre
>
>
>
> From: postgis-users-bounces at postgis.refractions.net
> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
> Sebastian E. Ovide
> Sent: 19 juillet 2010 12:28
> To: PostGIS Users Discussion
> Subject: [postgis-users] WKTRaster : gdal2wktraster.py cannot read
> AIG/Arc/Info Binary Grid
>
>
>
> Hi All,
>
> trying to create a SQL with gdal2wktraster.py. It works on PNG but it
> doesn't on AIG files...
>
> Note: As Gdal works fine.
>
> C:\Program Files\PostgreSQL\8.4\bin>gdalinfo c:\tmp\raster
> Driver: AIG/Arc/Info Binary Grid
> Files: c:\tmp\raster
> c:\tmp\raster\dblbnd.adf
> c:\tmp\raster\hdr.adf
> c:\tmp\raster\metadata.xml
> c:\tmp\raster\prj.adf
> c:\tmp\raster\sta.adf
> c:\tmp\raster\vat.adf
> c:\tmp\raster\w001000.adf
> c:\tmp\raster\w001000x.adf
> c:\tmp\raster\w001001.adf
> c:\tmp\raster\w001001x.adf
> c:\tmp\raster\z001001.adf
> c:\tmp\raster\z001001x.adf
> c:\tmp\raster\z001002.adf
> c:\tmp\raster\z001002x.adf
> c:\tmp\raster\z001003.adf
> c:\tmp\raster\z001003x.adf
> c:\tmp\raster\z001004.adf
> c:\tmp\raster\z001004x.adf
> c:\tmp\raster\z001005.adf
> c:\tmp\raster\z001005x.adf
> c:\tmp\raster\z001006.adf
> c:\tmp\raster\z001006x.adf
> c:\tmp\raster\z001007.adf
> c:\tmp\raster\z001007x.adf
> c:\tmp\raster\z001008.adf
> c:\tmp\raster\z001008x.adf
> c:\tmp\raster\z001009.adf
> c:\tmp\raster\z001009x.adf
> c:\tmp\raster\z001010.adf
> c:\tmp\raster\z001010x.adf
> c:\tmp\raster\z001011.adf
> c:\tmp\raster\z001011x.adf
> c:\tmp\raster\z001012.adf
> c:\tmp\raster\z001012x.adf
> c:\tmp\raster\z001013.adf
> c:\tmp\raster\z001013x.adf
> c:\tmp\raster\z001014.adf
> c:\tmp\raster\z001014x.adf
> c:\tmp\raster\z001015.adf
> c:\tmp\raster\z001015x.adf
> Size is 107759, 168633
> Coordinate System is:
> PROJCS["unnamed",
> GEOGCS["Unknown datum based upon the Airy 1830 ellipsoid",
> DATUM["Not_specified_based_on_Airy_1830_ellipsoid",
> SPHEROID["Airy 1830",6377563.396,299.3249646,
> AUTHORITY["EPSG","7001"]],
> AUTHORITY["EPSG","6001"]],
> PRIMEM["Greenwich",0,
> AUTHORITY["EPSG","8901"]],
> UNIT["degree",0.01745329251994328,
> AUTHORITY["EPSG","9122"]],
> AUTHORITY["EPSG","4001"]],
> PROJECTION["Transverse_Mercator"],
> PARAMETER["latitude_of_origin",49],
> PARAMETER["central_meridian",-2],
> PARAMETER["scale_factor",0.9996012717],
> PARAMETER["false_easting",400000],
> PARAMETER["false_northing",-100000],
> UNIT["METERS",1]]
> Origin = (128110.000000000000000,813270.000000000000000)
> Pixel Size = (5.000000000000000,-5.000000000000000)
> Corner Coordinates:
> Upper Left ( 128110.000, 813270.000) ( 6d29'37.32"W, 57d 7'47.53"N)
> Lower Left ( 128110.000, -29895.000) ( 5d45'40.00"W, 49d34'10.24"N)
> Upper Right ( 666905.000, 813270.000) ( 2d24'41.72"E, 57d 7'58.04"N)
> Lower Right ( 666905.000, -29895.000) ( 1d41'32.29"E, 49d34'18.23"N)
> Center ( 397507.500, 391687.500) ( 2d 2'15.04"W, 53d25'18.19"N)
> Band 1 Block=256x4 Type=Byte, ColorInterp=Undefined
> Min=1.000 Max=4.000
> NoData Value=255
>
>
> C:\Program Files\PostgreSQL\8.4\bin>python gdal2wktraster.py -r
> c:\tmp\raster -t sebastable -o c:\tmp\sebas.sql
> gdal2wktraster.py:644: DeprecationWarning: 'H' format requires 0 <= number
> <= 65535
> hexstr = binascii.hexlify(struct.pack(fmt_little, data)).upper()
> gdal2wktraster.py:644: DeprecationWarning: integer argument expected, got
> float
> hexstr = binascii.hexlify(struct.pack(fmt_little, data)).upper()
> ERROR 2: Multiplication overflow : 107759 * 168633 * 1
> Traceback (most recent call last):
> File "gdal2wktraster.py", line 1013, in <module>
> main()
> File "gdal2wktraster.py", line 976, in main
> wkblify_raster(opts, filename, i)
> File "gdal2wktraster.py", line 921, in wkblify_raster
> summary = wkblify_raster_level(options, ds, options.overview_level,
> band_range, infile, i)
> File "gdal2wktraster.py", line 888, in wkblify_raster_level
> hexwkb += wkblify_band(options, band, level, xoff, yoff,
> read_block_size, block_size, infile, b)
> File "gdal2wktraster.py", line 777, in wkblify_band
> target_block_size[0], target_block_size[1])
> File "C:\OSGeo4W\apps\gdal-16\pymod\osgeo\gdal.py", line 835, in
> ReadAsArray
> buf_xsize, buf_ysize, buf_obj )
> File "C:\OSGeo4W\apps\gdal-16\pymod\osgeo\gdal_array.py", line 140, in
> BandReadAsArray
> ar = numpy.reshape(ar, [buf_ysize,buf_xsize])
> File
> "C:\OSGeo4W\apps\Python25\lib\site-packages\numpy\core\fromnumeric.py", line
> 116, in reshape
> return reshape(newshape, order=order)
> ValueError: total size of new array must be unchanged
>
> Any ideas ?
> --
> Sebastian E. Ovide
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
> --
> Sebastian E. Ovide
>
> skype: sebastian.ovide
>
> +353 (0) 87 6340149
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
> --
> Sebastian E. Ovide
>
>
>
>
>
>
>
>
>
>
> --
>
> Dr. Peter Baumann
>
> - Professor of Computer Science, Jacobs University Bremen
>
> www.faculty.jacobs-university.de/pbaumann
>
> mail: p.baumann at jacobs-university.de
>
> tel: +49-421-200-3178, fax: +49-421-200-493178
>
> - Executive Director, rasdaman GmbH Bremen (HRB 147737)
>
> www.rasdaman.com, mail: baumann at rasdaman.com
>
> tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
> ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
> destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD
> 10xx)
>
>
>
>
>
>
> --
>
> Dr. Peter Baumann
>
> - Professor of Computer Science, Jacobs University Bremen
>
> www.faculty.jacobs-university.de/pbaumann
>
> mail: p.baumann at jacobs-university.de
>
> tel: +49-421-200-3178, fax: +49-421-200-493178
>
> - Executive Director, rasdaman GmbH Bremen (HRB 147737)
>
> www.rasdaman.com, mail: baumann at rasdaman.com
>
> tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
> ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
> destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD
> 10xx)
>
>
>
>
>
> --
>
> Dr. Peter Baumann
>
> - Professor of Computer Science, Jacobs University Bremen
>
> www.faculty.jacobs-university.de/pbaumann
>
> mail: p.baumann at jacobs-university.de
>
> tel: +49-421-200-3178, fax: +49-421-200-493178
>
> - Executive Director, rasdaman GmbH Bremen (HRB 147737)
>
> www.rasdaman.com, mail: baumann at rasdaman.com
>
> tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
> ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
> destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD
> 10xx)
>
>
>
>
>
> --
> Dr. Peter Baumann
> - Professor of Computer Science, Jacobs University Bremen
> www.faculty.jacobs-university.de/pbaumann
> mail: p.baumann at jacobs-university.de
> tel: +49-421-200-3178, fax: +49-421-200-493178
> - Executive Director, rasdaman GmbH Bremen (HRB 147737)
> www.rasdaman.com, mail: baumann at rasdaman.com
> tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
> ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
> destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD
> 10xx)
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
More information about the postgis-users
mailing list