[postgis-users] WKTRaster : gdal2wktraster.py cannot read AIG/Arc/Info Binary Grid
Pierre Racine
Pierre.Racine at sbf.ulaval.ca
Tue Jul 20 08:47:08 PDT 2010
Since we have this bug with big rasters, I would suggest you split your raster with something like GDAL gdal_retile.py and then import your tiles into a single WKT Raster table following the instruction provided in our tutorial. Each tile will be stored in a column cell of type RASTER similar to the PostGIS GEOMETRY type.
As I said previously you can then query the raster in SQL like this:
SELECT ST_Value(rast, ST_Geomfromtext('Point(-78.1 58.1)', 4326))
FROM srtm_tiled_100x100
WHERE ST_Intersects(rast::geometry, ST_Geomfromtext('Point(-78.1 58.1)', 4326)) AND whatever you want.
Basically WKT Raster is the first true SQL interface with which is is simple to do such things... It's proven, any-size, cloud-scalable, and open source :) Maybe not that prooven. But who is?
There is also rasdaman but I don't think you can use SQL. It would be nice if you could compare both... I can't find time for this. Jorge has started comparing PostGIS WKT Raster with Oracle Georaster though.
Pierre
From: postgis-users-bounces at postgis.refractions.net [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of Sebastian E. Ovide
Sent: 20 juillet 2010 10:46
To: PostGIS Users Discussion
Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read AIG/Arc/Info Binary Grid
yes... with png worked... but it was a different png (a smaller one)...
so I've converted the ESRI into a png and tried to import it... and it didn't work neither...
so this is the situation:
A have huge rusters (from 150kx150k).. In Oracle I would just load it (the huge raster) in a single row of a GeoRaster table and then Oracle GeoRaster would split it in small tiles and store one tile per line of another table (Raster Data Table)... then I can run a query similar to this one: SELECT getcellvalue(t.rastercolumn,x,y) from GeoRasterTable t where t.id<http://t.id>=1; and GeoRaster will query automatically the spatial indexes and the Raster Data Table and it will find the right tile etc...
I do not know how WKTRaster works.... If I cannot import a such big image, of course I can split it in smaller georeferenced tiles... (how?)... but.... My main question is: after that, how will the table look like ? how can I do the same query (where id=1 or where name="UK" etc...) ?
On Tue, Jul 20, 2010 at 2:31 PM, Pierre Racine <Pierre.Racine at sbf.ulaval.ca<mailto:Pierre.Racine at sbf.ulaval.ca>> wrote:
Wait. You first said the png was working. Now it's not? Did you try gdal_translate with the ESRI grid? For sure I haven't test yet with such big rasters. Is this the result of a merge or all your original raster are all this size? The point is that with WKT Raster you don't have to merge your raster first into a gigantic raster in order to get it store in a unique table like with Oracle Spatial.
Pierre
From: postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net> [mailto:postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net>] On Behalf Of Sebastian E. Ovide
Sent: 20 juillet 2010 05:51
To: PostGIS Users Discussion
Subject: Re: [postgis-users] WKTRaster : gdal2wktraster.py cannot read AIG/Arc/Info Binary Grid
Hi Pierre,
Does gdal2wktraster.py have any limitation on the maximum number of columnsxrows ?
in my case, my raster is 107759 x 168633...
gdal works well:
sebas at SeansPC:~/rasters$ gdal_translate -of PNG raster/ test.png
Input file size is 107759, 168633
0...10...20...30...40...50...60...70...80...90...100 - done.
sebas at SeansPC:~/rasters$ python gdal2wktraster.py -r raster/ -t sebastable -o ok.sql
gdal2wktraster.py:695: DeprecationWarning: 'H' format requires 0 <= number <= 65535
hexwkb += wkblify('H', xsize)
gdal2wktraster.py:696: DeprecationWarning: 'H' format requires 0 <= number <= 65535
hexwkb += wkblify('H', ysize)
gdal2wktraster.py:727: DeprecationWarning: integer argument expected, got float
hexwkb += wkblify(pt2fmt(pixtype), nodata)
Traceback (most recent call last):
File "gdal2wktraster.py", line 1013, in <module>
main()
File "gdal2wktraster.py", line 976, in main
wkblify_raster(opts, filename, i)
File "gdal2wktraster.py", line 921, in wkblify_raster
summary = wkblify_raster_level(options, ds, options.overview_level, band_range, infile, i)
File "gdal2wktraster.py", line 888, in wkblify_raster_level
hexwkb += wkblify_band(options, band, level, xoff, yoff, read_block_size, block_size, infile, b)
File "gdal2wktraster.py", line 777, in wkblify_band
target_block_size[0], target_block_size[1])
File "/usr/lib/python2.6/dist-packages/osgeo/gdal.py", line 895, in ReadAsArray
buf_xsize, buf_ysize, buf_obj )
File "/usr/lib/python2.6/dist-packages/osgeo/gdal_array.py", line 228, in BandReadAsArray
ar = numpy.empty([buf_ysize,buf_xsize], dtype = typecode)
MemoryError
sebas at SeansPC:~/rasters$ python gdal2wktraster.py -r test.png -t sebastable -o ok.sql
gdal2wktraster.py:695: DeprecationWarning: 'H' format requires 0 <= number <= 65535
hexwkb += wkblify('H', xsize)
gdal2wktraster.py:696: DeprecationWarning: 'H' format requires 0 <= number <= 65535
hexwkb += wkblify('H', ysize)
gdal2wktraster.py:727: DeprecationWarning: integer argument expected, got float
hexwkb += wkblify(pt2fmt(pixtype), nodata)
Traceback (most recent call last):
File "gdal2wktraster.py", line 1013, in <module>
main()
File "gdal2wktraster.py", line 976, in main
wkblify_raster(opts, filename, i)
File "gdal2wktraster.py", line 921, in wkblify_raster
summary = wkblify_raster_level(options, ds, options.overview_level, band_range, infile, i)
File "gdal2wktraster.py", line 888, in wkblify_raster_level
hexwkb += wkblify_band(options, band, level, xoff, yoff, read_block_size, block_size, infile, b)
File "gdal2wktraster.py", line 777, in wkblify_band
target_block_size[0], target_block_size[1])
File "/usr/lib/python2.6/dist-packages/osgeo/gdal.py", line 895, in ReadAsArray
buf_xsize, buf_ysize, buf_obj )
File "/usr/lib/python2.6/dist-packages/osgeo/gdal_array.py", line 228, in BandReadAsArray
ar = numpy.empty([buf_ysize,buf_xsize], dtype = typecode)
MemoryError
On Mon, Jul 19, 2010 at 5:56 PM, Pierre Racine <Pierre.Racine at sbf.ulaval.ca<mailto:Pierre.Racine at sbf.ulaval.ca>> wrote:
Hi Sebastian,
I can convert ESRI Grid file to .sql without problem using gdal2wktraster.py and the same parameters as you. I can do both integer and floating point rasters.
Maybe this is a GDAL problem. Try to convert it using gdal_translate (to tiff for example). This would be a better test than just gdalinfo.
Could you provide us with a file sample?
Pierre
From: postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net> [mailto:postgis-users-bounces at postgis.refractions.net<mailto:postgis-users-bounces at postgis.refractions.net>] On Behalf Of Sebastian E. Ovide
Sent: 19 juillet 2010 12:28
To: PostGIS Users Discussion
Subject: [postgis-users] WKTRaster : gdal2wktraster.py cannot read AIG/Arc/Info Binary Grid
Hi All,
trying to create a SQL with gdal2wktraster.py. It works on PNG but it doesn't on AIG files...
Note: As Gdal works fine.
C:\Program Files\PostgreSQL\8.4\bin>gdalinfo c:\tmp\raster
Driver: AIG/Arc/Info Binary Grid
Files: c:\tmp\raster
c:\tmp\raster\dblbnd.adf
c:\tmp\raster\hdr.adf
c:\tmp\raster\metadata.xml
c:\tmp\raster\prj.adf
c:\tmp\raster\sta.adf
c:\tmp\raster\vat.adf
c:\tmp\raster\w001000.adf
c:\tmp\raster\w001000x.adf
c:\tmp\raster\w001001.adf
c:\tmp\raster\w001001x.adf
c:\tmp\raster\z001001.adf
c:\tmp\raster\z001001x.adf
c:\tmp\raster\z001002.adf
c:\tmp\raster\z001002x.adf
c:\tmp\raster\z001003.adf
c:\tmp\raster\z001003x.adf
c:\tmp\raster\z001004.adf
c:\tmp\raster\z001004x.adf
c:\tmp\raster\z001005.adf
c:\tmp\raster\z001005x.adf
c:\tmp\raster\z001006.adf
c:\tmp\raster\z001006x.adf
c:\tmp\raster\z001007.adf
c:\tmp\raster\z001007x.adf
c:\tmp\raster\z001008.adf
c:\tmp\raster\z001008x.adf
c:\tmp\raster\z001009.adf
c:\tmp\raster\z001009x.adf
c:\tmp\raster\z001010.adf
c:\tmp\raster\z001010x.adf
c:\tmp\raster\z001011.adf
c:\tmp\raster\z001011x.adf
c:\tmp\raster\z001012.adf
c:\tmp\raster\z001012x.adf
c:\tmp\raster\z001013.adf
c:\tmp\raster\z001013x.adf
c:\tmp\raster\z001014.adf
c:\tmp\raster\z001014x.adf
c:\tmp\raster\z001015.adf
c:\tmp\raster\z001015x.adf
Size is 107759, 168633
Coordinate System is:
PROJCS["unnamed",
GEOGCS["Unknown datum based upon the Airy 1830 ellipsoid",
DATUM["Not_specified_based_on_Airy_1830_ellipsoid",
SPHEROID["Airy 1830",6377563.396,299.3249646,
AUTHORITY["EPSG","7001"]],
AUTHORITY["EPSG","6001"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.01745329251994328,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4001"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.9996012717],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["METERS",1]]
Origin = (128110.000000000000000,813270.000000000000000)
Pixel Size = (5.000000000000000,-5.000000000000000)
Corner Coordinates:
Upper Left ( 128110.000, 813270.000) ( 6d29'37.32"W, 57d 7'47.53"N)
Lower Left ( 128110.000, -29895.000) ( 5d45'40.00"W, 49d34'10.24"N)
Upper Right ( 666905.000, 813270.000) ( 2d24'41.72"E, 57d 7'58.04"N)
Lower Right ( 666905.000, -29895.000) ( 1d41'32.29"E, 49d34'18.23"N)
Center ( 397507.500, 391687.500) ( 2d 2'15.04"W, 53d25'18.19"N)
Band 1 Block=256x4 Type=Byte, ColorInterp=Undefined
Min=1.000 Max=4.000
NoData Value=255
C:\Program Files\PostgreSQL\8.4\bin>python gdal2wktraster.py -r c:\tmp\raster -t sebastable -o c:\tmp\sebas.sql
gdal2wktraster.py:644: DeprecationWarning: 'H' format requires 0 <= number <= 65535
hexstr = binascii.hexlify(struct.pack(fmt_little, data)).upper()
gdal2wktraster.py:644: DeprecationWarning: integer argument expected, got float
hexstr = binascii.hexlify(struct.pack(fmt_little, data)).upper()
ERROR 2: Multiplication overflow : 107759 * 168633 * 1
Traceback (most recent call last):
File "gdal2wktraster.py", line 1013, in <module>
main()
File "gdal2wktraster.py", line 976, in main
wkblify_raster(opts, filename, i)
File "gdal2wktraster.py", line 921, in wkblify_raster
summary = wkblify_raster_level(options, ds, options.overview_level, band_range, infile, i)
File "gdal2wktraster.py", line 888, in wkblify_raster_level
hexwkb += wkblify_band(options, band, level, xoff, yoff, read_block_size, block_size, infile, b)
File "gdal2wktraster.py", line 777, in wkblify_band
target_block_size[0], target_block_size[1])
File "C:\OSGeo4W\apps\gdal-16\pymod\osgeo\gdal.py", line 835, in ReadAsArray
buf_xsize, buf_ysize, buf_obj )
File "C:\OSGeo4W\apps\gdal-16\pymod\osgeo\gdal_array.py", line 140, in BandReadAsArray
ar = numpy.reshape(ar, [buf_ysize,buf_xsize])
File "C:\OSGeo4W\apps\Python25\lib\site-packages\numpy\core\fromnumeric.py", line 116, in reshape
return reshape(newshape, order=order)
ValueError: total size of new array must be unchanged
Any ideas ?
--
Sebastian E. Ovide
_______________________________________________
postgis-users mailing list
postgis-users at postgis.refractions.net<mailto:postgis-users at postgis.refractions.net>
http://postgis.refractions.net/mailman/listinfo/postgis-users
--
Sebastian E. Ovide
skype: sebastian.ovide
+353 (0) 87 6340149
_______________________________________________
postgis-users mailing list
postgis-users at postgis.refractions.net<mailto:postgis-users at postgis.refractions.net>
http://postgis.refractions.net/mailman/listinfo/postgis-users
--
Sebastian E. Ovide
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20100720/c25e7886/attachment.html>
More information about the postgis-users
mailing list