[postgis-users] [!! SPAM] Re: may have postgis store limits for a big project?

Tue Jul 20 02:47:11 PDT 2010

Sorry yuriesky for the unintended hijack of your thread...

Both arguments are very interesting... and I am interested in both of
them...

Probably we could split the thread.

Guys, do you follow any convention in this list for splitting treads ?

On Mon, Jul 19, 2010 at 11:18 PM, salas <fsalas at geocuba.cu> wrote:

> Hi, this topic is very interesting but in my situation we need to know
> about
> the vectorial data, not storing of raster data.
> could someone tell me about this? store limits for vectorial data.
>
> yuriesky
>
> ----- Original Message ----- From: "Pierre Racine" <
> Pierre.Racine at sbf.ulaval.ca>
> To: "PostGIS Users Discussion" <postgis-users at postgis.refractions.net>
> Sent: Monday, July 19, 2010 10:25 AM
> Subject: [!! SPAM] Re: [postgis-users] may have postgis store limits for a
>
> big project?
>
>
>  >We do not know yet.. it could be even hundreds !
>>
>> WKT Raster is not really limited in size. But we still have to test on
>> datasets the size you are using.
>>
>>  each region (european country, US state, province, etc...) would have few
>>> raster (even 20) >representing different kind of information... (each
>>> pixel would be 5mx5rm)...
>>>
>>
>> With WKT Raster, you can store all those rasters as a unique tiled
>> coverage. In Oracle probably you stored them one table per raster?
>>
>>  and I need to know the value of a pixel for all the rasters in that
>>> particular coordinates...
>>>
>>
>> For this you typically use ST_Value() like this:
>>
>> SELECT ST_Value(rast, ST_Geomfromtext('Point(-78.1 58.1)', 4326))
>> FROM srtm_tiled_100x100
>> WHERE ST_Intersects(rast::geometry, ST_Geomfromtext('Point(-78.1 58.1)',
>> 4326))
>>
>> One query like this takes 16 milliseconds on my machine on a 100 pixels x
>> 100 pixels tiled coverage. There are 46800 tiles.
>>
>> Don't forget to assign a reference system to your raster when converting
>> it to .sql (-s gdal2wktraster's option) and to construct an index on the
>> table (-I option).
>>
>>  with Oracle 11g GEORaster it is possible to compress the rasters (and get
>>> lower performance of >course)... in my case the compression rate is up to
>>> 600 !!!... so the amount of data is >drastically reduced...
>>>
>>
>> WKT Raster relies on PostgreSQL for compression. I don't know the
>> compression rate.
>>
>>  nevertheless I need to be able to do a few (up to 20) queries like this
>>> at
>>> the same time in in a >fraction of second...: select pixel from raster
>>> where (x=X and y=Y). One query for each raster, >representing a different
>>> information
>>>
>>
>> If you tile and index everything properly that should not be a big deal.
>> Let us know!
>>
>>  If WKTRaster is mature enough, we can create a cluster without the need
>>> of
>>> paying a prohibitive license per CPU as for Oracle !
>>>
>>
>> If you save extra money by not having buying an Oracle license and you are
>> interested in collaborating in the development of WKT Raster, let us know
>> ;-)
>>
>> Pierre
>>
>> Sebastian
>> On Mon, Jul 19, 2010 at 5:00 PM, Pierre Racine
>> <Pierre.Racine at sbf.ulaval.ca> wrote:
>> Hi Sebastian,
>>
>> What kind of query are you planning on the raster? You know you can also
>> use WKT Raster to simply "register" filesystem rasters, creating a king of
>> geospatial catalog, without actually storing them in the database...
>>
>> How many 16GB raster do you have? Are your raster all representing the
>> same theme? Are they overlapping (or they form a regular grid)?
>>
>> We don't know of that many users cases up to now, but WKT Raster prooved
>> to be very stable working on 1 GB raster coverage (see the tutorial).
>>
>> Pierre
>>
>> From: postgis-users-bounces at postgis.refractions.net
>> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
>> Sebastian E. Ovide
>> Sent: 19 juillet 2010 11:34
>> To: PostGIS Users Discussion
>> Subject: Re: [postgis-users] may have postgis store limits for a big
>> project?
>>
>> Hi All,
>>
>> I am interesting too... I am considering PostGIS+WKTRaster instead of
>> Oracle GeoRaster... but for that can happen, Postgresql must be able of
>> managing some simple tables of 30+M (Millions) rows plus a lot of Rasters
>> of 16GB each... (16GB not compress data... so are huge images !). Each
>> query (simple look up) must response in a fraction of second....
>>
>> any successful case ?
>>
>> Thanks !
>> On Fri, Jul 16, 2010 at 1:13 AM, salas <fsalas at geocuba.cu> wrote:
>> Hello to all:
>> I am working in a project where they are managed more than 20 geoespatials
>> thematics. Each thematic one has a considerable volume of since
>> information it is of the whole country (I am speaking of a lot more than a
>> million of records). In the project we need to make (mostly) intersection
>> consultations keeping in mind literal attributes.
>> I need to know the experience of somebody in a project of this span and if
>> PostGIS would present some limitation therewith.
>>
>> regards yuriesky
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> --
>> Sebastian E. Ovide
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> --
>> Sebastian E. Ovide
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>

-- 
Sebastian E. Ovide
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20100720/eb3fe433/attachment.html>