[Pywps-dev] [ GSOC 2014 ] Database output storage idea.

Tue Mar 11 00:31:41 PDT 2014

OK


On Tue, Mar 11, 2014 at 08:18:19AM +0100, Jorge de Jesus wrote:
> Hi Jachym lets discuss the topic on the hangout then I can take some
> notes and prepare things for Naveen
> 
> I'm not so certain how the schema in DB will be done, I was thinking a
> sort of pre-defined schema/relations like the one in the istSOS
> (http://istgeo.ist.supsi.ch/software/istsos/)
> 
> Cheers
> 
> 
> 
> On 03/11/2014 12:31 AM, Jachym Cepicky wrote:
> > Hi,
> > 
> > 
> > On Tue, Mar 11, 2014 at 01:02:08AM +0530, Naveen Panwar wrote:
> >>    Respected Sir,
> >>
> >>    As you suggested in previous mail I have gone through Pywps [1]Cookbook
> >>    Documentation[1].  As there are three types of inputs and outputs are
> >>    defined in the OGC standard: LiteralData, ComplexData, BoundingBox data.
> >>    But as you mentioned we have to take care of ComplexData.
> > 
> > Not true: also literal and bbox data can be stored in database (but they are
> > much simplier task to be done thought).
> > 
> >>
> >>    Currently ComplexData data store in two manner, One with xml as GML
> >>    format, Second with binary raster data encoded in base64.
> >>    these data stored as a file in system. Now we need database for the better
> >>    access and availability.
> >>
> >>    I following questions in my mind.
> >>
> >>    1. Do we have write database module so that user can configure there own
> >>    database server by providing db URL and user credential, like standard db
> >>    connectivity ?
> > 
> > This case
> > 
> >>
> >>    2. Or we have to design our own database and provide db connection object
> >>    to user, so that they can write to the DB ?
> > 
> > User should be able to setup everything using configuration file.
> > 
> >>
> >>    3. How are we supposed to store the data in DB as both vector and raster
> >>    data present in different format ?
> > 
> > PostGIS should do this. SQLite has support for some raster formats as well
> > 
> >>    4. What kind of database support do we need ? Relational DB(PostgreSQL,
> >>    MySQL) or NOSql DB (as we have Cassandra by pycassa ) ?
> > 
> > PostgreSQL/PostGIS, SQLite for the first steps. NOSql you nemntioned at the
> > second.
> > 
> >>
> >>    5. What should be the schema of the database ? is it predefine or define
> >>    by the user ?
> > 
> > User should define, which schema schould be used.
> > 
> >>
> >>    and last but not the least when will be next hangout 13 march or 26 march
> >>    ? sorry bit confuse. 
> > 
> > sorry for the mess
> > 
> > http://www.timeanddate.com/worldclock/fixedtime.html?iso=20140313T20&ah=1
> > 
> > we are still aranging, how to define timeslots
> > 
> > J
> > 
> >>    [1] -
> >>    [2]http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> >>
> >>    On Mon, Mar 10, 2014 at 2:33 AM, Jachym Cepicky
> >>    <[3]jachym.cepicky at gmail.com> wrote:
> >>
> >>      Hi,
> >>
> >>      On Sun, Mar 09, 2014 at 05:46:09PM +0000, Mendes de Jesus, Jorge wrote:
> >>      > Hi to all
> >>      >
> >>      > @Naveen, first give a look at the WPS standard 1.0.0[1] and we have a
> >>      wiki [2]and doc online [3] (all document refer to the old 3.2 version
> >>      but still a good source of information and technical know-how
> >>      > @Jachym, correct me when wrong
> >>      >
> >>      > WPS defines one output as ComplexData, meaning a raster images,
> >>      vetorial data etc etc can be contained between the element tags
> >>      <ComplexData /> The output data can be stored in the server when the
> >>      user requests an async result, the data stored is then fetch by an URL
> >>      location indicates by a status response.
> >>      >
> >>      > So,
> >>      > 1) The data stored in the server is currently saved on the filesystem
> >>      and we need it in a DB
> >>
> >>      better say: we would like to offer option, that all generated output
> >>      data could
> >>      be stored in some database, instead of file system
> >>      > 2) URLs will have to point to some function/service/DBrequest that
> >>      will fetch the results to the user
> >>
> >>      so service will have to be written/configured (using
> >>      mapserver/geoserver/custom
> >>      something), which will do the serving
> >>      > 3) DB structure should contain metadata on the request, datatype  etc
> >>      etc etc
> >>
> >>      > 4) We are currently  working on new code and things are not 100%
> >>      stable, nor I have a clear knowledge of the code therefore I can't give
> >>      you a direct answer to were in the code this has to be hookedup (Maybe
> >>      during the next hangout 13-March I/we will check it out).Please give a
> >>      look at the pywps4.0 documentation [4]
> >>
> >>      currently, first version of file storage is implemented - just storing,
> >>      no data
> >>      exporting
> >>      [4]https://github.com/jachym/pywps-4/blob/master/pywps/inout.py#L227
> >>      (will be moved to separate file soon, or maybe package)
> >>      J
> >>
> >>      >
> >>      >
> >>      > [1] [5]http://www.opengeospatial.org/standards/wps
> >>      > [2] [6]http://wiki.rsg.pml.ac.uk/pywps/Main_Page
> >>      > [3] [7]http://pywps.wald.intevation.org/documentation/
> >>      > [4] [8]http://pywps.readthedocs.or
> >>      >
> >>      > I will try to give you more information on the following days
> >>      >
> >>      > Jorge
> >>      >
> >>      >
> >>      >
> >>      > ________________________________________
> >>      > From: Naveen Panwar [[9]panwarn09 at gmail.com]
> >>      > Sent: 07 March 2014 10:00
> >>      > To: Mendes de Jesus, Jorge; Jáchym Čepický
> >>      > Subject: [ GSOC 2014 ] Database output storage idea.
> >>      >
> >>      > Hello Sir,
> >>      >
> >>      > As we know students application starts on March 10th.
> >>      > For that I have to write proposal for the idea "Database output
> >>      storage - PostGIS and SQLite".
> >>      > So could you suggest anything in this direction. What should be in the
> >>      IDEA related to Pywps.
> >>      >
> >>      > Also could you suggest on which modules we have to add database
> >>      interaction
> >>      > in Pywps code base.
> >>      >
> >>      > --
> >>      >
> >>      > Regards,
> >>      > Naveen Panwar
> >>      > Senior Undergraduate Student
> >>      > IIIT-Hyderabad, India.
> >>      >
> >>      > [10]http://lsi.iiit.ac.in/naveen.panwar
> >>      >
> >>
> >>      --
> >>      Jachym Cepicky
> >>      URL: [11]http://les-ejk.cz
> >>      e-mail: jachym.cepicky at gmail com
> >>      PGP: [12]http://les-ejk.cz/pgp/JachymCepicky.pgp
> >>      @jachymc
> >>
> >>    --
> >>    Regards,
> >>    Naveen Panwar
> >>    Senior Undergraduate Student
> >>    IIIT-Hyderabad, India.
> >>    [13]http://lsi.iiit.ac.in/naveen.panwar
> >>
> >> References
> >>
> >>    Visible links
> >>    1. http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> >>    2. http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> >>    3. mailto:jachym.cepicky at gmail.com
> >>    4. https://github.com/jachym/pywps-4/blob/master/pywps/inout.py#L227
> >>    5. http://www.opengeospatial.org/standards/wps
> >>    6. http://wiki.rsg.pml.ac.uk/pywps/Main_Page
> >>    7. http://pywps.wald.intevation.org/documentation/
> >>    8. http://pywps.readthedocs.or/
> >>    9. mailto:panwarn09 at gmail.com
> >>   10. http://lsi.iiit.ac.in/naveen.panwar
> >>   11. http://les-ejk.cz/
> >>   12. http://les-ejk.cz/pgp/JachymCepicky.pgp
> >>   13. http://lsi.iiit.ac.in/naveen.panwar
> > 
> 
> 
> -- 
> ISRIC - World Soil Information Post: PO box 353, 6700 AJ, Wageningen,
> The Netherlands Visiting Address: Droevendaalsesteeg 3, 6708 PB
> Wageningen (Bdg. 101), Office: C.013 Office Phone: +31 (0) 317 4 83715
> Mobile Phone: +31 (0) 613 9 06950 OpenPGPKey: 0xA3D0065A
> 

-- 
Jachym Cepicky
URL: http://les-ejk.cz
e-mail: jachym.cepicky at gmail com
PGP: http://les-ejk.cz/pgp/JachymCepicky.pgp
@jachymc