[Pywps-dev] [ GSOC 2014 ] Database output storage idea.
Naveen Panwar
panwarn09 at gmail.com
Wed Mar 12 09:36:07 PDT 2014
Respected Sir,
Here is the summary of our last conversations
1. We have to enable users to configure their own database connectivity, by
providing appropriate information in the config file.
2. They will be using PostgreSQL, and SQLite according to their need.
3. In later phase of project we can provide support for NoSql(pycass).
4. Schema for the data we have to generate on the fly based on user demand.
Also Google has started application for the Students and Mentors for
GSOC-2014. I think, we should put a draft of our proposal on melange, So
that other people/mentors in OSGeo can get to know about idea.
On Tue, Mar 11, 2014 at 1:01 PM, Jachym Cepicky <jachym.cepicky at gmail.com>wrote:
> OK
>
>
> On Tue, Mar 11, 2014 at 08:18:19AM +0100, Jorge de Jesus wrote:
> > Hi Jachym lets discuss the topic on the hangout then I can take some
> > notes and prepare things for Naveen
> >
> > I'm not so certain how the schema in DB will be done, I was thinking a
> > sort of pre-defined schema/relations like the one in the istSOS
> > (http://istgeo.ist.supsi.ch/software/istsos/)
> >
> > Cheers
> >
> >
> >
> > On 03/11/2014 12:31 AM, Jachym Cepicky wrote:
> > > Hi,
> > >
> > >
> > > On Tue, Mar 11, 2014 at 01:02:08AM +0530, Naveen Panwar wrote:
> > >> Respected Sir,
> > >>
> > >> As you suggested in previous mail I have gone through Pywps
> [1]Cookbook
> > >> Documentation[1]. As there are three types of inputs and outputs
> are
> > >> defined in the OGC standard: LiteralData, ComplexData, BoundingBox
> data.
> > >> But as you mentioned we have to take care of ComplexData.
> > >
> > > Not true: also literal and bbox data can be stored in database (but
> they are
> > > much simplier task to be done thought).
> > >
> > >>
> > >> Currently ComplexData data store in two manner, One with xml as GML
> > >> format, Second with binary raster data encoded in base64.
> > >> these data stored as a file in system. Now we need database for
> the better
> > >> access and availability.
> > >>
> > >> I following questions in my mind.
> > >>
> > >> 1. Do we have write database module so that user can configure
> there own
> > >> database server by providing db URL and user credential, like
> standard db
> > >> connectivity ?
> > >
> > > This case
> > >
> > >>
> > >> 2. Or we have to design our own database and provide db connection
> object
> > >> to user, so that they can write to the DB ?
> > >
> > > User should be able to setup everything using configuration file.
> > >
> > >>
> > >> 3. How are we supposed to store the data in DB as both vector and
> raster
> > >> data present in different format ?
> > >
> > > PostGIS should do this. SQLite has support for some raster formats as
> well
> > >
> > >> 4. What kind of database support do we need ? Relational
> DB(PostgreSQL,
> > >> MySQL) or NOSql DB (as we have Cassandra by pycassa ) ?
> > >
> > > PostgreSQL/PostGIS, SQLite for the first steps. NOSql you nemntioned
> at the
> > > second.
> > >
> > >>
> > >> 5. What should be the schema of the database ? is it predefine or
> define
> > >> by the user ?
> > >
> > > User should define, which schema schould be used.
> > >
> > >>
> > >> and last but not the least when will be next hangout 13 march or
> 26 march
> > >> ? sorry bit confuse.
> > >
> > > sorry for the mess
> > >
> > >
> http://www.timeanddate.com/worldclock/fixedtime.html?iso=20140313T20&ah=1
> > >
> > > we are still aranging, how to define timeslots
> > >
> > > J
> > >
> > >> [1] -
> > >> [2]
> http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> > >>
> > >> On Mon, Mar 10, 2014 at 2:33 AM, Jachym Cepicky
> > >> <[3]jachym.cepicky at gmail.com> wrote:
> > >>
> > >> Hi,
> > >>
> > >> On Sun, Mar 09, 2014 at 05:46:09PM +0000, Mendes de Jesus, Jorge
> wrote:
> > >> > Hi to all
> > >> >
> > >> > @Naveen, first give a look at the WPS standard 1.0.0[1] and we
> have a
> > >> wiki [2]and doc online [3] (all document refer to the old 3.2
> version
> > >> but still a good source of information and technical know-how
> > >> > @Jachym, correct me when wrong
> > >> >
> > >> > WPS defines one output as ComplexData, meaning a raster images,
> > >> vetorial data etc etc can be contained between the element tags
> > >> <ComplexData /> The output data can be stored in the server when
> the
> > >> user requests an async result, the data stored is then fetch by
> an URL
> > >> location indicates by a status response.
> > >> >
> > >> > So,
> > >> > 1) The data stored in the server is currently saved on the
> filesystem
> > >> and we need it in a DB
> > >>
> > >> better say: we would like to offer option, that all generated
> output
> > >> data could
> > >> be stored in some database, instead of file system
> > >> > 2) URLs will have to point to some function/service/DBrequest
> that
> > >> will fetch the results to the user
> > >>
> > >> so service will have to be written/configured (using
> > >> mapserver/geoserver/custom
> > >> something), which will do the serving
> > >> > 3) DB structure should contain metadata on the request,
> datatype etc
> > >> etc etc
> > >>
> > >> > 4) We are currently working on new code and things are not
> 100%
> > >> stable, nor I have a clear knowledge of the code therefore I
> can't give
> > >> you a direct answer to were in the code this has to be hookedup
> (Maybe
> > >> during the next hangout 13-March I/we will check it out).Please
> give a
> > >> look at the pywps4.0 documentation [4]
> > >>
> > >> currently, first version of file storage is implemented - just
> storing,
> > >> no data
> > >> exporting
> > >> [4]
> https://github.com/jachym/pywps-4/blob/master/pywps/inout.py#L227
> > >> (will be moved to separate file soon, or maybe package)
> > >> J
> > >>
> > >> >
> > >> >
> > >> > [1] [5]http://www.opengeospatial.org/standards/wps
> > >> > [2] [6]http://wiki.rsg.pml.ac.uk/pywps/Main_Page
> > >> > [3] [7]http://pywps.wald.intevation.org/documentation/
> > >> > [4] [8]http://pywps.readthedocs.or
> > >> >
> > >> > I will try to give you more information on the following days
> > >> >
> > >> > Jorge
> > >> >
> > >> >
> > >> >
> > >> > ________________________________________
> > >> > From: Naveen Panwar [[9]panwarn09 at gmail.com]
> > >> > Sent: 07 March 2014 10:00
> > >> > To: Mendes de Jesus, Jorge; Jáchym Čepický
> > >> > Subject: [ GSOC 2014 ] Database output storage idea.
> > >> >
> > >> > Hello Sir,
> > >> >
> > >> > As we know students application starts on March 10th.
> > >> > For that I have to write proposal for the idea "Database output
> > >> storage - PostGIS and SQLite".
> > >> > So could you suggest anything in this direction. What should
> be in the
> > >> IDEA related to Pywps.
> > >> >
> > >> > Also could you suggest on which modules we have to add database
> > >> interaction
> > >> > in Pywps code base.
> > >> >
> > >> > --
> > >> >
> > >> > Regards,
> > >> > Naveen Panwar
> > >> > Senior Undergraduate Student
> > >> > IIIT-Hyderabad, India.
> > >> >
> > >> > [10]http://lsi.iiit.ac.in/naveen.panwar
> > >> >
> > >>
> > >> --
> > >> Jachym Cepicky
> > >> URL: [11]http://les-ejk.cz
> > >> e-mail: jachym.cepicky at gmail com
> > >> PGP: [12]http://les-ejk.cz/pgp/JachymCepicky.pgp
> > >> @jachymc
> > >>
> > >> --
> > >> Regards,
> > >> Naveen Panwar
> > >> Senior Undergraduate Student
> > >> IIIT-Hyderabad, India.
> > >> [13]http://lsi.iiit.ac.in/naveen.panwar
> > >>
> > >> References
> > >>
> > >> Visible links
> > >> 1.
> http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> > >> 2.
> http://netmar.nersc.no/sites/netmar.nersc.no/files/D7.7_WPS_Cookbook_r1_20111229.pdf
> > >> 3. mailto:jachym.cepicky at gmail.com
> > >> 4.
> https://github.com/jachym/pywps-4/blob/master/pywps/inout.py#L227
> > >> 5. http://www.opengeospatial.org/standards/wps
> > >> 6. http://wiki.rsg.pml.ac.uk/pywps/Main_Page
> > >> 7. http://pywps.wald.intevation.org/documentation/
> > >> 8. http://pywps.readthedocs.or/
> > >> 9. mailto:panwarn09 at gmail.com
> > >> 10. http://lsi.iiit.ac.in/naveen.panwar
> > >> 11. http://les-ejk.cz/
> > >> 12. http://les-ejk.cz/pgp/JachymCepicky.pgp
> > >> 13. http://lsi.iiit.ac.in/naveen.panwar
> > >
> >
> >
> > --
> > ISRIC - World Soil Information Post: PO box 353, 6700 AJ, Wageningen,
> > The Netherlands Visiting Address: Droevendaalsesteeg 3, 6708 PB
> > Wageningen (Bdg. 101), Office: C.013 Office Phone: +31 (0) 317 4 83715
> > Mobile Phone: +31 (0) 613 9 06950 OpenPGPKey: 0xA3D0065A
> >
>
> --
> Jachym Cepicky
> URL: http://les-ejk.cz
> e-mail: jachym.cepicky at gmail com
> PGP: http://les-ejk.cz/pgp/JachymCepicky.pgp
> @jachymc
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/pywps-dev
--
Regards,
Naveen Panwar
Senior Undergraduate Student
IIIT-Hyderabad, India.
http://lsi.iiit.ac.in/naveen.panwar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20140312/9b7c102f/attachment.html>
More information about the pywps-dev
mailing list