[fdo-internals] Vote: FDO RFC 16 - FDO Provider for SQLite

Tue Apr 1 12:43:32 EDT 2008

Looks like my previous email got eaten by a misbehaving Exchange
server... anyway:

For me the most important part of this RFC is the creation of a format
that is single file, open, and can be easily shared between open source
applications.  This includes things like being able to take advantage of
standard SQL queries, etc.

The current SDF format is not documented, and it is my (possibly
mistaken) understanding that it uses an FDO structure for metadata
rather than the OGC SFSQL metadata tables.  Maybe I'm confusing provider
with format here, but if the SDF provider is reading non-SDF files... ?

If the SDF provider was re-purposed for this non-SDF format, would it
still be able to handle native SQLite queries?  Would it allow
SelectOrdering and SelectGrouping? (OK, that last one was gratuitous -
but it's a pain point for me)

Regardless, the important point for me here is that Traian is willing to
write this open provider (already has) and it fills an important need.
I don't want to see this opportunity go away (Traian loses interest,
nobody else has resources to modify the SDF provider).

Jason

-----Original Message-----
From: Badreddine Karoui
Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for SQLite

Up until now, the SDF file format is the defacto FDO file format. I have
2 questions: 1) does Fdo need an another file format? And 2) if Fdo
needs an other file format, then what this format brings new that the
upgraded SDF will not?

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian
Stanev
Sent: Tuesday, April 01, 2008 12:29 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for SQLite

SDF would end up implementing its own row encoder/decoder for SQLite.
Why not use the one that SQLite itself has?
SDF would end up using its own query engine. Why not use the one that
SQLite itself has (SQL)?
SDF has a persisted R-Tree.

Those are design decisions that are different from I have in mind.

Besides, I don't see what the argument is all about. We can have both
the SDF provider and my provider support the same format.

Traian

> -----Original Message-----
> From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> bounces at lists.osgeo.org] On Behalf Of Badreddine Karoui
> Sent: Tuesday, April 01, 2008 12:20 PM
> To: FDO Internals Mail List
> Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
SQLite
>
> Only the existing binary encoder/decoder is maintained for backward
> compatibity. A new binary encoder/decoder will be created to write
> SQLite compliant data. I wouldn't call that 2 providers in one DLL.
> The SDF provider is at best 3 years old and can hardly be called
> "legacy". I'm sure any performance used in the new code can apply to
> the SDF provider, wouldn't?
>
> Badreddine
>
> -----Original Message-----
> From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> Sent: Tuesday, April 01, 2008 12:09 PM
> To: FDO Internals Mail List
> Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
SQLite
>
>
> It would be like having two providers in one DLL. There would be two
> code paths for everything, and I don't think all that much code would
> be shared between the two, apart from query execution.
>
> The SQLite provider (which I have already written), is right now
> significantly faster than SDF in use cases that MapGuide cares about,
> even with the virtual machine in the way. For BBOX spatial query on
> several data sets I tried, it is at least 10 times faster than SDF. I
> think that if the SDF codebase is used as a starting point, this would
> not have been the case, due to all the legacy code that is in SDF.
>
>
>
> Traian
>
>
>
>
> > -----Original Message-----
> > From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> > bounces at lists.osgeo.org] On Behalf Of Badreddine Karoui
> > Sent: Tuesday, April 01, 2008 11:56 AM
> > To: FDO Internals Mail List
> > Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
> SQLite
> >
> > Regardless of the terminology; these two providers persist their
data
> > to an SQLite file. The idea would be to upgrade the SDF provider to
> > write SQLite complaint data blocks that can be consumed by other
> SQLite
> > tools/applications. At the same time the SDF provider maintains its
> > backward compatibility to its current format. We get a seemless
> > transition to the new format and stability since the bulk of the SDF
> > provider code will be re-used. Also, since the SDF will continue to
> > bypass the SQLite virtual machine, the performance will not be
> degraded
> > at least not by much.
> >
> > Badreddine
> >
> > -----Original Message-----
> > From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> > bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> > Sent: Tuesday, April 01, 2008 11:44 AM
> > To: FDO Internals Mail List
> > Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
> SQLite
> >
> >
> > The RFC does not propose "enhancements". It proposes a new format.
> >
> >
> > Traian
> >
> >
> >
> > > -----Original Message-----
> > > From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> > > bounces at lists.osgeo.org] On Behalf Of Greg Boone
> > > Sent: Tuesday, April 01, 2008 11:32 AM
> > > To: FDO Internals Mail List
> > > Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
> > SQLite
> > >
> > > Maybe this question comes a bit too late, but why can't the
> > > enhancements proposed in the RFC be made in the existing SDF
> provider
> > > as opposed to building a whole new provider?
> > >
> > > Greg
> > >
> > > -----Original Message-----
> > > From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-
> > > bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
> > > Sent: Monday, March 31, 2008 5:29 PM
> > > To: FDO Internals Mail List
> > > Subject: RE: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
> > SQLite
> > >
> > > +1
> > > Haris
> > >
> > > -----Original Message-----
> > > From: fdo-internals-bounces at lists.osgeo.org
> > > [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Jason
> > Birch
> > > Sent: Monday, March 31, 2008 7:51 PM
> > > To: FDO Internals Mail List
> > > Subject: [fdo-internals] Vote: FDO RFC 16 - FDO Provider for
SQLite
> > >
> > > Given the modifications to the RFC to address questions raised on
> > this
> > > list, and a lack of further discussion I hereby move to approve
FDO
> > RFC
> > > 16 - FDO Provider for SQLite.
> > >
> > > https://trac.osgeo.org/fdo/wiki/FDORfc16
> > >
> > > +1 from me
> > >
> > > Jason
> > >
> > > -----Original Message-----
> > > From: Traian Stanev
> > > Subject: RE: [fdo-internals] FDO RFC 16 - FDO Provider for SQLite
> > >
> > > I think it has enough stuff in it to vote on.
> > >
> > > _______________________________________________
> > > fdo-internals mailing list
> > > fdo-internals at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/fdo-internals
> > >
> > > No virus found in this incoming message.
> > > Checked by AVG.
> > > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date:
> > > 3/30/2008 12:32 PM
> > >
> > > _______________________________________________
> > > fdo-internals mailing list
> > > fdo-internals at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/fdo-internals
> > > _______________________________________________
> > > fdo-internals mailing list
> > > fdo-internals at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/fdo-internals
> > _______________________________________________
> > fdo-internals mailing list
> > fdo-internals at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/fdo-internals
> > _______________________________________________
> > fdo-internals mailing list
> > fdo-internals at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/fdo-internals
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals