[fdo-internals] New FDO RFC 50 posted - Extend FDO API to Select from Multiple Classes and Select Joins

Tue Jun 8 10:31:56 EDT 2010

Having a join will force provider to generate a join code and return all properties of those classes. Some properties might be selected only to use them in Filters and not as returned properties (user might chose to do that with "private" data which cannot be shared in a data grid however those can be used in filters)
Implementing a select against multiple classes I can chose the properties returned.

Regarding sub-select "IN()" FDO parser don't know to parse SQL, so it will not be the same as FeatId IN (2,3)

Re 2.: Adding a computed class in select list will make all properties returned by the sub-select to be returned and that's not good since I might need them fro Where clause, e.g.:
SELECT a1.FeatId, A2.Geom, A3.Name FROM A1, A2, (select V1, Name FROM Class5 WHERE COND2) AS A3 WHERE A3.V1=A2.V1 AND COND1

Thanks,
Romy.
________________________________________
From: fdo-internals-bounces at lists.osgeo.org [fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic [haris at sl-king.com]
Sent: Tuesday, June 08, 2010 10:28 AM
To: FDO Internals Mail List
Subject: Re: [fdo-internals] New FDO RFC 50 posted - Extend FDO API to Select   from Multiple Classes and Select Joins

For 3 or more classes we use more FdoJoinCriteria as in sample in RFC
"Handling Joins"
Use 2 or more FdojoinCriteria.
In sample "FdoPtr<FdoIdentifierCollection> fclasscoll = =
sel->GetFeatureClassNames();" is used to set only name of the first
class. Could be SetFeatureClassName as well.

Sub-Select doesn't need to be defined in join at all. It will be part
of filter. It is expression.
Same as I would set filter FeatId IN (2,3) or FeatId = 2.

Haris

On Tue, Jun 8, 2010 at 4:17 PM, Romica Dascalescu
<Romica.Dascalescu at autodesk.com> wrote:
> Hi Haris,
>
> A few comments below:
>
> Re: 1. When we create Joins we can use SetFeatureClassName and FdoJoinCriteria
>
> Re: 2. How this will work? since we have samples like: SELECT * FROM point WHERE FeatId IN (SELECT pt.FeatId FROM pt WHERE COND) and the FdoComputedClassIdentifier is not in the select list.
>
> Removing GetFeatureClassNames wow we can select from 3 classes? e.g.:
> SELECT a1.FeatId, A2.Geom, A3.Name from A1, A2, A3 [ - optional WHERE COND]
>
> Each SQL statement have columns and the providers will generate a feature class definition same as SQLite is doing right now.
>
> Thanks,
> Romy.
>
> ________________________________________
> From: fdo-internals-bounces at lists.osgeo.org [fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic [haris at sl-king.com]
> Sent: Tuesday, June 08, 2010 10:04 AM
> To: FDO Internals Mail List
> Subject: Re: [fdo-internals] New FDO RFC 50 posted - Extend FDO API to Select   from Multiple Classes and Select Joins
>
> I will try to summarize couple of my suggestions :
>
> 1. To not have SetFeatureClassName and GetFeatureClassNames to set classes
>    Just use SetFeatureClassName in base select and FdoJoinCriteria to
> define joins.
>
> 2. Use sub-select as newly introduced FdoComputedClassIdentifier class
> (change name) as property value too in property list.
>    I would also suggest to investigate / test use of the class in
> insert and updates but as value (not as filter).
>    I have feeling it may just work.
>
> Two for now :)
>
> BTW, regarding feature class which is supposed to be generated after
> executing select command  we had issues before and we discuss it. Also
> from text in this RFC it seems that it is very unclear what resulting
> class should look like in many cases. It is pretty unclear that It
> seems almost equally good to leave class def empty.
> I am not suggesting to change it now. Just pointing out to the issue
> which has repeated itself few times.
>
> Thank you,
> Haris
>
> On Tue, Jun 8, 2010 at 2:29 PM, Orest Halustchak
> <orest.halustchak at autodesk.com> wrote:
>> Hi,
>>
>> Thanks Haris for the comments. Some more feedback below ...
>>
>> Orest.
>>
>> -----Original Message-----
>> From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
>> Sent: Monday, June 07, 2010 7:24 PM
>> To: FDO Internals Mail List
>> Subject: Re: [fdo-internals] New FDO RFC 50 posted - Extend FDO API to Select from Multiple Classes and Select Joins
>>
>> I have feeling that we perhaps misunderstood on few questions
>> (basically not questions but more call to discuss it more then it is
>> written in RFC already )
>> I don't know if i am able to add comment into those inlines but will try
>>
>> Haris
>>
>> On Tue, Jun 8, 2010 at 12:44 AM, Romica Dascalescu
>> <Romica.Dascalescu at autodesk.com> wrote:
>>> Hi Haris,
>>>
>>> Thank you for your comments Haris. Below ([RD]) you can find a few answers for your questions.
>>>
>>> Regards,
>>> Romy.
>>> ________________________________________
>>> From: fdo-internals-bounces at lists.osgeo.org [fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic [haris at sl-king.com]
>>> Sent: Monday, June 07, 2010 4:40 PM
>>> To: FDO Internals Mail List
>>> Subject: Re: [fdo-internals] New FDO RFC 50 posted - Extend FDO API to Select   from Multiple Classes and Select Joins
>>>
>>> Hi ,
>>>
>>> I am glad that you are putting effort into this RFC. This RFC is
>>> certainly needed.
>>> It is also quite big topic and enhancement to FDO.
>>> I read it couple of times and have few question / remarks. I am pretty
>>> sure there will be more of it.
>>>
>>> 1. I prefer abstract functions. When I recompile providers for new
>>> version it helps me remind there is something new (at least).
>>> [RD] This involves that we have to 'fix' all providers to add a (the same) default implementation for these two new methods. Having a default implementation like AddRef & Release will lead to not modify providers which do not support such select.
>> [HK] with new FDO version providers will be rebuild and couple of
>> abstract functions definition doesn't heart and I can see error in
>> compile as motivation to implement it more ... but that is not so
>> important for this rfc
>>>
>>> 2. There had to be some cons on proposed solution :)
>>> [RD] I thought about cons but I could not find one. Did you have some other than above statement?
>>>
>>> 3. SetFeatureClassName of FeatureCommand will be left unused and that
>>> sounds like a bit confusing API
>>> [RD] True, but we did not find a nice way to be able to set more feature class names. In order to make that possible we need a collection, one solution would be to "convert"/remove SetFeatureClassName and use only GetFeatureClassNames, but this will generate a huge change in the providers side, or add a separated command...
>> [HK] perhaps  GetFeatureClassNames is not necessary at all
>>
>> [OH] GetFeatureClassNames is needed to be able to specify a collection of class names to use. But, it would only be used for providers that support the new functionality. Also, we need to maintain backwards compatibility, so we end up with both FeatureClassName and FeatureClassNames. Is there an alternative idea that doesn't break backwards compatibility (for existing applications that are using the api)?
>>
>>>
>>> 4. In "Selecting from Multiple Classes" example when I saw example and
>>> read that it was mentioning "aliases"
>>>    My first taught is that FdoComputedIdentifier is used to define
>>> aliases for names used in filter not to be used as actual alias in
>>> generated SQL statement. Am I right here ? If not I see lot of
>>> problems in user defining actual alias for SQL statement.
>>> 5. Why using alias at all ? Shouldn't be name of the class way to go ?
>>> [RD] Yes and no. The idea is we can get rid of aliases for classes, however for properties is good to keep them because in case we select from two feature classes and those classes have a property named the same (e.g. FeatId) we will have a name conflict (FDO API do not allow two properties with the same name in a property collection). This way we offer to the user a way to rename properties to avoid such cases. In the end alias is an expression :)
>> [HK] I still don't understand , why use aliases ? why not use
>> class.property or schema.class.property in filter ? If using aliases
>> then user need to use what in property list or order list
>> (class.property or alias.property )?
>>
>> [OH] Aliases allow you to specify self-joins in cases where classes have references back to themselves. If I include the same class name twice in the list, I need aliases to distinguish the first one from the second one.
>>
>>>
>>> 6. How we select which properties to be returned ? All properties from
>>> all class names from GetFeatureClassNames by deafult ?
>>> [RD] Same as now: in case we do not specify any property all properties are returned (select * from...), in case caller adds some properties in collection GetPropertyNames() then only  those properties will be selected.
>>>
>>> 7. How we name properties to be returned (again alias/class name ?) ?
>>> Same question for ordering properties ?
>>> [RD] Here caller needs to be a bit careful: in case a property exist in both classes then it must be provided with class name in the front in filter or select collection. In the returned feature class the original class name is removed and in case an alias is provided we return alias name and not property name. I think here is a provider implementation decision, e.g. in case we have a name conflict provider either can throw an exception or generate a name on the fly for that property.
>>>
>>> 8. Why we have two ways for "Selecting from Multiple Classes" and
>>> "Handling Joins" ? Could be same interface ?
>>> [RD] There are 3 examples "Selecting from Multiple Classes", "Handling Joins" and "Handle Sub-Selects", we do not have different interfaces for them. I just provided examples what we can/want to do with proposed changes.
>> [HK] Yes there are 3 in RFC. I am saying why not use FdoJoinCriteria
>> only for first two cases/samples? In that case  GetFeatureClassNames
>> doesn't need to be added into  base class.
>>>
>>> 9. I believe example in "Handling Joins" is missing implemented filter
>>> "WHERE p.FeatId >= 0;"
>>> [RD] I will update the example.
>>>
>>> I like sub-queries.
>>>
>>> 10. I don't like name of new expression type "ComputedClassIdentifier" as it can be treated as property too
>>> [RD] This is a sub-select and can behave like a source to select so we can named computed. Imagine a select like:
>>> "SELECT * FROM cls1, (SELECT * FROM C1, C2 WHERE Filter1) AS cls2 WHERE cls1.FeatId = cls2.FeatId;
>>> The computed class is actually cls2 having the definition "SELECT * FROM C1, C2 WHERE Filter1"
>> [HK] Sub-Query is also "SELECT Property FROM C1, C2 WHERE Filter1" and
>> it is value. In such case can be used as value in select part of
>> statement or insert, update as value.
>>
>> [OH] We could call it just "ComputedIdentifier". So, your idea is to allow that computed identifier to be added to the select property list?
>>
>>>
>>> 11. Why sub-query FdoISelect is not using standard SetFeatureClassName in example ? It is non-join select.
>>> [RD] In the example we just wanted to show how we can use the new methods, in case caller has only one class to select from then he can use SetFeatureClassName, however in case we have a join select or a select from multiple classes then we need to use the new methods.
>> [HK] But Sub-query is simple query. Recommended would be to use
>> "standard" SetFeatureClassName? We come again to little confusion
>> about where we can set class names.
>>>
>>> 12. Haven't checked at class defs but this sub-queries should be accepted in property list to be returned as well as to be accepted as values for update and insert
>>> [RD] We could improve update, insert and delete (by adding this collection) but we said we can keep it as a future enhancement (limited time to implement this).
>> [HK] If sub-query is  treated as property value  . I have feeling that
>> it would be very simple to use it as part of insert or update value.
>> Actually, I could imagine that it could just work right now as it is.
>>
>> [OH] There are a number of commands that use filters that possibly could be improved to support sub-selects. As Romy said, it seemed that we could leave those for another RFC rather than trying to do it all at once.
>>
>>>
>>> Perhaps a bit off this RFC but it was mentioned in the RFC ( in a bit scary way).
>>> I think that reader shouldn't bother to return FDO class. Reader should be able to return count, name, type of properties,... in it but not to generate some fictive FDO class.
>>> [RD] In many cases a feature reader will be more useful than a normal reader. To return count in many cases would mean to execute the select and get the count: e.g. select COUNT(*) from (our select statement). This approach will not be fast since aggregate functions are not too fast from some providers. Aggregate select can be used in this case, if count is needed.
>> [HK} With count I mean number of properties in reader.
>>
>> [OH] We have separate commands, Select versus SelectAggregates, for returning features and just non-feature data. So, to keep the Select command as returning features, even computed features, it should return a feature reader.
>>
>>>
>>> If application needs it, application will generate FDO class for it's use ( it can be even utility in fdo itself).
>>
>> [OH] I agree that would be a useful utility, but I think it's a separate topic. The Select command already returns a feature reader and I don't think we should change that.
>>
>>>
>>>
>>> And,  thank you for working on this RFC.
>>>
>>> Haris
>>>
>>> On Fri, Jun 4, 2010 at 4:05 PM, Orest Halustchak
>>> <orest.halustchak at autodesk.com> wrote:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> Please have a look at a new FDO RFC 50 that proposes to extend the FDO api
>>>> to support the ability to use multiple classes in a select and in a filter.
>>>> This would provide the ability the express joins at the fdo level.
>>>>
>>>>
>>>>
>>>> http://trac.osgeo.org/fdo/wiki/FDORfc50
>>>>
>>>>
>>>>
>>>> It is a common capability that users need, the ability to join together two
>>>> or more classes for a result to support analysis, rendering, or other uses.
>>>> Until now, with FDO, this was done at a higher level such as the client-side
>>>> join capabilities within MapGuide. While that capability is useful, it does
>>>> not perform well and is error prone. This enhancement provides us a clear
>>>> way to push that capability down to providers that support it such that they
>>>> can translate the join into native server expressions. In general, I think
>>>> it adds more power and flexibility to the FDO api.
>>>>
>>>>
>>>>
>>>> As usual, this includes capability functions so that applications can
>>>> discover if a provider supports this. We don't expect all providers to
>>>> support this, mainly the ones that pass through fdo queries to sql.
>>>>
>>>>
>>>>
>>>> I hope you like it. J
>>>>
>>>>
>>>>
>>>> Thanks very much to Romy Dascalescu for putting this together and trying out
>>>> various alternatives.
>>>>
>>>>
>>>>
>>>> Please provide your comments and suggestions.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Orest.
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> fdo-internals mailing list
>>>> fdo-internals at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>>>>
>>>>
>>> _______________________________________________
>>> fdo-internals mailing list
>>> fdo-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/fdo-internals_______________________________________________
>>> fdo-internals mailing list
>>> fdo-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>>>
>> _______________________________________________
>> fdo-internals mailing list
>> fdo-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>> _______________________________________________
>> fdo-internals mailing list
>> fdo-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>>
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals_______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals