[postgis-users] MultiDimensional Indexing / $$$

[Paul Ramsey]

Multidimensional indexing sounds silly and useless, but could be hugely
useful for all sorts of "conventional" datasets as they get really
large. As noted in an earlier post, TIGER is 47 million rows strong.
Suppose you want to get just the highways out of TIGER, for an arbitrary
window. Well, either you use the attribute index to get *every* highway
out, then sequentially scan for your window, or you use the spatial
index to get *every* road out, then sequentially scan for your highways.
With a multidimensional index, you could add a "third dimension" to your
features based on their "importance level", basically creating a "mylar
coordinate", so that a query could ask for all features within a spatial
area and also within a certain importance level. By making the 2d
features synthetically 3d and then indexing on that 3rd dimension, a
whole new method of doing scale-dependant access on huge datasets is
opened up.

Someone please fund this! It'll get done eventually, I am increasingly
convinced it is direly necessary for all kinds of useful apps, but on my
current funding appropriation schedule (based on benevolant Canadian
government agencies), it is likely that work would not even *commence*
for at least another 6 months.

On the other funding side, I have to remind everyone on the list that
both ourselves (Refractions Research) and Vivid Solutions have put our
own *cash money* on the table to fund the UVic project porting
(geos.refractions.net) the JTS topological operators to C++ for use in
PostGIS. Specialized open source projects like PostGIS can only compete
feature-wise with commercial products if everyone who is using these
things pitches in to push the project forward. The beauty of open-source
is that if everyone contributes, the individual level of contribution
will probably not even come close to the simple licencing costs of a
proprietary alternative. (Because we do not spend 25% of revenue on
marketing, and we do not have 35% profit margins.)

I pitch this missive particularly at people in organizations with
software budgets. If you want your choice of an open-source solution to
"look good" in the long term, you have to help keep the project moving
forward in terms of capabilities relative to the proprietary options you
did not choose. You can do that by hiring your own people to work on the
project, or you can do it by funding the existing project team to do
enhancements. People may have noticed that compared to the glory days of
   2001 (when we went from Zero -> OracleSpatial8i -> Beyond in
capabilities in 6 months), PostGIS development is running a little slow
-- that is because we have paying non-PostGIS work taking our staff
time. We like doing PostGIS work *more*, but we like meeting payroll
*even more than that*.

If you would like to be kept aware of opportunities help out with
project funding, please send me an email. No commitment is necessary,
this just means that if I write up a proposal for multidimensional
indexing (for example), you will be kept in the loop and have the
opportunity to be part of a joint funding group if you wish.

This has been your bi-annual harangue, I now return you to reguarly
schedule technical discussions.

P.

-- 
        __
       /
       | Paul Ramsey
       | Refractions Research
       | Email: pramsey at refractions.net
       | Phone: (250) 885-0632
       \_